High scalability for web apps

Tod Hoff gave some beautiful insights to scalability on his analysis of the Mailinator architecuture.

Lessons Learned

  • Perfection is a trap. How many systems are made much more complicated by the drive to be 100% everything. If you’ve been in those meetings you know what they are like. Oh, we can’t do it this way or that way because there’s .01% chance of something going wrong. Instead ask: how imperfect can you be and be good enough?
  • What you throw out is as important as what you keep in. We have many preconceptions of how to design systems. We make take for granted that you need to scale-out, you need to have email accessible days later, and that you must provide
    private accounts for everyone. But you really need these things? What can you toss?
  • Know the purpose of your system and design accordingly. Being everything to everyone means you are nothing to nobody. Keeping emails for a short period of time, allowing some SPAM to get through, and accepting less than 100% uptime create a strong vision for the system that help drive the design in all areas. You would only build your own SMTP server if you had a very strong idea of what your system was about and what you needed. I know this would have never occurred to me as an idea. I would have added more hardware.
  • Fail fast for the common case before committing resources. A high percentage of email is rejected so it makes sense to reject it as early as possible in the stack to minimize resources to accomplish the task. Figure out how to short circuit frequently failed items as fast as possible. This is important and often over looked scaling strategy.
  • Efficiency often means build it yourself. Off the shelf tools tend to do the whole job. If you only need part of the job done you may be able to write a custom component that runs much faster.
  • Adaptively forget. A little failure is OK. All the blocked IP addresses don’t need to be remembered forever. Let the block decisions build up from local data rather than global state. This is powerfully simple and robust architecture.
  • Java doesn’t have to be slow. Enough said.
  • Avoid the disk. Many applications need to hit the disk, but the disk is always a bottleneck. Can you design around the disk using other creative strategies?
  • Constrain resource usage. Put in constraints, like inbox size, that will keep your system for spiking uncontrollably. Unconstrained resource usage must be avoided with limited resources.
  • Compress data. Compression can be a major win
    when trying to conserve RAM. I’ve seen memory usage drop by more than half when using compression with very little overhead. If you are communicating locally, just have the client encode the data and keep it encoded. Build APIs to access the data without have to decode the full message.
  • Use fixed size resource pools to handle load. Many applications don’t control resource usage, like memory, and they crash when too much is used. To create a really robust system fix your resources and drop work when those resources are full. You can age resources, give priority access, give fair access, or use any other logic to arbitrate resource access, but because the resource will be limited, you will stay up under load.
  • If you don’t keep data it can’t be subpoenaed. Because Mailinator doesn’t store email or logs on disk noting can be subpoenaed.
  • Use what you know. We’ve seen this lesson a few times. Paul knew Java better than anything else, so he used it, made it work, and he got the job got done.
  • Find your own Mailinators. Sure, Mailinator is a small system. In a large system it would just be a small feature, but your system is composed of many Mailinator sized projects. What if you developed some of
    those like Mailinator?
  • KISS exists, though it’s rare. Keeping it simple is always talked about, but rarely are we shown real examples. It’s mostly just your way is complex and my way is simple because it’s my way. Mailinator is a good example of simple design.
  • Robustness is a function of architecture. To create a design that efficiently uses memory and survives massive spam attacks required an architectural approach that looked at the entire stack.
  • An Africa based entrepreneur in the pursuit of opportunities without regard to resources currently controlled striving to build services that have real-world value for my beloved continent and beyond while having fun along the way.

    Share this: