A bit of a plug for A10 Networks here since we’re still very pleased after having them in production for  a little over a two months. Not only is their thermal tolerance top notch (as tested from last weekends data center meltdown) the feature set is pretty damn impressive for the price point too.

One of the features we make use of to help reduce every last little bit of latency possible is connection reuse. What this does is cause the load balancer to establish persistent connections to your servers removing the second setup cost of any new inbound connection.  This actually reduces total round trip time for a connection-request-response by a few milliseconds. This doesn’t seem like much but when you have a ton of small requests that need to be made across a few domains it tends to add up.

The additional benefit if this is that it tends to reduce your overall connection count across your cluster since the devices are amalgamating external potentially spotty keep alive connections into a smaller set of persistent connections on the internal end. For instance, we may see on the public facing side around 150 concurrent connections, however on the internal side we’re only making use of 47, so it ends up helping you scale as well.

Now the negatives to this setup, particularly if your load balancers sit as your back end servers gateway is that you need to a have a NAT policy for traffic between the load balancer and your servers; As such all requests are going to appear to be coming from a single NATed IP. Now this isn’t really too much of an issue if your application makes use of the X-CLIENT-IP or X-FORWARDED-FOR headers that get injected anyway but if they don’t then you’ll probably get annoyed by this if you happen to use source IP information at all.

Our applications are aware of the X-CLIENT-IP and X-FOWARDED-FOR headers so this was a breeze for us to start using and our servers are happier for it.