I was having some long load time problems, and noticed that when I was in Eyewire and doing other things on the internet at the same time, my connection would start crapping out at the same time as eyewire. I did some investigation and found that the 1024-connection limit on my router was being reached, and almost all of the connections were showing up as
18.4.45.11 TIME_WAIT – brainiac.mit.edu.
I of course upped the connection limit to 4096, and eyewire (and everything else) is running better now. I still wanted to know why this was happening.
I did a little digging, and found that each HTTP connection that the eyewire app makes to the server is being closed when the response comes back(Connection: Close header), and that all of the connections were being proxied (I assume to some backend server pool).
Eyewire isn’t making use of HTTP pipelining at all, which is probably one of the reasons why some people are having long load time problems. Windows XP systems I imagine are having a really bad time of it due to the maximum incomplete connection limit of 10 connections.
Beyond that, there’s even something strange going on with the FIN/FIN,ACK/ACK sequencing with the connection termination. I’m not entirely sure what is going on there, but I’d look into if the connection is being gracefully terminated on the server side.
I’d also look into using a different proxy solution for the front end – one that responds nicely to the Connection: Keep-alive header and actually allows the pipelining features included in modern browsers to work properly. The perks from that would include:
- No more bad TCP behavior on intervening routers
- Lower bandwidth usage (less TCP/HTTPS overhead – about 6kb per connection * thousands per play)
- Lower latency – No need to SYN/SYN,ACK/ACK and exchange SSL keys for every single connection
- Happier users – all of the above
Just my $.02.