A number of developers have reported abandoned connection issues on the Streaming API starting, perhaps, about two weeks ago. The symptoms include a long-established TCP connection to stream.twitter.com going quiet, with the connection mysteriously held open for perhaps hours afterward. After sorting through a lot of conflicting data and chasing a few wild geese, I finally reproduced this problem at Feb 22 15:55 UTC (7:55am PST). I'd imagine that a number of streams were abandoned at this time. If you had a correlative experience within a minute or so of 15:55 UTC, please respond to this message.
We currently suspect an infrequent hardware load balancer issue, perhaps related to a recent configuration change. The appearance is that the load balancer is, for whatever reason, dropping valid connections, closing the connection to the Streaming API servers, but not sending a TCP FIN or TCP RST to the client. This is bad. We're treating this as a critical production issue and working through the details with network operations. I'll follow up as we learn more. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc.
