Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-24 Thread John Kalucki
Network operations believes that they found and fixed the proximate cause of the connection abandonment issue -- a periodically overwhelmed LB CPU. The LB should close connections in this case, but for some reason it wasn't closing them over the last two weeks. Load will be more carefully managed o

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-23 Thread M. Edward (Ed) Borasky
On 02/23/2010 09:37 AM, John Kalucki wrote: > Judging my some additional data found yesterday, this drop apparently > happens most often at around say 14:30 and 16:00 UTC, a time period that we > also happen to steeply climb into our daily peak traffic. By our monitoring, > we have not experienced

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-23 Thread John Kalucki
Judging my some additional data found yesterday, this drop apparently happens most often at around say 14:30 and 16:00 UTC, a time period that we also happen to steeply climb into our daily peak traffic. By our monitoring, we have not experienced a connection drop so far this morning. But, I have n

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-22 Thread Scott Wilcox
Hi John, I've had the same setup for a while now without issue, but from 17:00GMT I started getting connections dropped, then once connected they'd just hang. I'll look into the lib and make sure its not my end. Scott. On 22 Feb 2010, at 22:25, John Kalucki wrote: > I don't believe that we've

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-22 Thread Marc Mims
> On Mon, Feb 22, 2010 at 1:53 PM, Scott Wilcox wrote: > > > Same issue here, over the past six hours more than ever. * John Kalucki [100222 14:25]: > I don't believe that we've seen this issue since 15:55 UTC. If you've been > seeing hangs more often, it's possible that you are experiencing an

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-22 Thread John Kalucki
I don't believe that we've seen this issue since 15:55 UTC. If you've been seeing hangs more often, it's possible that you are experiencing another problem. Or, equally likely, there is another set of things going wrong. I've restarted the cluster twice today, but this shouldn't cause hanging -- j

Re: [twitter-dev] Re: Streaming API connection abandonment & separation anxiety issues

2010-02-22 Thread Scott Wilcox
Same issue here, over the past six hours more than ever. On 22 Feb 2010, at 19:13, John Kalucki wrote: > One further note: A reasonable workaround for the moment is to, if your > client allows iotcls, set a socket timeout of about 90 seconds on your > streaming api connection. The servers curre