Sorry, John but this is really happening and I am having it on a daily basis in the last 2 weeks on both dev machine and production. I am using Apache httpClient and this code was working fine for months before this started to happen 2 weeks ago. I have a java call "String line =reader.readLine(); " and this call at least once a day just does not return (100% sure since I am logging all the call chain in debug mode).
So please take a look at whatever data you are monitoring to see if anything can explain this. In the meantime, I am building a monitoring thread to force a restart on the connection which is really saddening Thanks On Feb 19, 2:55 pm, John Kalucki <j...@twitter.com> wrote: > This shouldn't be happening, and having developers build these sorts of > workarounds saddens me. > > It is possible that the server side is holding dead connections open, but I > doubt it -- as I've a considerable amount of data to the contrary. I suspect > that the socket code does not detect a close, driven by either a TCP Close > or a TCP Reset. I've run connections over the public internet with close > monitoring and rarely noticed a timeout. > > If you point the same client at a file of streaming data on a web server, > does the client detect the end of file at the correct point? > > -John Kaluckihttp://twitter.com/jkalucki > Infrastructure, Twitter Inc. > > > > On Fri, Feb 19, 2010 at 9:16 AM, Dima Brodsky <ddbrod...@gmail.com> wrote: > > Hi, > > > Yup, I saw it the last couple of weeks, this week has been considerably > > better. I use the delimited stream so I do read(entry size), read(entry), > > repeat ... I just put a 30 second timeout on the read operation (this is all > > in python) and if a read fails I close the stream and reconnect. This seems > > to work quite well ... depending on the chunk size you are reading you could > > probably lower the timeout if you are afraid of losing data. > > > ttyl > > Dima > > > On 19-Feb-10, at 7:36 AM, rob wrote: > > > Has anyone else ran into an issue where over time the Streaming API > >> just stops sending results? > > >> We are using a Ruby library to connect (twitter-stream) which uses > >> EventMachine to open a persistent connection to the API (we are > >> tracking and following). > > >> The library properly handles reconnection (from dropped connections) > >> and the various error conditions. > > >> All works well for a period of time (8+ hours in some cases, sometimes > >> a full day) after that the connection does no get dropped but no data > >> gets sent. (At least that's what is seems as EventMachine feels its > >> still connected) > > >> If we just drop the connection and reconnect all is well and the data > >> starts to flow again. (Which we could do but that seems like a hack) > > >> Anyone else have this issue or should I dive into the EventMachine > >> code and see if there is an issue in there? > > >> Thanks in advance, > > >> Rob > > > -- > > ddbrod...@gmail.com > > > "The price of reliability is the pursuit of the utmost simplicity. > > It is a price which the very rich find the most hard to pay." > > (Sir > > Antony Hoare, 1980)