Sorry, John but this is really happening and I am having it on a daily
basis in the last 2 weeks on both dev machine and production. I am
using Apache httpClient and this code was working fine for months
before this started to happen 2 weeks ago. I have a java call "String
line =reader.readLine(); "  and this call at least once a day just
does not return (100% sure since I am logging all the call chain in
debug mode).

So please take a look at whatever data you are monitoring to see if
anything can explain this. In the meantime, I am building a monitoring
thread to force a restart on the connection which is really saddening


On Feb 19, 2:55 pm, John Kalucki <> wrote:
> This shouldn't be happening, and having developers build these sorts of
> workarounds saddens me.
> It is possible that the server side is holding dead connections open, but I
> doubt it -- as I've a considerable amount of data to the contrary. I suspect
> that the socket code does not detect a close, driven by either a TCP Close
> or a TCP Reset. I've run connections over the public internet with close
> monitoring and rarely noticed a timeout.
> If you point the same client at a file of streaming data on a web server,
> does the client detect the end of file at the correct point?
> -John Kalucki
> Infrastructure, Twitter Inc.
> On Fri, Feb 19, 2010 at 9:16 AM, Dima Brodsky <> wrote:
> > Hi,
> > Yup, I saw it the last couple of weeks, this week has been considerably
> > better.  I use the delimited stream so I do read(entry size), read(entry),
> > repeat ... I just put a 30 second timeout on the read operation (this is all
> > in python) and if a read fails I close the stream and reconnect.  This seems
> > to work quite well ... depending on the chunk size you are reading you could
> > probably lower the timeout if you are afraid of losing data.
> > ttyl
> > Dima
> > On 19-Feb-10, at 7:36 AM, rob wrote:
> >  Has anyone else ran into an issue where over time the Streaming API
> >> just stops sending results?
> >> We are using a Ruby library to connect (twitter-stream) which uses
> >> EventMachine to open a persistent connection to the API (we are
> >> tracking and following).
> >> The library properly handles reconnection (from dropped connections)
> >> and the various error conditions.
> >> All works well for a period of time (8+ hours in some cases, sometimes
> >> a full day)  after that the connection does no get dropped but no data
> >> gets sent. (At least that's what is seems as EventMachine feels its
> >> still connected)
> >> If we just drop the connection and reconnect all is well and the data
> >> starts to flow again. (Which we could do but that seems like a hack)
> >> Anyone else have this issue or should I dive into the EventMachine
> >> code and see if there is an issue in there?
> >> Thanks in advance,
> >> Rob
> > --
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> >                                                                    (Sir
> > Antony Hoare, 1980)

Reply via email to