This shouldn't be happening, and having developers build these sorts of
workarounds saddens me.

It is possible that the server side is holding dead connections open, but I
doubt it -- as I've a considerable amount of data to the contrary. I suspect
that the socket code does not detect a close, driven by either a TCP Close
or a TCP Reset. I've run connections over the public internet with close
monitoring and rarely noticed a timeout.

If you point the same client at a file of streaming data on a web server,
does the client detect the end of file at the correct point?

-John Kalucki
http://twitter.com/jkalucki
Infrastructure, Twitter Inc.




On Fri, Feb 19, 2010 at 9:16 AM, Dima Brodsky <ddbrod...@gmail.com> wrote:

> Hi,
>
> Yup, I saw it the last couple of weeks, this week has been considerably
> better.  I use the delimited stream so I do read(entry size), read(entry),
> repeat ... I just put a 30 second timeout on the read operation (this is all
> in python) and if a read fails I close the stream and reconnect.  This seems
> to work quite well ... depending on the chunk size you are reading you could
> probably lower the timeout if you are afraid of losing data.
>
> ttyl
> Dima
>
>
> On 19-Feb-10, at 7:36 AM, rob wrote:
>
>  Has anyone else ran into an issue where over time the Streaming API
>> just stops sending results?
>>
>> We are using a Ruby library to connect (twitter-stream) which uses
>> EventMachine to open a persistent connection to the API (we are
>> tracking and following).
>>
>> The library properly handles reconnection (from dropped connections)
>> and the various error conditions.
>>
>> All works well for a period of time (8+ hours in some cases, sometimes
>> a full day)  after that the connection does no get dropped but no data
>> gets sent. (At least that's what is seems as EventMachine feels its
>> still connected)
>>
>> If we just drop the connection and reconnect all is well and the data
>> starts to flow again. (Which we could do but that seems like a hack)
>>
>> Anyone else have this issue or should I dive into the EventMachine
>> code and see if there is an issue in there?
>>
>> Thanks in advance,
>>
>> Rob
>>
>>
>>
>>
>>
>>
>>
> --
> ddbrod...@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
>                                                                    (Sir
> Antony Hoare, 1980)
>
>

Reply via email to