Thanks very much for the reply.

On Nov 14, 8:30 pm, John Kalucki <> wrote:
> This sounds like you were ignoring HTTP error codes and eventually got
> blacklisted. 
> Consider:

Hmm... I was launching single curl requests, making one connection
then breaking it after max 3 seconds. I would then wait 6 minutes
before trying to connect again.  I didn't record the HTTP result code
I got back, but it seems that according to Streaming-API-
Documentation#Connecting I was being tremendously conservative.  That
doc recommends backing off for 10 to 240 seconds on an HTTP error code
(>200); I always backed off for 360 seconds immediately, whether the
HTTP error code was good or bad.

How would backing off by *more* than the docs call for get me

> You can tell for sure by turning off --silent and using -v to see
> what's going on. You should be getting some sort of message back, or
> absolutely nothing back. Those codes are not HTTP error codes, they
> must be some curl artifact.

Correct, the codes "6" and "52" are defined by curl. See . Using -v and other curl
options, I see clearly that what I'm getting back is "absolutely
nothing back": 0 bytes in response to my HTTP query. (That's the
meaning of the code "52".)

For the last 6 hours, I've polled once per hour (once per 3600
seconds), and this null response has not changed.

The docs don't say how to confirm that I've been blacklisted. Any
suggestions for how to confirm that? Nor do they say what to do if I
am in fact blacklisted. They say that the blacklist lasts "an
indeterminate period of time", so maybe they are implying I should
just wait and the system will list the blacklist itself.

The biggest issue, though, is to understand why I could have become
blacklisted, when I backed off for 360 seconds after each attempt.
Because right now, I don't know what I should do differently.

Thanks again for the guidance.
   --Jim DeLaHunt, Vancouver, Canada    @jdlh
   Twanguages: a language census of Twitter @twanguages

> Tcpdump is also sometimes useful.
> -John Kalucki
> Services, Twitter Inc.
> On Nov 14, 6:13 pm, Jim DeLaHunt <> wrote:
> > Am I the only one seeing this? I call the Streaming API 10x/hour. For
> > the last 23 hours or so, I've been getting bad responses every time.
> > I use a cron job to call from the Linux shell:
> > curl --user myid:mypassword --silent --fail --max-time 3 --retry 
> > 0
> > and I get usually a curl return code "(52) Empty reply from server",
> > though sometimes "(6) name lookup timed out". Same thing happens when
> > I ask for .json instead of .xml.
> > The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC
> > (Friday early morning PST), though they became continuous as of
> > 200/11/14 03:24h UTC (Friday evening PST), and remain continuous.
> > Is anyone else calling this API and failing? Or succeeding? in the
> > last 24 hours?
> > Thank you,
> >    --Jim DeLaHunt, Vancouver, Canada   �...@jdlh
> >    Twanguages: a language census of Twitter   
> > @twanguages

Reply via email to