There are two levels of blacklisting. One is a temporary band that
resets every few minutes. This one gives you 401 errors. Then there's
an IP black hole that is removed by an operator. Currently the IP
black hole sends a TCP RST, but we might might also null route you.
You can verify an IP block by attempting to connect from a different
network.

If you provide an account name, I can look through the logs and see
what happened. An IP address can also be helpful. In the absence of
these keys, I can only speculate as to what occurred.

-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.

On Nov 15, 12:54 am, Jim DeLaHunt <jim.delah...@gmail.com> wrote:
> John:
>
> Thanks very much for the reply.
>
> On Nov 14, 8:30 pm, John Kalucki <jkalu...@gmail.com> wrote:
>
> > This sounds like you were ignoring HTTP error codes and eventually got
> > blacklisted. 
> > Consider:http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting
>
> Hmm... I was launching single curl requests, making one connection
> then breaking it after max 3 seconds. I would then wait 6 minutes
> before trying to connect again.  I didn't record the HTTP result code
> I got back, but it seems that according to Streaming-API-
> Documentation#Connecting I was being tremendously conservative.  That
> doc recommends backing off for 10 to 240 seconds on an HTTP error code
> (>200); I always backed off for 360 seconds immediately, whether the
> HTTP error code was good or bad.
>
> How would backing off by *more* than the docs call for get me
> blacklisted?
>
> > You can tell for sure by turning off --silent and using -v to see
> > what's going on. You should be getting some sort of message back, or
> > absolutely nothing back. Those codes are not HTTP error codes, they
> > must be some curl artifact.
>
> Correct, the codes "6" and "52" are defined by curl. 
> Seehttp://curl.haxx.se/docs/manpage.html. Using -v and other curl
> options, I see clearly that what I'm getting back is "absolutely
> nothing back": 0 bytes in response to my HTTP query. (That's the
> meaning of the code "52".)
>
> For the last 6 hours, I've polled once per hour (once per 3600
> seconds), and this null response has not changed.
>
> The docs don't say how to confirm that I've been blacklisted. Any
> suggestions for how to confirm that? Nor do they say what to do if I
> am in fact blacklisted. They say that the blacklist lasts "an
> indeterminate period of time", so maybe they are implying I should
> just wait and the system will list the blacklist itself.
>
> The biggest issue, though, is to understand why I could have become
> blacklisted, when I backed off for 360 seconds after each attempt.
> Because right now, I don't know what I should do differently.
>
> Thanks again for the guidance.
>    --Jim DeLaHunt, Vancouver, Canada   �...@jdlh
>    Twanguages: a language census of Twitter 
> @twanguageshttp://jdlh.com/en/pr/twanguages.html
>
> > Tcpdump is also sometimes useful.
>
> > -John Kaluckihttp://twitter.com/jkalucki
> > Services, Twitter Inc.
>
> > On Nov 14, 6:13 pm, Jim DeLaHunt <jim.delah...@gmail.com> wrote:
>
> > > Am I the only one seeing this? I call the Streaming API 10x/hour. For
> > > the last 23 hours or so, I've been getting bad responses every time.
>
> > > I use a cron job to call from the Linux shell:
>
> > > curl --user myid:mypassword --silent --fail --max-time 3 --retry 
> > > 0http://stream.twitter.com/1/statuses/sample.xml
>
> > > and I get usually a curl return code "(52) Empty reply from server",
> > > though sometimes "(6) name lookup timed out". Same thing happens when
> > > I ask for .json instead of .xml.
>
> > > The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC
> > > (Friday early morning PST), though they became continuous as of
> > > 200/11/14 03:24h UTC (Friday evening PST), and remain continuous.
>
> > > Is anyone else calling this API and failing? Or succeeding? in the
> > > last 24 hours?
>
> > > Thank you,
> > >    --Jim DeLaHunt, Vancouver, Canada   �...@jdlh
> > >    Twanguages: a language census of Twitter   
> > > @twanguageshttp://jdlh.com/en/pr/twanguages.html

Reply via email to