[twitter-dev] Re: Empty reply from server on Streaming API?
Just to close the loop on my issue: I got some off-list help, Twitter investigated, and it turns out my IP address had been blacklisted in error. The blacklisting is removed, and I'm back in business. I must say it's nice that I could ask a question on this list, and get pretty much immediate attention from the proper Twitter person, and over a weekend at that. Thanks, John Kalucki, and thanks, Twitter. --Jim DeLaHunt, Vancouver, Canada@jdlh http://jdlh.com/en/pr/twanguages.html Twanguages: a language census of Twitter @twanguages On Nov 15, 6:52 am, John Kalucki jkalu...@gmail.com wrote: There are two levels of blacklisting. One is a temporary band that resets every few minutes. This one gives you 401 errors. Then there's an IP black hole that is removed by an operator. Currently the IP black hole sends a TCP RST, but we might might also null route you. You can verify an IP block by attempting to connect from a different network. If you provide an account name, I can look through the logs and see what happened. An IP address can also be helpful. In the absence of these keys, I can only speculate as to what occurred. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Nov 15, 12:54 am, Jim DeLaHunt jim.delah...@gmail.com wrote: John: Thanks very much for the reply. On Nov 14, 8:30 pm, John Kalucki jkalu...@gmail.com wrote: This sounds like you were ignoring HTTP error codes and eventually got blacklisted. Consider:http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting Hmm... I was launching single curl requests, making one connection then breaking it after max 3 seconds. I would then wait 6 minutes before trying to connect again. I didn't record the HTTP result code I got back, but it seems that according to Streaming-API- Documentation#Connecting I was being tremendously conservative. That doc recommends backing off for 10 to 240 seconds on an HTTP error code (200); I always backed off for 360 seconds immediately, whether the HTTP error code was good or bad. How would backing off by *more* than the docs call for get me blacklisted? You can tell for sure by turning off --silent and using -v to see what's going on. You should be getting some sort of message back, or absolutely nothing back. Those codes are not HTTP error codes, they must be some curl artifact. Correct, the codes 6 and 52 are defined by curl. Seehttp://curl.haxx.se/docs/manpage.html. Using -v and other curl options, I see clearly that what I'm getting back is absolutely nothing back: 0 bytes in response to my HTTP query. (That's the meaning of the code 52.) For the last 6 hours, I've polled once per hour (once per 3600 seconds), and this null response has not changed. The docs don't say how to confirm that I've been blacklisted. Any suggestions for how to confirm that? Nor do they say what to do if I am in fact blacklisted. They say that the blacklist lasts an indeterminate period of time, so maybe they are implying I should just wait and the system will list the blacklist itself. The biggest issue, though, is to understand why I could have become blacklisted, when I backed off for 360 seconds after each attempt. Because right now, I don't know what I should do differently. Thanks again for the guidance. --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html Tcpdump is also sometimes useful. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Nov 14, 6:13 pm, Jim DeLaHunt jim.delah...@gmail.com wrote: Am I the only one seeing this? I call the Streaming API 10x/hour. For the last 23 hours or so, I've been getting bad responses every time. I use a cron job to call from the Linux shell: curl --user myid:mypassword --silent --fail --max-time 3 --retry 0http://stream.twitter.com/1/statuses/sample.xml and I get usually a curl return code (52) Empty reply from server, though sometimes (6) name lookup timed out. Same thing happens when I ask for .json instead of .xml. The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC (Friday early morning PST), though they became continuous as of 200/11/14 03:24h UTC (Friday evening PST), and remain continuous. Is anyone else calling this API and failing? Or succeeding? in the last 24 hours? Thank you, --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html
[twitter-dev] Re: Empty reply from server on Streaming API?
John: Thanks very much for the reply. On Nov 14, 8:30 pm, John Kalucki jkalu...@gmail.com wrote: This sounds like you were ignoring HTTP error codes and eventually got blacklisted. Consider:http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting Hmm... I was launching single curl requests, making one connection then breaking it after max 3 seconds. I would then wait 6 minutes before trying to connect again. I didn't record the HTTP result code I got back, but it seems that according to Streaming-API- Documentation#Connecting I was being tremendously conservative. That doc recommends backing off for 10 to 240 seconds on an HTTP error code (200); I always backed off for 360 seconds immediately, whether the HTTP error code was good or bad. How would backing off by *more* than the docs call for get me blacklisted? You can tell for sure by turning off --silent and using -v to see what's going on. You should be getting some sort of message back, or absolutely nothing back. Those codes are not HTTP error codes, they must be some curl artifact. Correct, the codes 6 and 52 are defined by curl. See http://curl.haxx.se/docs/manpage.html . Using -v and other curl options, I see clearly that what I'm getting back is absolutely nothing back: 0 bytes in response to my HTTP query. (That's the meaning of the code 52.) For the last 6 hours, I've polled once per hour (once per 3600 seconds), and this null response has not changed. The docs don't say how to confirm that I've been blacklisted. Any suggestions for how to confirm that? Nor do they say what to do if I am in fact blacklisted. They say that the blacklist lasts an indeterminate period of time, so maybe they are implying I should just wait and the system will list the blacklist itself. The biggest issue, though, is to understand why I could have become blacklisted, when I backed off for 360 seconds after each attempt. Because right now, I don't know what I should do differently. Thanks again for the guidance. --Jim DeLaHunt, Vancouver, Canada@jdlh Twanguages: a language census of Twitter @twanguages http://jdlh.com/en/pr/twanguages.html Tcpdump is also sometimes useful. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Nov 14, 6:13 pm, Jim DeLaHunt jim.delah...@gmail.com wrote: Am I the only one seeing this? I call the Streaming API 10x/hour. For the last 23 hours or so, I've been getting bad responses every time. I use a cron job to call from the Linux shell: curl --user myid:mypassword --silent --fail --max-time 3 --retry 0http://stream.twitter.com/1/statuses/sample.xml and I get usually a curl return code (52) Empty reply from server, though sometimes (6) name lookup timed out. Same thing happens when I ask for .json instead of .xml. The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC (Friday early morning PST), though they became continuous as of 200/11/14 03:24h UTC (Friday evening PST), and remain continuous. Is anyone else calling this API and failing? Or succeeding? in the last 24 hours? Thank you, --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html
[twitter-dev] Re: Empty reply from server on Streaming API?
I had the same issue. And i was blacklisted. On 11/15/09, Jim DeLaHunt jim.delah...@gmail.com wrote: John: Thanks very much for the reply. On Nov 14, 8:30 pm, John Kalucki jkalu...@gmail.com wrote: This sounds like you were ignoring HTTP error codes and eventually got blacklisted. Consider:http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting Hmm... I was launching single curl requests, making one connection then breaking it after max 3 seconds. I would then wait 6 minutes before trying to connect again. I didn't record the HTTP result code I got back, but it seems that according to Streaming-API- Documentation#Connecting I was being tremendously conservative. That doc recommends backing off for 10 to 240 seconds on an HTTP error code (200); I always backed off for 360 seconds immediately, whether the HTTP error code was good or bad. How would backing off by *more* than the docs call for get me blacklisted? You can tell for sure by turning off --silent and using -v to see what's going on. You should be getting some sort of message back, or absolutely nothing back. Those codes are not HTTP error codes, they must be some curl artifact. Correct, the codes 6 and 52 are defined by curl. See http://curl.haxx.se/docs/manpage.html . Using -v and other curl options, I see clearly that what I'm getting back is absolutely nothing back: 0 bytes in response to my HTTP query. (That's the meaning of the code 52.) For the last 6 hours, I've polled once per hour (once per 3600 seconds), and this null response has not changed. The docs don't say how to confirm that I've been blacklisted. Any suggestions for how to confirm that? Nor do they say what to do if I am in fact blacklisted. They say that the blacklist lasts an indeterminate period of time, so maybe they are implying I should just wait and the system will list the blacklist itself. The biggest issue, though, is to understand why I could have become blacklisted, when I backed off for 360 seconds after each attempt. Because right now, I don't know what I should do differently. Thanks again for the guidance. --Jim DeLaHunt, Vancouver, Canada@jdlh Twanguages: a language census of Twitter @twanguages http://jdlh.com/en/pr/twanguages.html Tcpdump is also sometimes useful. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Nov 14, 6:13 pm, Jim DeLaHunt jim.delah...@gmail.com wrote: Am I the only one seeing this? I call the Streaming API 10x/hour. For the last 23 hours or so, I've been getting bad responses every time. I use a cron job to call from the Linux shell: curl --user myid:mypassword --silent --fail --max-time 3 --retry 0http://stream.twitter.com/1/statuses/sample.xml and I get usually a curl return code (52) Empty reply from server, though sometimes (6) name lookup timed out. Same thing happens when I ask for .json instead of .xml. The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC (Friday early morning PST), though they became continuous as of 200/11/14 03:24h UTC (Friday evening PST), and remain continuous. Is anyone else calling this API and failing? Or succeeding? in the last 24 hours? Thank you, --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html -- Sent from my mobile device A K M Mokaddim http://talk.cmyweb.net http://twitter.com/shiplu Stop Top Posting !! বাংলিশ লেখার চাইতে বাংলা লেখা অনেক ভাল
[twitter-dev] Re: Empty reply from server on Streaming API?
There are two levels of blacklisting. One is a temporary band that resets every few minutes. This one gives you 401 errors. Then there's an IP black hole that is removed by an operator. Currently the IP black hole sends a TCP RST, but we might might also null route you. You can verify an IP block by attempting to connect from a different network. If you provide an account name, I can look through the logs and see what happened. An IP address can also be helpful. In the absence of these keys, I can only speculate as to what occurred. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Nov 15, 12:54 am, Jim DeLaHunt jim.delah...@gmail.com wrote: John: Thanks very much for the reply. On Nov 14, 8:30 pm, John Kalucki jkalu...@gmail.com wrote: This sounds like you were ignoring HTTP error codes and eventually got blacklisted. Consider:http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting Hmm... I was launching single curl requests, making one connection then breaking it after max 3 seconds. I would then wait 6 minutes before trying to connect again. I didn't record the HTTP result code I got back, but it seems that according to Streaming-API- Documentation#Connecting I was being tremendously conservative. That doc recommends backing off for 10 to 240 seconds on an HTTP error code (200); I always backed off for 360 seconds immediately, whether the HTTP error code was good or bad. How would backing off by *more* than the docs call for get me blacklisted? You can tell for sure by turning off --silent and using -v to see what's going on. You should be getting some sort of message back, or absolutely nothing back. Those codes are not HTTP error codes, they must be some curl artifact. Correct, the codes 6 and 52 are defined by curl. Seehttp://curl.haxx.se/docs/manpage.html. Using -v and other curl options, I see clearly that what I'm getting back is absolutely nothing back: 0 bytes in response to my HTTP query. (That's the meaning of the code 52.) For the last 6 hours, I've polled once per hour (once per 3600 seconds), and this null response has not changed. The docs don't say how to confirm that I've been blacklisted. Any suggestions for how to confirm that? Nor do they say what to do if I am in fact blacklisted. They say that the blacklist lasts an indeterminate period of time, so maybe they are implying I should just wait and the system will list the blacklist itself. The biggest issue, though, is to understand why I could have become blacklisted, when I backed off for 360 seconds after each attempt. Because right now, I don't know what I should do differently. Thanks again for the guidance. --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html Tcpdump is also sometimes useful. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Nov 14, 6:13 pm, Jim DeLaHunt jim.delah...@gmail.com wrote: Am I the only one seeing this? I call the Streaming API 10x/hour. For the last 23 hours or so, I've been getting bad responses every time. I use a cron job to call from the Linux shell: curl --user myid:mypassword --silent --fail --max-time 3 --retry 0http://stream.twitter.com/1/statuses/sample.xml and I get usually a curl return code (52) Empty reply from server, though sometimes (6) name lookup timed out. Same thing happens when I ask for .json instead of .xml. The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC (Friday early morning PST), though they became continuous as of 200/11/14 03:24h UTC (Friday evening PST), and remain continuous. Is anyone else calling this API and failing? Or succeeding? in the last 24 hours? Thank you, --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html
[twitter-dev] Re: Empty reply from server on Streaming API?
This sounds like you were ignoring HTTP error codes and eventually got blacklisted. Consider: http://apiwiki.twitter.com/Streaming-API-Documentation#Connecting You can tell for sure by turning off --silent and using -v to see what's going on. You should be getting some sort of message back, or absolutely nothing back. Those codes are not HTTP error codes, they must be some curl artifact. Tcpdump is also sometimes useful. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Nov 14, 6:13 pm, Jim DeLaHunt jim.delah...@gmail.com wrote: Am I the only one seeing this? I call the Streaming API 10x/hour. For the last 23 hours or so, I've been getting bad responses every time. I use a cron job to call from the Linux shell: curl --user myid:mypassword --silent --fail --max-time 3 --retry 0http://stream.twitter.com/1/statuses/sample.xml and I get usually a curl return code (52) Empty reply from server, though sometimes (6) name lookup timed out. Same thing happens when I ask for .json instead of .xml. The failures started at the rate of 1-2/hour on 2009/11/13 09:00h UTC (Friday early morning PST), though they became continuous as of 200/11/14 03:24h UTC (Friday evening PST), and remain continuous. Is anyone else calling this API and failing? Or succeeding? in the last 24 hours? Thank you, --Jim DeLaHunt, Vancouver, Canada �...@jdlh Twanguages: a language census of Twitter @twanguageshttp://jdlh.com/en/pr/twanguages.html
