Hey guys, Wanted to share a few details about last nights experience in case anyone else gets hit with it. Hopefully it can save you a few hours troubleshooting if it happens to you.
Favstar's IP address was blacklisted by twitter yesterday. When this occurs, they don't inform you of it. Instead, you start seeing percentage of your requests blocked. Not all of them, just some of them. For me it varied between the 50% and 80% range. In the way I do my logging, these appeared as timeouts, so at first I thought the API was suffering overload, and when @mccv told me there was no overload, I fell in to trap of trying to diagnose either what was wrong with my server, or what was wrong with the network in between. What I should have done, is ran a curl in verbose mode (-v). This tells you that your connections are being refused: ~/current: curl -i -u my_account:fuuu! http://api.twitter.com/1/account/rate_limit_status.json -v * About to connect() to api.twitter.com port 80 (#0) * Trying 22.214.171.124... Connection refused * Trying 126.96.36.199... Connection refused * Trying 188.8.131.52... Connection refused * Trying 184.108.40.206... connected <snip correct/incorrect response> When I tried this from another server, my connections were never refused. When I tried this from the blacklisted server, I would see something like the above. Sometimes I'd get a successful response, sometimes I'd get "curl: (52) Empty reply from server" which googling for is useless, and sometimes I'd get "curl: (7) couldn't connect to host". If you'd like to see Twitter make a reasonable attempt to notify 3rd parties when they are blacklisted, please vote on this issue: http://code.google.com/p/twitter-api/issues/detail?id=1658 Cheers, Tim.