Chris ,
        We implemented something like this "network status" using the  
rate_limit_status call (for the IP), while some of the numbers are  
sometimes wonky with this api right now we poll this every 5 minutes  
and set a flag to enable or disable all twitter requests from the  
server depending on what is returned. This has the effect of backing  
off twitter when we get limited, as well as automatically bring  
twitter service up when the blackout period subsides.. Even with the  
wonky numbers, we blindly accept what the call returns and disable  
when api credits say they are 0 and if the call times out you know for  
sure its time to back off.  ( I know these criteria are overly  
aggressive, but we figure its better to aggressively back off in the  
hopes it helps things recover quicker)

When the server is in blackout, simply report that twitter is still  
having ongoing issues to the user with a note and a link to the  
twitter status page, keeps them informed as to what is going on  
without flooding your inbox or you having to respond to all the users  
emails, If the message in the UI saves you from having to read through  
and answer 10 emails, it is worth the work to implement. Its all about  
keeping the user informed, just like we like to be informed by twitter  
about what the status is.

Another tip is to make sure you set a resonable timeout for all your  
twitter requests or people get stuck waiting for your page to load and  
think its broken, better to timeout and again report that twitter is  
still recovering for DDoS attack and direct them to the twitter status  

While this may seem like unnecessary work if twitter is going to get  
back to normal, it makes your app more robust to issues in the future,  
keeps your users better informed as to what is happening (without  
contacting you), and also helps reduce number of requests to twitter  
that they have to ignore while they are recovering

On that note, it seems our servers are disabling significantly less  
frequently over the past few hours. Heres hoping it lasts.. Not sure  
if it was the back off or if Twitter has just managed to cope better  
with the traffic issues.. Doesn't really matter to me which..

Oh and thanks for the update Chad, it really is nice just to hear  
something, even if its not great news.

On Aug 8, 2009, at 4:19 AM, Chris Babcock wrote:

> On Fri, 7 Aug 2009 11:05:32 -0700
> Ryan Sarver <> wrote:
>> I wanted to send everyone an update to let you know what has been
>> happening, the known issues, some suggestions on how to resolve them
>> and some idea of how to move forward.
> This was really appreciated. When the dust clears, maybe one more
> suggestion? An API to check on the network status? I think
> infrastucture attacks aren't going away anytime soon. We've got a
> diversity of applications here, some of which can chew up the  
> bandwidth
> pretty well and some of those just don't make sense to run if other
> users can't get on-line. Instead of answering, "Is it OK to restart my
> cron jobs?" the cron jobs could shut themselves down for increments of
> so many hours.
> Chris Babcock
> PS - Of course it could be misused, but I think the benefit is net.

Reply via email to