I've deployed a new version that records the retry-after responses and more details. But on this data I would say there's something broken or radically changed in the search API:
2011-01-25 21:47:06.633772 response_status: 420 (retry_after: 1487) 2011-01-25 21:52:12.933658 still must wait: 0:24:47 2011-01-25 21:57:12.976032 still must wait: 0:24:47 2011-01-25 22:04:16.410047 still must wait: 0:24:47 2011-01-25 22:05:15.735935 still must wait: 0:24:47 2011-01-25 22:06:15.776125 still must wait: 0:24:47 2011-01-25 22:07:15.818147 still must wait: 0:24:47 2011-01-25 22:08:15.803914 still must wait: 0:24:47 2011-01-25 22:09:15.841937 still must wait: 0:24:47 2011-01-25 22:09:16.677202 still must wait: 0:24:47 2011-01-25 22:11:24.344873 still must wait: 0:24:47 2011-01-25 23:10:14.798485 response_status: 420 (retry_after: 99) 2011-01-25 23:10:28.962504 still must wait: 0:01:39 2011-01-25 23:11:16.013676 still must wait: 0:01:39 2011-01-25 23:11:24.657504 still must wait: 0:01:39 2011-01-25 23:11:28.754002 still must wait: 0:01:39 2011-01-25 23:15:29.337179 response_status: 420 (retry_after: 1584) 2011-01-25 23:17:14.925825 still must wait: 0:26:24 2011-01-25 23:17:15.920827 still must wait: 0:26:24 2011-01-25 23:17:26.990475 still must wait: 0:26:24 2011-01-25 23:19:16.208185 still must wait: 0:26:24 2011-01-25 23:19:28.786134 still must wait: 0:26:24 2011-01-25 23:20:16.054580 still must wait: 0:26:24 Please note that requests logged as "still must wait" where not launched. They were just acknowledging the retry after. Basically in over 1.5 hour I haven't been able to make any request. :- alex On Jan 25, 8:53 am, Alex Popescu <the.mindstorm.mailingl...@gmail.com> wrote: > Hi guys, > > It is a bit over an year since I've been running a small volume > Twitter based app deployed on Google App Engine. > > Basically this app is using the search API to poll some results. > According to the app stats it is making less than 100 request/hour. > > But since last weekend, it seems it is continuously getting a rate > limitation error and even if I backdown this doesn't change. > > Unfortunately, switching to the streamapi is not possible as Google > App Engine does not allow long lived responses. So, my questions is: > > 1. how should I proceed? > > 2. is there any way I could whitelist this little app so it doesn't > get rate limited? (nb: I am aware that the search API doesn't have an > official whitelisting process) > > 3. considering I'm running on Google App Engine, I cannot provide an > IP, but I could definitely make sure to submit some whitelisted > headers (or something in this area). > > While little, this app is quite essential for me. Please advise how > should I proceed. > > Many thanks in advance, > > :- alex -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
