Re: [twitter-dev] Re: Search API rate limit change?
I just wanted to add to this. The 420s have let up for the most part and I'm no longer seeing rate limiting behavior significantly different from the norm. I've noticed that many result pages are coming back with empty results but if I re-request the same page (after a couple second delay), I can often get results for that page. These are for queries with very low tweet velocity, so it's not like these are new results coming in. Is this related to http://status.twitter.com/post/3785043723/slow-searches ? Thanks. Hayes On Mon, Mar 21, 2011 at 6:13 PM, Matt Harris thematthar...@twitter.comwrote: In many cases we are forced to change the rate limits in response to a significant increase in requests, which means it isn't always possible to give advanced notice of rate limit changes. For some of you it sounds like your code that handles rate limiting didn't react appropriately. When receiving a 420 response we recommend you stop making requests and then after the retry-after, slowly build up the number of requests you make. Put another way it isn't a good idea to make requests to the API at the velocity that caused the 420 response before. As always, the rate limits are there to ensure the system is responsive and available to as many users as possible. This means it is necessary to reduce the number of queries you can make without notice. The best place to stay informed about issues like this are posted through @twitterapi and published on the Twitter status blog: http://status.twitter.com/post/3785043723/slow-searches Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Mon, Mar 21, 2011 at 7:41 AM, Colin Surprenant colin.surpren...@gmail.com wrote: By adjusting the rate limits to reduce the stress on your search api without notice you have significantly increased the stress level on our end :P Seriously, advanced notice of the situation would have been welcome. In particular what created lots of confusion on our end is that even after pausing for the specified retry_after delay we would immediately get repeated 420s at which point we started to assume our IPs were banned (which also contributed to increase the stress level). Colin On Mar 21, 9:12 am, Jeffrey Greenberg jeffreygreenb...@gmail.com wrote: Taylor, Yeah this was definitely NOT good.In the past, when there is a service disruption, your api group would post something on your status page and tweet about it... Instead, I'm finding out about this from my customers... Did y'all tweet about this or present this somewhere where I could find it? Jeffrey Tweettronics.com On Sun, Mar 20, 2011 at 3:14 PM, Waldron Faulkner waldronfaulk...@gmail.com wrote: Without prior notice, I can understand (circumstances), but without any kind of subsequent announcement?? Means we have to discover issues ourselves, verify that they're Twitter related (and not internal), then search around for existing discussion on the topic. Saves us a lot of time and headaches if Twitter would just announce stuff like this. On Mar 18, 2:51 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about
Re: [twitter-dev] Re: Search API rate limit change?
Taylor, Yeah this was definitely NOT good.In the past, when there is a service disruption, your api group would post something on your status page and tweet about it... Instead, I'm finding out about this from my customers... Did y'all tweet about this or present this somewhere where I could find it? Jeffrey Tweettronics.com On Sun, Mar 20, 2011 at 3:14 PM, Waldron Faulkner waldronfaulk...@gmail.com wrote: Without prior notice, I can understand (circumstances), but without any kind of subsequent announcement?? Means we have to discover issues ourselves, verify that they're Twitter related (and not internal), then search around for existing discussion on the topic. Saves us a lot of time and headaches if Twitter would just announce stuff like this. On Mar 18, 2:51 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API rate limit change?
By adjusting the rate limits to reduce the stress on your search api without notice you have significantly increased the stress level on our end :P Seriously, advanced notice of the situation would have been welcome. In particular what created lots of confusion on our end is that even after pausing for the specified retry_after delay we would immediately get repeated 420s at which point we started to assume our IPs were banned (which also contributed to increase the stress level). Colin On Mar 21, 9:12 am, Jeffrey Greenberg jeffreygreenb...@gmail.com wrote: Taylor, Yeah this was definitely NOT good. In the past, when there is a service disruption, your api group would post something on your status page and tweet about it... Instead, I'm finding out about this from my customers... Did y'all tweet about this or present this somewhere where I could find it? Jeffrey Tweettronics.com On Sun, Mar 20, 2011 at 3:14 PM, Waldron Faulkner waldronfaulk...@gmail.com wrote: Without prior notice, I can understand (circumstances), but without any kind of subsequent announcement?? Means we have to discover issues ourselves, verify that they're Twitter related (and not internal), then search around for existing discussion on the topic. Saves us a lot of time and headaches if Twitter would just announce stuff like this. On Mar 18, 2:51 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list Change your membership to this group:http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: Search API rate limit change?
In many cases we are forced to change the rate limits in response to a significant increase in requests, which means it isn't always possible to give advanced notice of rate limit changes. For some of you it sounds like your code that handles rate limiting didn't react appropriately. When receiving a 420 response we recommend you stop making requests and then after the retry-after, slowly build up the number of requests you make. Put another way it isn't a good idea to make requests to the API at the velocity that caused the 420 response before. As always, the rate limits are there to ensure the system is responsive and available to as many users as possible. This means it is necessary to reduce the number of queries you can make without notice. The best place to stay informed about issues like this are posted through @twitterapi and published on the Twitter status blog: http://status.twitter.com/post/3785043723/slow-searches Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Mon, Mar 21, 2011 at 7:41 AM, Colin Surprenant colin.surpren...@gmail.com wrote: By adjusting the rate limits to reduce the stress on your search api without notice you have significantly increased the stress level on our end :P Seriously, advanced notice of the situation would have been welcome. In particular what created lots of confusion on our end is that even after pausing for the specified retry_after delay we would immediately get repeated 420s at which point we started to assume our IPs were banned (which also contributed to increase the stress level). Colin On Mar 21, 9:12 am, Jeffrey Greenberg jeffreygreenb...@gmail.com wrote: Taylor, Yeah this was definitely NOT good.In the past, when there is a service disruption, your api group would post something on your status page and tweet about it... Instead, I'm finding out about this from my customers... Did y'all tweet about this or present this somewhere where I could find it? Jeffrey Tweettronics.com On Sun, Mar 20, 2011 at 3:14 PM, Waldron Faulkner waldronfaulk...@gmail.com wrote: Without prior notice, I can understand (circumstances), but without any kind of subsequent announcement?? Means we have to discover issues ourselves, verify that they're Twitter related (and not internal), then search around for existing discussion on the topic. Saves us a lot of time and headaches if Twitter would just announce stuff like this. On Mar 18, 2:51 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via
[twitter-dev] Re: Search API rate limit change?
Without prior notice, I can understand (circumstances), but without any kind of subsequent announcement?? Means we have to discover issues ourselves, verify that they're Twitter related (and not internal), then search around for existing discussion on the topic. Saves us a lot of time and headaches if Twitter would just announce stuff like this. On Mar 18, 2:51 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API rate limit change?
We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: Search API rate limit change?
We're working to reinstate the usual limits on the Search API; due to the impact of the Japanese earthquake and resultant query increase against the Search API, some rates were adjusted to cope better serve queries. Will give everyone an update with the various limits are adjusted. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis ha...@appozite.com wrote: Hi, We're seeing this as well starting at approximately the same time as described. We've backed off on searching but are seeing no reduction in the sporadic limiting. It also appears that the amount of results returned on successful queries is severely limited. Some queries that often have 1500 tweets from the last 5 days are returning far fewer results from only the last day. Could we get an update on this? Hayes On Fri, Mar 18, 2011 at 10:13 AM, Eric e...@telvetto.com wrote: We're also seeing 400s on different boxes across different IP addresses with different queries (so it does not appear to be server or query specific). These began on all boxes at 2 a.m. UTC. We've backed off on both number and rate of queries with no effect. We've also noticed an increase in sporadic fail whales via browser based search (atom and html) from personal accounts, although we haven't attempted to quantify it. On Mar 18, 7:40 am, zaver zave...@hotmail.com wrote: Hello, After the latest performance issues with the search api i have been seeing a lot of 420 response codes.From yesterday until now i only get 420 responses on the every search i make. In particular, i search for about 100 keywords simultaneously every 6 mins. Why is this happening? Was there any change on the Search API limit? Any help is greatly appreciated. Thanks, Zaver -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API rate limit on some keywords
have observed that sometimes some of the keywords get a 420 code? Any ideas why is this happening? You get a 420 NOT USED when a search term hasn't been used recently where the recently is whatever small timeframe (sometimes 7 days, often less) is currently available in the search index. I get it all the time for things like #stlcards :) -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API rate limit on some keywords
Thanks for the reply. So as i understand it i am not being rate limited, it just doesn't find any results to return for the specific keywords. In that case should i wait for the amount of time specified on the Retry-after field to make a new search? If i don't wait will that lead to my ip getting blacklisted or it will just not return accurate search results? Thanks in advance. On Feb 2, 3:04 am, @IDisposable idisposa...@gmail.com wrote: have observed that sometimes some of the keywords get a 420 code? Any ideas why is this happening? You get a 420 NOT USED when a search term hasn't been used recently where the recently is whatever small timeframe (sometimes 7 days, often less) is currently available in the search index. I get it all the time for things like #stlcards :) -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API rate limit
Matt, What is exact limit..Whether I can write to twitter for whitelisting of the IP? Whether whitelisting of the IP would do any good? Shan On Jul 7, 12:16 am, Matt Harris thematthar...@twitter.com wrote: Hi Shan, The Search API is anonymous so authenticating makes no difference to the rate limit there. If you are requesting a lot of information from the search API you may want to look at the streaming API instead:http://dev.twitter.com/pages/streaming_api The majority of search cases can be handled by the default filter limits available through the streaming API. Best, Matt On Tue, Jul 6, 2010 at 7:31 AM, Ramanean shang...@gmail.com wrote: I am developing a normal twitter search application Inorder to beat the search rate limit if I ask a user to authenticate whether that would be helpful? Whether the calls made by the user for search api will be counted in the user's account ? or whether that would be still counted as a call from the IP address of the website? I am little bit confused here... Thanks Shan -- Matt Harris Developer Advocate, Twitterhttp://twitter.com/themattharris
Re: [twitter-dev] Re: Search API rate limit
Shan, as far as I know twitter has been reluctant to state definite numbers, so you'll have to experiment and implement a backoff mechanism in your app. Here is the relevant part of the docs: Search API Rate Limiting The Search API is rate limited by IP address. The number of search requests that originate from a given IP address are counted against the search rate limiter. The specific number of requests a client is able to make to the Search API for a given hour is not released. Note that the Search API is not limited by the same 150 requests per hour limit as the REST API. The number is quite a bit higher and we feel it is both liberal and sufficient for most applications. We do not give the exact number because we want to discourage unnecessary search usage. Search API usage requires that applications include a unique and identifying User Agent string. A HTTP Referrer is expected but is not required. Consumers using the Search API but failing to include a User Agent string will receive a lower rate limit. An application that exceeds the rate limitations of the Search API will receive HTTP 420 response codes to requests. It is a best practice to watch for this error condition and honor the Retry-After header that instructs the application when it is safe to continue. The Retry-After header's value is the number of seconds your application should wait before submitting another query (for example: Retry-After: 67). Cheers, Pascal On Jul 7, 2010, at 1:55 , Ramanean wrote: Matt, What is exact limit..Whether I can write to twitter for whitelisting of the IP? Whether whitelisting of the IP would do any good? Shan
[twitter-dev] Re: Search API rate limit IP address question
Thank you for your reply! If this were true then sometimes your request works and other times it doesn't. Is that the case? Yes, each time I run my app, it makes ~80 calls to the Search API. I can only run a full test of the app 2 or 3 times before I get the Stream Error. But if I run a partial test of only 10 or so calls, I can run it a bunch more times before getting the error. If I wait 30 minutes or so, I can continue testing...but that really affects my workflow! There are multiple requests happening here. I assume the following, which may or may not be correct: - From your browser you call your app - Your app runs some call through the twitter API - Twitter servers process the call and send it back to your app - Your app returns processed code back to your browser Yes, this is correct. From the above processes your IP address is passed through by the Twitter API to the twitter service. I'd suggest try running your request from a completely different network and see what happens. I tried running it from a friend's computer. I get the same frequency of Error, but when he changes his computer's IP address, I'm suddenly able to run the app again... How can I shift the load to my webserver's IP (the one that's whitelisted) rather than each individual computer's IP? Is it possible with Search API? Thank you!
[twitter-dev] Re: Search API rate limit
Hi, There is a rate limit for the Search API but it is higher than 100 requests per second imposed by the REST API. The limiting is performed by the IP address. The default limit is high enough that most applications shouldn't be affected. As the search architecture has no notion of accounts, it would be difficult to add account-based limiting. We do however whitelist IP addresses in the event that the higher limit is warranted. Doug Williams Twitter API Support http://twitter.com/dougw On Sat, Mar 14, 2009 at 11:14 AM, benjamin brande...@gmail.com wrote: I've noticed that the Search API page has recently been changed to say: The standard API rate limiting is liberal enough for most use cases. If you find yourself encountering these limits, please contact us and describe your app's requirements. Recently, this page stated that the rate for search is not limited. A week ago, Alex Payne stated here that there is not limit on the search API use; is this still the case, and the wording on the API Wiki is irrelevant? Or has the 100 req/hr limit been imposed on the search API as well? If the search API has been limited, is there a possibility of adding an authenticated search so that developers can apply for rate limit increases via their accounts, rather than a static IP that may not be feasible for their application? Thanks.
