You may want to take a look at http://datasift.net/
-Stuart -- Stuart Dallas 3ft9 Ltd http://3ft9.com/ On Monday, 11 April 2011 at 16:14, Corey Ballou wrote: > I tried speaking with Ryan Sarver directly, but he's forwarding me > here to the community advocates to answer. I believe this answer will > need to come top down from Twitter, as it's your rate limiting that > I'm most worried about. > > I have a technical question for all of you in regards to the Search > API as I want to maintain full compliancy. Currently, the old Search > API implementation (albeit slower) provides a fuller result set and > allows for more flexibility in the types and combinations of searches > allowed. The manner I have developed my application would allow for a > number of daemonized worker instances running on different IP > addresses to make calls to the search API on behalf of the stored > OAuth credentials to avoid rate limiting issues. > > I had a conversation with the Pluggio developer in which he stated > Twitter had threatened to shutdown his application if he didn't switch > to a different implementation of the Search API. The problem indicated > was that he was performing searches for multiple Twitter accounts, > which is exactly my use case. Site streams does not make as much sense > for my application given the search queries I wish to perform and the > necessity for logical AND operations on geo-location. > > Do you foresee any problems with my current method of using different > IP addresses to stay under the rate limit? I'm trying to stay in full > compliance with Twitter's TOS and would love to find the most > applicable and API friendly solution. I know headway is being made > with Twitter's new search implementation so I would like to stay ahead > of the curve and not get myself stuck in a box. > > I still need a method for polling for new search results (say, every > 30 minutes, dependent upon the pricing plan) for non-logged in users. > > Below is a scaled down representation of how I'm currently handling > searches to help you decide the best plan of action: > > 1) Searches are performed on a rolling queue basis, say one search > every thirty minutes. There can be a finite number of searches per > Twitter user (say 5 searches per Twitter account). There can be any > number of Twitter accounts. > 2) Search results are stored locally for retrieval by a javascript > AJAX long-poller every minute to check for frequent changes. > 3) When a user visits the search results page and filters results, no > API calls to Twitter are made, only a local query is required > > Due to this process, the queue is constantly searching for the next > searches and mentions to perform. I foresee rate limiting concerns > cropping up with searches being performed for any number of users. > > Can you steer me in the right direction to avoid shutdown notices or > access revocation? > > Regards, > > Corey > @cballou > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
