[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-13 Thread owkaye
> I concur with Matt. > > Track in the Streaming API is, in part, intended for > applications just like yours. Hit the Search API and use > track together to get the highest proportion of statuses > possible. The default track limit is intended for human > readable scale applications. Email me abo

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-13 Thread owkaye
> We tried allowing access to follower information in a > one-query method like this and it failed. The main reason > is that when there are tens of thousands of matches > things start timing out. While all matches sounds like a > perfect solution, in practice staying connected for > minutes at a

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-13 Thread John Kalucki
I concur with Matt. Track in the Streaming API is, in part, intended for applications just like yours. Hit the Search API and use track together to get the highest proportion of statuses possible. The default track limit is intended for human readable scale applications. Email me about elevated t

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-13 Thread Matt Sanford
Hi there, Some comments in-line: On Jul 13, 2009, at 8:51 AM, owkaye wrote: First, I wouldn't expect that thousands are going to post your promo code per minute. That doesn't seem realistic. Hi John, It's more than just a promo code. There are other aspects of this promotion that mig

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-13 Thread owkaye
> First, I wouldn't expect that thousands are going to post > your promo code per minute. That doesn't seem realistic. Hi John, It's more than just a promo code. There are other aspects of this promotion that might create an issue with thousands of tweets per minute. If it happens and I have

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread David Fisher
Err, the Search API isn't limited to 150 requests per hour. It's much higher than that. Much, but not unlimited. As John said, read into the Search API more, and check into the Streaming API as well. It is certainly possible to get more than 1,500 results for a term, but not by using simple pagi

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread John Kalucki
First, I wouldn't expect that thousands are going to post your promo code per minute. That doesn't seem realistic. Second, in addition to the Search API, which is quite liberal, you can use the /track method on the Streaming API, which will return all keyword matches up to a certain limit with no

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread Scott Haneda
You are correct, you have to do 15 requests. However, you can cache the results in your end, so when you come back, you are only getting the new stuff. Thanks Scott. I'm storing the results in a database on my server but that doesn't stop the search from retrieving the same results repetitiv

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread owkaye
> You are correct, you have to do 15 requests. However, > you can cache the results in your end, so when you come > back, you are only getting the new stuff. Thanks Scott. I'm storing the results in a database on my server but that doesn't stop the search from retrieving the same results repeti

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread owkaye
Thanks Chad, that's what I was afraid of. I wonder if you know about this next question: Twitter API docs say search is rate limited to something more than REST which is 150 requests per hour, but for the sake of argument let's say the search rate limit is actually 150 hits per hour ... Since I

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread Scott Haneda
You are correct, you have to do 15 requests. However, you can cache the results in your end, so when you come back, you are only getting the new stuff. Twitter has pretty good date handling, so you specify your last date, and pull forward from there. You may even be able to get the last

[twitter-dev] Re: How to insure that all tweets are retrieved in a search?

2009-07-09 Thread Chad Etzel
Yep, you gotta do 15 requests at 100 rpp each. -Chad On Thu, Jul 9, 2009 at 5:45 PM, owkaye wrote: > > I'm building an app that uses the atom search API to retrieve recent > posts which contain a specific keyword.  The API docs say: > > "Clients may request up to 1,500 statuses via the page and r