Historical queries are always going to be the strong point of the Search
API. You should perform initial queries against Search to collect the
history, then transition to Streaming for the duration of the query demand.

The Streaming API offers a few minutes of backlog to allow connections to
cycle, request the last few minutes of statuses, and avoid missing data. The
status count is a proxy for seconds that you were disconnected. If you know
the approximate rate of statuses created per second, and how many seconds
you were off, you can adjust the backlog more finely. This is easiest to see
at the Firehose level, otherwise you have to overrequest and deduplicate.

If you are using a filtered resource at an elevated access level, you
specify how many statuses to examine, not the number of results. The idea is
the same- to paper over the time between connections, not specify a result
set size.

We only offer the count parameter on the firehose and higher access levels
of follow. It's probably too expensive to run for track and counter to
the philosophy of the sample feeds. (It probably should work on the Retweet
and Links stream, but it doesn't.) If there's popular demand, we can look
into filling these gaps.

-John Kalucki
Services, Twitter Inc.

On Tue, Jan 12, 2010 at 8:11 AM, Edward Hinchliffe <
redders6...@googlemail.com> wrote:

> Thanks John, that's been helpful. It looks like storing tweets is fine...
> On related note, the documentation suggests that tweets using the "search
> API" (REST), are time limited to ~1.5 weeks.
> I have read that developers are now encouraged to use the streaming API. I
> see that it's possible to get historic tweets from the streaming API, but
> reading the documentation quoted below, I get the impression that this is
> limited to 150,000 tweets *before* they have been filtered. So if I searched
> for a term that has not been tweeted in the last 150,000 tweets, I'll get no
> historic results? Or have I interpreted that completely wrong? :)
> ~~~
> count
> Indicates the number of previous statuses to consider for delivery before
> transitioning to live stream delivery. On unfiltered streams, all considered
> statuses are delivered, so the number requested is the number returned. On
> filtered streams, the number requested is the number of statuses that are
> applied to the filter predicate, and not the number of statuses returned.
> *Values:* -150,000 to 150,000. This range is subject to change on short
> notice. Positive values transition seamlessly to the live stream. Negative
> values terminate when the historical stream has finished, useful for
> debugging.
> ~~~
> Thanks in advance,
> ~redders.
> 2010/1/12 John Kalucki <j...@twitter.com>
> The following also apply:
>> http://twitter.com/tos
>> http://twitter.com/apirules
>> http://help.twitter.com/forums/26257/entries/18311
>> On Tue, Jan 12, 2010 at 5:56 AM, redders <redders6...@googlemail.com>wrote:
>>> Hi All,
>>> Just wanted to ask  a quick question regarding historic tweets. Is
>>> there anything in the twitter T&Cs to say that search results can't be
>>> stored?
>>> I ask because the API only returns results from the last week (?) or
>>> so... what if I want to use data from previous searches I've
>>> conducted.
>>> Also, is there any other T&C doc other than this page:
>>> http://apiwiki.twitter.com/Terms-of-Service
>>> Cheers,
>>> ~redders.

Reply via email to