Thanks a lot for your reply, Bjoern.

***from twitter api wiki***

Returns the 20 most recent statuses from non-protected users who have
set a custom user icon. The public timeline is cached for 60 seconds
so requesting it more often than that is a waste of resources.

***end of api wiki****

So it is not random according to the documentation. The docs say that
they do cache the responses for about a minute, yet i can see new data
every 2 seconds.(NOT enough).
And the server has already been whitelisted. 20K requests is not gonna
be enough for this anyway.

Sneaking into - I am not sure that would solve any of the problems.

On Jul 17, 5:28 pm, Bjoern <> wrote:
> On Jul 17, 1:57 pm, CreativeEye <> wrote:
> > 1) Get Twitter Public timeline repeatedly.
> My understanding is that this does not give you all tweets, just a
> random selection.
> > 2) Get follower network - user profiles and get their statuses.
> You would reach the API limit quickly, I'd expect.
> I don't remember the robots.txt definition very well, but I think
> twitter also disallows classic web crawlers:
> > I do know Firehose is an option, but that would again be something
> > like Approach 1. right?
> Firehose is only an option if Twitter allows you to use it.
> > Please guide me how to proceed.
> I think there is no reliable way to get ALL tweets, though I would be
> pleased to learn otherwise. (with the exception of the Firehose, which
> I suppose one can not plan for).
> Maybe by being "sneaky" about it one can get a lot of tweets. For
> example by getting people to use your service to access twitter, so
> that you are using up their API limits, not your own. Or at least get
> the service whitelisted so that you can make lots of requests (I doubt
> they would be enough to get ALL tweets, though).

Reply via email to