This could help - http://www.flotzam.com/archivist/
On Jul 17, 6:57 am, CreativeEye <creativv...@gmail.com> wrote:
> Myself and my friend are doing a research based on twitter. We need to
> analyse each and every tweet real time. Can you guide how to approach
> There could be 2 ways of doing this (without Firehose):
> 1) Get Twitter Public timeline repeatedly.
> Thankfully Twitter's caching has not been problem to me, they seem to
> fetch me new data every request. But there are a lot of limitation for
> According to TweeSpeed.com:
> - Rate of New tweets in the Twitter Server is right now (Wed Jul 17
> 11:47:02 - GMT) at 9233 tweets/minute.
> - Ranges between 7K to 20K on an average Weekday.
> - On June 26 (MJ's death) - reached 25K tweets/minute.
> Let us now consider the limitation of API requests per hour.
> - Currently @ 20K per hour.
> - 1 Req = 20 Tweets
> - Need 1K Req per minute = 60K req per hour.
> To Use 1K Requests per minute, we should be using around 17 requests
> per second. But my server is able to process only 28-33 requests/
> Is this the right way to proceed, or am I fundamentally wrong on the
> 2) Get follower network - user profiles and get their statuses.
> Frequency of request their new status updates could be set against
> their general update frequency. But this is Google-like old way of
> indexing things, which does not quite stand today in the REAL TIME
> I do know Firehose is an option, but that would again be something
> like Approach 1. right?
> Please guide me how to proceed.