@sv - Not quite what i am looking for. But thanks a lot for the link. I think firehose is the only way to go, but thats something not very much in control. @jkalucki - Thanks for your info. I will move to streaming API. I did not know the current approach would be frowned upon. And hopefully hosebird would come out of its cocoon and serve us soon. Search API would not fit into my requirements.
Thanks a lot. On Jul 17, 9:38 pm, SV <shl...@gmail.com> wrote: > This could help -http://www.flotzam.com/archivist/ > > On Jul 17, 6:57 am, CreativeEye <creativv...@gmail.com> wrote: > > > > > Myself and my friend are doing a research based on twitter. We need to > > analyse each and every tweet real time. Can you guide how to approach > > this. > > > There could be 2 ways of doing this (without Firehose): > > > 1) Get Twitter Public timeline repeatedly. > > > Thankfully Twitter's caching has not been problem to me, they seem to > > fetch me new data every request. But there are a lot of limitation for > > this: > > > According to TweeSpeed.com: > > - Rate of New tweets in the Twitter Server is right now (Wed Jul 17 > > 11:47:02 - GMT) at 9233 tweets/minute. > > - Ranges between 7K to 20K on an average Weekday. > > - On June 26 (MJ's death) - reached 25K tweets/minute. > > > Let us now consider the limitation of API requests per hour. > > - Currently @ 20K per hour. > > - 1 Req = 20 Tweets > > - Need 1K Req per minute = 60K req per hour. > > > To Use 1K Requests per minute, we should be using around 17 requests > > per second. But my server is able to process only 28-33 requests/ > > minute. > > > Is this the right way to proceed, or am I fundamentally wrong on the > > approach. > > > 2) Get follower network - user profiles and get their statuses. > > Frequency of request their new status updates could be set against > > their general update frequency. But this is Google-like old way of > > indexing things, which does not quite stand today in the REAL TIME > > twitter. > > > I do know Firehose is an option, but that would again be something > > like Approach 1. right? > > > Please guide me how to proceed.