[twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
So it would be cool if some way were provided for me to gauge tweet volumes at regular intervals (currently every 2 minutes). Take a look to Tweespeed http://www.tweespeed.com But with the change annonced, this site is dead at term ... pas...@tweespeed On Mar 26, 10:01 pm, jerememonteau m...@jmoe.com wrote: Whoops, accidentally just replied to author the first time...but... I build this little site about 9 months ago, depending on the monotonically increasing nature of tweet IDs : http://www.tweelocity.com This is a fun graph : http://tweelocity.com/chart/60/300/ So it would be cool if some way were provided for me to gauge tweet volumes at regular intervals (currently every 2 minutes). I also think it's super cool that the twitter team is even giving a heads up like this. On Mar 26, 1:41 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: Hi Developers, It's no secret that Twitter is growing exponentially. The tweets keep coming with ever increasing velocity, thanks in large part to your great applications. Twitter has adapted to the increasing number of tweets in ways that have affected you in the past: We moved from 32 bit unsigned integers to 64-bit unsigned integers for status IDs some time ago. You all weathered that storm with ease. The tweetapoclypse was averted, and the tweets kept flowing. Now we're reaching the scalability limit of our current tweet ID generation scheme. Unlike the previous tweet ID migrations, the solution to the current issue is significantly different. However, in most cases the new approach we will take will not result in any noticeable differences to you the developer or your users. We are planning to replace our current sequential tweet ID generation routine with a simple, more scalable solution. IDs will still be 64-bit unsigned integers. However, this new solution is no longer guaranteed to generate sequential IDs. Instead IDs will be derived based on time: the most significant bits being sourced from a timestamp and the least significant bits will be effectively random. Please don't depend on the exact format of the ID. As our infrastructure needs evolve, we might need to tweak the generation algorithm again. If you've been trying to divine meaning from status IDs aside from their role as a primary key, you won't be able to anymore. Likewise for usage of IDs in mathematical operations -- for instance, subtracting two status IDs to determine the number of tweets in between will no longer be possible. For the majority of applications we think this scheme switch will be a non-event. Before implementing these changes, we'd like to know if your applications currently depend on the sequential nature of IDs. Do you depend on the density of the tweet sequence being constant? Are you trying to analyze the IDs as anything other than opaque, ordered identifiers? Aside for guaranteed sequential tweet ID ordering, what APIs can we provide you to accomplish your goals? Taylor Singletary Developer Advocate, Twitterhttp://twitter.com/episod To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Re: Almost real time
Repeated automated searches should not be run against search.twitter.com So what would you recommend as search method to get potentially huge amount of tweets (note that I need AND and OR criterias) Thanks On Mar 23, 12:42 am, John Kalucki j...@twitter.com wrote: Pick the lowest frequency term to track upon, then post-filter on your end. Repeated automated searches should not be run against search.twitter.com -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Mon, Mar 22, 2010 at 3:30 PM, Alberty Pascal alberty.pas...@gmail.comwrote: Hi all, while it's not possible to use AND search predicate in filter streaming, how it could be possible to get back almost real time search results ? My goal is to be sure that I got all the tweets for a complex search as fast as possible. Get a small amount (rpp) of tweets very often (every 10 seconds ?) ? Work with pagination ? How to be sure that I got every tweets ? Thanks To unsubscribe from this group, send email to twitter-development-talk+ unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject. To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Re: Almost real time
Thanks ! On Mar 23, 2:34 pm, John Kalucki j...@twitter.com wrote: The Streaming API is designed for bulk operations. You can retrieve millions of tweets per day with the default access level, but you may need elevated access to meet your requirements. You will have to do the AND on your end. Contact a...@twitter.com to begin the process for elevated access. -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Mar 23, 2010 at 4:02 AM, Alberty Pascal alberty.pas...@gmail.comwrote: Repeated automated searches should not be run against search.twitter.com So what would you recommend as search method to get potentially huge amount of tweets (note that I need AND and OR criterias) Thanks On Mar 23, 12:42 am, John Kalucki j...@twitter.com wrote: Pick the lowest frequency term to track upon, then post-filter on your end. Repeated automated searches should not be run against search.twitter.com -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Mon, Mar 22, 2010 at 3:30 PM, Alberty Pascal alberty.pas...@gmail.comwrote: Hi all, while it's not possible to use AND search predicate in filter streaming, how it could be possible to get back almost real time search results ? My goal is to be sure that I got all the tweets for a complex search as fast as possible. Get a small amount (rpp) of tweets very often (every 10 seconds ?) ? Work with pagination ? How to be sure that I got every tweets ? Thanks To unsubscribe from this group, send email to twitter-development-talk+ unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject. To unsubscribe from this group, send email to twitter-development-talk+ unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject. To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Almost real time
Hi all, while it's not possible to use AND search predicate in filter streaming, how it could be possible to get back almost real time search results ? My goal is to be sure that I got all the tweets for a complex search as fast as possible. Get a small amount (rpp) of tweets very often (every 10 seconds ?) ? Work with pagination ? How to be sure that I got every tweets ? Thanks To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
