[twitter-dev] Re: What does tweet id means?
So that means we can not infer how many tweets are being sent from tweet id. I was wondering who is counting daily number of tweets. Of course Twitter is doing this itself, but the result goes public very late. On Apr 20, 8:24 pm, Tim Meadowcroft meer...@gmail.com wrote: I think you can only really rely on IDs having different values. In general, at the moment with Twitter, you could assume they increase over time, but (and I don't work for Twitter) typically ID allocation on large multihost systems don't work by allocating strictly sequential IDs without gaps - it's too hard to sequence and not really necessary. So, for example, one way is that you build a system that gives different ID-assigning-hosts small blocks of IDs that they can use so they can allocate a series of IDs knowing they're unique without having to take out any kind of global lock (they only take the lock to ask for a new block every now and then). Another approach might be to have clocks synchronised to some known accuracy and have IDs calculated as period-since-epoch * some-suitable-multiplier + unique-offset-per-host + incrementing-counter-for-this-host. I'm sure people can come up with other schemes as quick as we could type them up, but in general you make your ID space many orders of magnitude bigger than you strictly need, and in return you gain some flexibility in the criteria needed for quick and cheap unique allocation in a distributed system. But I wouldn't assume that every possible ID value is necessarily allocated. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: What does tweet id means?
Hey Leo, We're indeed using Snowflake (a Tweet ID generation service we developed). Tweets IDs are no longer sequential (but k-sorted with k = 1 second), and there is no way to count the total number of tweet sent every day. More info about Snowflake on our Engineering Blog: http://engineering.twitter.com/2010/06/announcing-snowflake.html Arnaud / @rno http://twitter.com/rno On Wed, Apr 20, 2011 at 10:22 PM, Leo leowll@gmail.com wrote: So that means we can not infer how many tweets are being sent from tweet id. I was wondering who is counting daily number of tweets. Of course Twitter is doing this itself, but the result goes public very late. On Apr 20, 8:24 pm, Tim Meadowcroft meer...@gmail.com wrote: I think you can only really rely on IDs having different values. In general, at the moment with Twitter, you could assume they increase over time, but (and I don't work for Twitter) typically ID allocation on large multihost systems don't work by allocating strictly sequential IDs without gaps - it's too hard to sequence and not really necessary. So, for example, one way is that you build a system that gives different ID-assigning-hosts small blocks of IDs that they can use so they can allocate a series of IDs knowing they're unique without having to take out any kind of global lock (they only take the lock to ask for a new block every now and then). Another approach might be to have clocks synchronised to some known accuracy and have IDs calculated as period-since-epoch * some-suitable-multiplier + unique-offset-per-host + incrementing-counter-for-this-host. I'm sure people can come up with other schemes as quick as we could type them up, but in general you make your ID space many orders of magnitude bigger than you strictly need, and in return you gain some flexibility in the criteria needed for quick and cheap unique allocation in a distributed system. But I wouldn't assume that every possible ID value is necessarily allocated. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: What does tweet id means?
I think you can only really rely on IDs having different values. In general, at the moment with Twitter, you could assume they increase over time, but (and I don't work for Twitter) typically ID allocation on large multihost systems don't work by allocating strictly sequential IDs without gaps - it's too hard to sequence and not really necessary. So, for example, one way is that you build a system that gives different ID-assigning-hosts small blocks of IDs that they can use so they can allocate a series of IDs knowing they're unique without having to take out any kind of global lock (they only take the lock to ask for a new block every now and then). Another approach might be to have clocks synchronised to some known accuracy and have IDs calculated as period-since-epoch * some-suitable-multiplier + unique-offset-per-host + incrementing-counter-for-this-host. I'm sure people can come up with other schemes as quick as we could type them up, but in general you make your ID space many orders of magnitude bigger than you strictly need, and in return you gain some flexibility in the criteria needed for quick and cheap unique allocation in a distributed system. But I wouldn't assume that every possible ID value is necessarily allocated. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: What does tweet id means?
On Wed, Apr 20, 2011 at 6:24 AM, Tim Meadowcroft meer...@gmail.com wrote: I think you can only really rely on IDs having different values. In general, at the moment with Twitter, you could assume they increase over time, but (and I don't work for Twitter) typically ID allocation on large multihost systems don't work by allocating strictly sequential IDs without gaps - it's too hard to sequence and not really necessary. Yes, this is how they do it. https://github.com/twitter/snowflake /damon -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
