On 10/20/10 8:12 AM, M. Edward (Ed) Borasky wrote:
>> 6) Why not restrict IDs to 53bits?
>> A Snowflake ID is composed:
>>  * 41bits for millisecond precision time (69 years)
>>  * 10bits for a configured machine identity (1024 machines)
>>  * 12bits for a sequence number (4096 per machine)
>> The factor influencing the length of the ID is the time. For a 53bit
>> ID this
>> would mean only 31bits are available for the time. 31bits is only
>> enough for
>> 24 days (2147483648/(1000*60*60*24)) of time.
>> Reducing the resolution of the timestamp would prevent a K-sorted
>> resolution
>> of 1 second or less.
>> Reducing the configured machine identity or sequence number by 1bit would
>> mean we couldn’t scale Twitter, or operate our infrastructure in an
>> uncoordinated high-available way.
> Interesting ... so you have the theoretical capacity to scale to 2**22
> (about 4 million) tweets per millisecond? Even 4 million tweets a second
> seems unrealistic, as does a single "machine" only being able to
> generate 4096 IDs. I think if you're really expecting this kind of
> volume, the FPGA vendors probably can help you out. We are talking
> clocks and counters, here, right, not Javascript interpreters or robust
> linear regressions? ;-)
> Ah, well, I'll check back on you guys in 69 years to see how you're
> holding up. ;-)

I'd say that you could remove a maximum of 2 bits from the time - this
would divide the 69 years by 4, making it a max of 17 years. By then,
I'd assume that we are past the 53-bit limit.

You could remove 3 bits from the machine ID and 5 bits from the sequence
number. It would mean that there could be only 128 ID servers with 128
IDs per millisecond per machine -> 16 million tweets per second.

In total you would have removed 10 bits from a number that had only 63
bits -> 53 bits. The question is: do you want that? I don't think you
do. I really prefer the current solution.


Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 

Reply via email to