I did some investigation into the snowflake algorithm recently and yes, it's
safe for 64bit signed longs. Even if Twitter moved away from using
scala/java longs internally (which are definitely signed), you'd still have
something like 65 years from now before the algorithm rolled past the 2^63-1
barrier.

I've posted a a gist[1] in ruby with a few little methods for playing with
the time part of a snowflake id if you're interested.

Hayes

1 http://gist.github.com/634586

On Tue, Oct 19, 2010 at 6:27 AM, Dan Checkoway <dchecko...@gmail.com> wrote:

> I'm also patiently awaiting a response from twitter about this.  Are the
> ids sane for 64-bit *signed* long?
>
> Dan
>
>
> On Mon, Oct 18, 2010 at 9:08 PM, jon <jonhoff...@gmail.com> wrote:
>
>> Hi,
>>
>> You wrote that the IDs are "unsigned" 64 bit ints, but the IdWorker is
>> pumping out java Longs which are signed.  I'm assuming that was a
>> typo, but please clarify.
>>
>>
>> http://github.com/twitter/snowflake/blob/master/src/main/scala/com/twitter/service/snowflake/IdWorker.scala
>>
>> Thanks,
>>
>> - Jon
>>
>> On Oct 18, 8:19 pm, Matt Harris <thematthar...@twitter.com> wrote:
>> > Last week you may remember Twitter planned to enable the new Status ID
>> > generator - 'Snowflake' but didn't. The purpose of this email is to
>> explain
>> > the reason why this didn't happen, what we are doing about it, and what
>> the
>> > new release plan is.
>> >
>> > So what is Snowflake?
>> > ------------------------------
>> > Snowflake is a service we will be using to generate unique Tweet IDs.
>> These
>> > Tweet IDs are unique 64bit unsigned integers, which, instead of being
>> > sequential like the current IDs, are based on time. The full ID is
>> composed
>> > of a timestamp, a worker number, and a sequence number.
>> >
>> > The problem
>> > -----------------
>> > Before launch it came to our attention that some programming languages
>> such
>> > as Javascript cannot support numbers with >53bits. This can be easily
>> > examined by running a command similar to: (90071992547409921).toString()
>> in
>> > your browsers console or by running the following JSON snippet through
>> your
>> > JSON parser.
>> >
>> >     {"id": 10765432100123456789, "id_str": "10765432100123456789"}
>> >
>> > In affected JSON parsers the ID will not be converted successfully and
>> will
>> > lose accuracy. In some parsers there may even be an exception.
>> >
>> > The solution
>> > ----------------
>> > To allow javascript and JSON parsers to read the IDs we need to include
>> a
>> > string version of any ID when responding in the JSON format. What this
>> means
>> > is Status, User, Direct Message and Saved Search IDs in the Twitter API
>> will
>> > now be returned as an integer and a string in JSON responses. This will
>> > apply to the main Twitter API, the Streaming API and the Search API.
>> >
>> > For example, a status object will now contain an id and an id_str. The
>> > following JSON representation of a status object shows the two versions
>> of
>> > the ID fields for each data point.
>> >
>> > [
>> >   {
>> >     "coordinates": null,
>> >     "truncated": false,
>> >     "created_at": "Thu Oct 14 22:20:15 +0000 2010",
>> >     "favorited": false,
>> >     "entities": {
>> >       "urls": [
>> >       ],
>> >       "hashtags": [
>> >       ],
>> >       "user_mentions": [
>> >         {
>> >           "name": "Matt Harris",
>> >           "id": 777925,
>> >           "id_str": "777925",
>> >           "indices": [
>> >             0,
>> >             14
>> >           ],
>> >           "screen_name": "themattharris"
>> >         }
>> >       ]
>> >     },
>> >     "text": "@themattharris hey how are things?",
>> >     "annotations": null,
>> >     "contributors": [
>> >       {
>> >         "id": 819797,
>> >         "id_str": "819797",
>> >         "screen_name": "episod"
>> >       }
>> >     ],
>> >     "id": 12738165059,
>> >     "id_str": "12738165059",
>> >     "retweet_count": 0,
>> >     "geo": null,
>> >     "retweeted": false,
>> >     "in_reply_to_user_id": 777925,
>> >     "in_reply_to_user_id_str": "777925",
>> >     "in_reply_to_screen_name": "themattharris",
>> >     "user": {
>> >       "id": 6253282
>> >       "id_str": "6253282"
>> >     },
>> >     "source": "web",
>> >     "place": null,
>> >     "in_reply_to_status_id": 12738040524
>> >     "in_reply_to_status_id_str": "12738040524"
>> >   }
>> > ]
>> >
>> > What should you do - RIGHT NOW
>> > ----------------------------------------------
>> > The first thing you should do is attempt to decode the JSON snippet
>> above
>> > using your production code parser. Observe the output to confirm the ID
>> has
>> > not lost accuracy.
>> >
>> > What you do next depends on what happens:
>> >
>> > * If your code converts the ID successfully without losing accuracy you
>> are
>> > OK but should consider converting to the _str versions of IDs as soon as
>> > possible.
>> > * If your code has lost accuracy, convert your code to using the _str
>> > version immediately. If you do not do this your code will be unable to
>> > interact with the Twitter API reliably.
>> > * In some language parsers, the JSON may throw an exception when reading
>> the
>> > ID value. If this happens in your parser you will need to ‘pre-parse’
>> the
>> > data, removing or replacing ID parameters with their _str versions.
>> >
>> > Summary
>> > -------------
>> > 1) If you develop in Javascript, know that you will have to update your
>> code
>> > to read the string version instead of the integer version.
>> >
>> > 2) If you use a JSON decoder, validate that the example JSON, above,
>> decodes
>> > without throwing exceptions. If exceptions are thrown, you will need to
>> > pre-parse the data. Please let us know the name, version, and language
>> of
>> > the parser which throws the exception so we can investigate.
>> >
>> > Timeline
>> > -----------
>> > by 22nd October 2010 (Friday): String versions of ID numbers will start
>> > appearing in the API responses
>> > 4th November 2010 (Thursday) : Snowflake will be turned on but at ~41bit
>> > length
>> > 26th November 2010 (Friday) : Status IDs will break 53bits in length and
>> > cease being usable as Integers in Javascript based languages
>> >
>> > We understand this isn’t as seamless a transition as we had planned and
>> > appreciate for some of you this change requires an update to your code.
>> > We’ve tried to give as much time as possible for you to make the
>> migration
>> > and update your code to use the new string representations.
>> >
>> > Our own products and tools are affected by the change and we will be
>> making
>> > available any pre-parsing snippets we have created to ensure code
>> continues
>> > to work with the new IDs.
>> >
>> > Thanks for your support and understanding.
>> >
>> > ---
>> > @themattharris
>> > Developer Advocate, Twitterhttp://twitter.com/themattharris
>>
>> --
>> Twitter developer documentation and resources: http://dev.twitter.com/doc
>> API updates via Twitter: http://twitter.com/twitterapi
>> Issues/Enhancements Tracker:
>> http://code.google.com/p/twitter-api/issues/list
>> Change your membership to this group:
>> http://groups.google.com/group/twitter-development-talk
>>
>
>  --
> Twitter developer documentation and resources: http://dev.twitter.com/doc
> API updates via Twitter: http://twitter.com/twitterapi
> Issues/Enhancements Tracker:
> http://code.google.com/p/twitter-api/issues/list
> Change your membership to this group:
> http://groups.google.com/group/twitter-development-talk
>

-- 
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 
http://groups.google.com/group/twitter-development-talk

Reply via email to