I'm also patiently awaiting a response from twitter about this.  Are the ids
sane for 64-bit *signed* long?

Dan

On Mon, Oct 18, 2010 at 9:08 PM, jon <jonhoff...@gmail.com> wrote:

> Hi,
>
> You wrote that the IDs are "unsigned" 64 bit ints, but the IdWorker is
> pumping out java Longs which are signed.  I'm assuming that was a
> typo, but please clarify.
>
>
> http://github.com/twitter/snowflake/blob/master/src/main/scala/com/twitter/service/snowflake/IdWorker.scala
>
> Thanks,
>
> - Jon
>
> On Oct 18, 8:19 pm, Matt Harris <thematthar...@twitter.com> wrote:
> > Last week you may remember Twitter planned to enable the new Status ID
> > generator - 'Snowflake' but didn't. The purpose of this email is to
> explain
> > the reason why this didn't happen, what we are doing about it, and what
> the
> > new release plan is.
> >
> > So what is Snowflake?
> > ------------------------------
> > Snowflake is a service we will be using to generate unique Tweet IDs.
> These
> > Tweet IDs are unique 64bit unsigned integers, which, instead of being
> > sequential like the current IDs, are based on time. The full ID is
> composed
> > of a timestamp, a worker number, and a sequence number.
> >
> > The problem
> > -----------------
> > Before launch it came to our attention that some programming languages
> such
> > as Javascript cannot support numbers with >53bits. This can be easily
> > examined by running a command similar to: (90071992547409921).toString()
> in
> > your browsers console or by running the following JSON snippet through
> your
> > JSON parser.
> >
> >     {"id": 10765432100123456789, "id_str": "10765432100123456789"}
> >
> > In affected JSON parsers the ID will not be converted successfully and
> will
> > lose accuracy. In some parsers there may even be an exception.
> >
> > The solution
> > ----------------
> > To allow javascript and JSON parsers to read the IDs we need to include a
> > string version of any ID when responding in the JSON format. What this
> means
> > is Status, User, Direct Message and Saved Search IDs in the Twitter API
> will
> > now be returned as an integer and a string in JSON responses. This will
> > apply to the main Twitter API, the Streaming API and the Search API.
> >
> > For example, a status object will now contain an id and an id_str. The
> > following JSON representation of a status object shows the two versions
> of
> > the ID fields for each data point.
> >
> > [
> >   {
> >     "coordinates": null,
> >     "truncated": false,
> >     "created_at": "Thu Oct 14 22:20:15 +0000 2010",
> >     "favorited": false,
> >     "entities": {
> >       "urls": [
> >       ],
> >       "hashtags": [
> >       ],
> >       "user_mentions": [
> >         {
> >           "name": "Matt Harris",
> >           "id": 777925,
> >           "id_str": "777925",
> >           "indices": [
> >             0,
> >             14
> >           ],
> >           "screen_name": "themattharris"
> >         }
> >       ]
> >     },
> >     "text": "@themattharris hey how are things?",
> >     "annotations": null,
> >     "contributors": [
> >       {
> >         "id": 819797,
> >         "id_str": "819797",
> >         "screen_name": "episod"
> >       }
> >     ],
> >     "id": 12738165059,
> >     "id_str": "12738165059",
> >     "retweet_count": 0,
> >     "geo": null,
> >     "retweeted": false,
> >     "in_reply_to_user_id": 777925,
> >     "in_reply_to_user_id_str": "777925",
> >     "in_reply_to_screen_name": "themattharris",
> >     "user": {
> >       "id": 6253282
> >       "id_str": "6253282"
> >     },
> >     "source": "web",
> >     "place": null,
> >     "in_reply_to_status_id": 12738040524
> >     "in_reply_to_status_id_str": "12738040524"
> >   }
> > ]
> >
> > What should you do - RIGHT NOW
> > ----------------------------------------------
> > The first thing you should do is attempt to decode the JSON snippet above
> > using your production code parser. Observe the output to confirm the ID
> has
> > not lost accuracy.
> >
> > What you do next depends on what happens:
> >
> > * If your code converts the ID successfully without losing accuracy you
> are
> > OK but should consider converting to the _str versions of IDs as soon as
> > possible.
> > * If your code has lost accuracy, convert your code to using the _str
> > version immediately. If you do not do this your code will be unable to
> > interact with the Twitter API reliably.
> > * In some language parsers, the JSON may throw an exception when reading
> the
> > ID value. If this happens in your parser you will need to ‘pre-parse’ the
> > data, removing or replacing ID parameters with their _str versions.
> >
> > Summary
> > -------------
> > 1) If you develop in Javascript, know that you will have to update your
> code
> > to read the string version instead of the integer version.
> >
> > 2) If you use a JSON decoder, validate that the example JSON, above,
> decodes
> > without throwing exceptions. If exceptions are thrown, you will need to
> > pre-parse the data. Please let us know the name, version, and language of
> > the parser which throws the exception so we can investigate.
> >
> > Timeline
> > -----------
> > by 22nd October 2010 (Friday): String versions of ID numbers will start
> > appearing in the API responses
> > 4th November 2010 (Thursday) : Snowflake will be turned on but at ~41bit
> > length
> > 26th November 2010 (Friday) : Status IDs will break 53bits in length and
> > cease being usable as Integers in Javascript based languages
> >
> > We understand this isn’t as seamless a transition as we had planned and
> > appreciate for some of you this change requires an update to your code.
> > We’ve tried to give as much time as possible for you to make the
> migration
> > and update your code to use the new string representations.
> >
> > Our own products and tools are affected by the change and we will be
> making
> > available any pre-parsing snippets we have created to ensure code
> continues
> > to work with the new IDs.
> >
> > Thanks for your support and understanding.
> >
> > ---
> > @themattharris
> > Developer Advocate, Twitterhttp://twitter.com/themattharris
>
> --
> Twitter developer documentation and resources: http://dev.twitter.com/doc
> API updates via Twitter: http://twitter.com/twitterapi
> Issues/Enhancements Tracker:
> http://code.google.com/p/twitter-api/issues/list
> Change your membership to this group:
> http://groups.google.com/group/twitter-development-talk
>

-- 
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 
http://groups.google.com/group/twitter-development-talk

Reply via email to