I'm also patiently awaiting a response from twitter about this. Are the ids sane for 64-bit *signed* long?
Dan On Mon, Oct 18, 2010 at 9:08 PM, jon <[email protected]> wrote: > Hi, > > You wrote that the IDs are "unsigned" 64 bit ints, but the IdWorker is > pumping out java Longs which are signed. I'm assuming that was a > typo, but please clarify. > > > http://github.com/twitter/snowflake/blob/master/src/main/scala/com/twitter/service/snowflake/IdWorker.scala > > Thanks, > > - Jon > > On Oct 18, 8:19 pm, Matt Harris <[email protected]> wrote: > > Last week you may remember Twitter planned to enable the new Status ID > > generator - 'Snowflake' but didn't. The purpose of this email is to > explain > > the reason why this didn't happen, what we are doing about it, and what > the > > new release plan is. > > > > So what is Snowflake? > > ------------------------------ > > Snowflake is a service we will be using to generate unique Tweet IDs. > These > > Tweet IDs are unique 64bit unsigned integers, which, instead of being > > sequential like the current IDs, are based on time. The full ID is > composed > > of a timestamp, a worker number, and a sequence number. > > > > The problem > > ----------------- > > Before launch it came to our attention that some programming languages > such > > as Javascript cannot support numbers with >53bits. This can be easily > > examined by running a command similar to: (90071992547409921).toString() > in > > your browsers console or by running the following JSON snippet through > your > > JSON parser. > > > > {"id": 10765432100123456789, "id_str": "10765432100123456789"} > > > > In affected JSON parsers the ID will not be converted successfully and > will > > lose accuracy. In some parsers there may even be an exception. > > > > The solution > > ---------------- > > To allow javascript and JSON parsers to read the IDs we need to include a > > string version of any ID when responding in the JSON format. What this > means > > is Status, User, Direct Message and Saved Search IDs in the Twitter API > will > > now be returned as an integer and a string in JSON responses. This will > > apply to the main Twitter API, the Streaming API and the Search API. > > > > For example, a status object will now contain an id and an id_str. The > > following JSON representation of a status object shows the two versions > of > > the ID fields for each data point. > > > > [ > > { > > "coordinates": null, > > "truncated": false, > > "created_at": "Thu Oct 14 22:20:15 +0000 2010", > > "favorited": false, > > "entities": { > > "urls": [ > > ], > > "hashtags": [ > > ], > > "user_mentions": [ > > { > > "name": "Matt Harris", > > "id": 777925, > > "id_str": "777925", > > "indices": [ > > 0, > > 14 > > ], > > "screen_name": "themattharris" > > } > > ] > > }, > > "text": "@themattharris hey how are things?", > > "annotations": null, > > "contributors": [ > > { > > "id": 819797, > > "id_str": "819797", > > "screen_name": "episod" > > } > > ], > > "id": 12738165059, > > "id_str": "12738165059", > > "retweet_count": 0, > > "geo": null, > > "retweeted": false, > > "in_reply_to_user_id": 777925, > > "in_reply_to_user_id_str": "777925", > > "in_reply_to_screen_name": "themattharris", > > "user": { > > "id": 6253282 > > "id_str": "6253282" > > }, > > "source": "web", > > "place": null, > > "in_reply_to_status_id": 12738040524 > > "in_reply_to_status_id_str": "12738040524" > > } > > ] > > > > What should you do - RIGHT NOW > > ---------------------------------------------- > > The first thing you should do is attempt to decode the JSON snippet above > > using your production code parser. Observe the output to confirm the ID > has > > not lost accuracy. > > > > What you do next depends on what happens: > > > > * If your code converts the ID successfully without losing accuracy you > are > > OK but should consider converting to the _str versions of IDs as soon as > > possible. > > * If your code has lost accuracy, convert your code to using the _str > > version immediately. If you do not do this your code will be unable to > > interact with the Twitter API reliably. > > * In some language parsers, the JSON may throw an exception when reading > the > > ID value. If this happens in your parser you will need to ‘pre-parse’ the > > data, removing or replacing ID parameters with their _str versions. > > > > Summary > > ------------- > > 1) If you develop in Javascript, know that you will have to update your > code > > to read the string version instead of the integer version. > > > > 2) If you use a JSON decoder, validate that the example JSON, above, > decodes > > without throwing exceptions. If exceptions are thrown, you will need to > > pre-parse the data. Please let us know the name, version, and language of > > the parser which throws the exception so we can investigate. > > > > Timeline > > ----------- > > by 22nd October 2010 (Friday): String versions of ID numbers will start > > appearing in the API responses > > 4th November 2010 (Thursday) : Snowflake will be turned on but at ~41bit > > length > > 26th November 2010 (Friday) : Status IDs will break 53bits in length and > > cease being usable as Integers in Javascript based languages > > > > We understand this isn’t as seamless a transition as we had planned and > > appreciate for some of you this change requires an update to your code. > > We’ve tried to give as much time as possible for you to make the > migration > > and update your code to use the new string representations. > > > > Our own products and tools are affected by the change and we will be > making > > available any pre-parsing snippets we have created to ensure code > continues > > to work with the new IDs. > > > > Thanks for your support and understanding. > > > > --- > > @themattharris > > Developer Advocate, Twitterhttp://twitter.com/themattharris > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
