Help.
I had been using BINARY(16) to store tweet id's in MYSQL. I can't find
a good resource to tell me the new data type I should use:
Reading the mysql documentation, I believe it should be unsigned
bigint - but would binary(64) be a better option?
I also read this comment against about BIGINT,
Signed bigint is what Snowflake itself uses.
Tom
On 12/3/10 7:26 PM, TCI wrote:
Help.
I had been using BINARY(16) to store tweet id's in MYSQL. I can't find
a good resource to tell me the new data type I should use:
Reading the mysql documentation, I believe it should be unsigned
bigint - but
Maybe this is a little naive, and I know you gotta' consider backwards
compatibility, but it seems like a bad idea to rely on status ID for
chronological sorting. If you want tweets displayed in order, sort by
the timestamp. If you get rid of the sorting requirement on the ID.
Why not do what Dean
I gather the reason for the 64-bit int type was to maintain some
backwards-compatibility around the old sequential IDs, so both the old-
style and Snowflake IDs could be sorted and you could glean that
smaller IDs are older than larger integers. U/GUIDs wouldn't be
sortable in any meaningful
Why not just use a GUID or UUID type for the ID type (IE:
3F2504E0-4F89-11D3-9A0C-0305E82C3301)? This way you're not restricted
by using a numeric data type that each language could potentially
define differently.
For languages that don't directly have a GUID or UUID type, they can
treat that ID
+1 for bumping the version number and maintaining backwards
compatibility.
On Oct 20, 2:23 pm, Josh Roesslein jroessl...@gmail.com wrote:
Isn't the point of having versioned API's so changes can be rolled out w/o
breaking a much of applications at once?
Why not increment to version 2 and
Thanks, Tom. I also hope the answer to my first question is No.
I asked the 2nd question because in Matt's message that started this
thread, he says:
* If your code converts the ID successfully without losing accuracy
you are
OK but should consider converting to the _str versions of IDs as soon
Hello. Couple questions:
1.) Are you planning on eventually eliminating the integer
representation and only using strings for id's?
2.) If an application doesn't use Javascript to parse JSON (for
example, YAJL-OBJC and NSNumbers in Obj-C), is it necessary to make
any changes at all?
Thanks.
1) Please don't, I don't want to have to convert everything back to
integers within my code. I consider the string representation a hack
around some issues with certain programming languages, and not an
optimal solution. Wouldn't want this to become the default option.
2) No
Tom
On 11/11/10
@Themattharris: was there any change to the implementation timeline?
Quote: by 22nd October 2010 (Friday): String versions of ID numbers
will start appearing in the API responses
I'm still not seeing id_str, to_user_id_str and from_user_id_str etc.
in the current search API output, example:
What about methods that list IDs without keys? e.g. blocks/ids which
produces a array like [ 12345, 6789, ]
These appear to be still cast as integers.
I don't see the point in having integer IDs at all. Surely the case of
people wishing to handle them as integers is far smaller than the case
It's there. But for some reason now in all types of json requests -
On Oct 25, 2010, at 6:28 AM, Johannes la Poutre wrote:
@Themattharris: was there any change to the implementation timeline?
Quote: by 22nd October 2010 (Friday): String versions of ID numbers
will start appearing in the API
The Search API hasn't rolled those fields in yet but they are due soon. The
API does include these fields but as Tim has highlighted the xyz/ids methods
don't yet have a String version.
The engineering teams are aware of this and are working on it. I'll post an
update once those additional
Hey everyone,
Further to this mornings email, the Search API now contains the _str
representations of it's IDs.
Best
@themattharris
Developer Advocate, Twitter
http://twitter.com/themattharris
On Mon, Oct 25, 2010 at 1:33 PM, Matt Harris thematthar...@twitter.comwrote:
The Search API hasn't
What's the point? That would still require a change in the JSON
output, to replace integers with strings.
On 21 oct, 15:38, David Nicol davidni...@gmail.com wrote:
Unless I did the math wrong, a 64 bit quantity is expressable in
(64 * log(2)) / log(62) = 10.7487219
eleven characters drawn
Unless I did the math wrong, a 64 bit quantity is expressable in
(64 * log(2)) / log(62) = 10.7487219
eleven characters drawn from A-Za-z0-9
and they can still be sortable!
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter:
6) Why not restrict IDs to 53bits?
A Snowflake ID is composed:
* 41bits for millisecond precision time (69 years)
* 10bits for a configured machine identity (1024 machines)
* 12bits for a sequence number (4096 per machine)
The factor influencing the length of the ID is the time. For a 53bit
On Oct 19, 5:19 am, Matt Harris thematthar...@twitter.com wrote:
So what is Snowflake?
--
Snowflake is a service we will be using to generate unique Tweet IDs. These
Tweet IDs are unique 64bit unsigned integers, which, instead of being
sequential like the current
Yes, you can.
Tom
On Oct 20, 2010, at 1:37 PM, Navin Kabra navin.ka...@gmail.com wrote:
On Oct 19, 5:19 am, Matt Harris thematthar...@twitter.com wrote:
So what is Snowflake?
--
Snowflake is a service we will be using to generate unique Tweet IDs. These
Tweet
Matt,
In your example you show an unsigned long that uses the full 64 bits
bits, which fails to be properly represented on a Java platform. In
a later explanation you say that there will only ever be 63 bits
used. Can you confirm one way or the other? i.e. will we ever see
an unsigned
Isn't the point of having versioned API's so changes can be rolled out w/o
breaking a much of applications at once?
Why not increment to version 2 and replace all ID's as strings in the JSON
format? Keep version 1 around for a few months
allowing everyone to upgrade and then kill it off. This can
Hi,
Will user ids be generated by snowflake in the near future? Is it
safe to parse and store them as signed 64bit integers?
Thanks.
On Oct 18, 8:34 pm, themattharris thematthar...@twitter.com wrote:
Thanks to @gotwalt for spotting the missing commas.
Fixed JSON sample ...
[
{
Hi,
You wrote that the IDs are unsigned 64 bit ints, but the IdWorker is
pumping out java Longs which are signed. I'm assuming that was a
typo, but please clarify.
http://github.com/twitter/snowflake/blob/master/src/main/scala/com/twitter/service/snowflake/IdWorker.scala
Thanks,
- Jon
On Oct
Out of curiosity; what advantage is there of including both the int
and the string? Why not only offer the string and put the parsing on
the client side?
Thanks,
@detroitpro
On Oct 18, 8:34 pm, themattharris thematthar...@twitter.com wrote:
Thanks to @gotwalt for spotting the missing commas.
Hi
In the case where an id is null (as in in_reply_to_status_id:null )
what will the value of in_reply_to_status_id_str be ?
Thanks
Xavier
On 19 oct, 02:34, themattharris thematthar...@twitter.com wrote:
Thanks to @gotwalt for spotting the missing commas.
Fixed JSON sample ...
[
{
Because a lot of Twitter applications (including my own) would go crazy
immediately.
Tom
On 10/19/10 6:09 AM, Detroitpro wrote:
Out of curiosity; what advantage is there of including both the int
and the string? Why not only offer the string and put the parsing on
the client side?
Thanks,
Probably null as well.
Tom
On 10/19/10 11:49 AM, Reivax wrote:
Hi
In the case where an id is null (as in in_reply_to_status_id:null )
what will the value of in_reply_to_status_id_str be ?
Thanks
Xavier
On 19 oct, 02:34, themattharris thematthar...@twitter.com wrote:
Thanks to
I'm also patiently awaiting a response from twitter about this. Are the ids
sane for 64-bit *signed* long?
Dan
On Mon, Oct 18, 2010 at 9:08 PM, jon jonhoff...@gmail.com wrote:
Hi,
You wrote that the IDs are unsigned 64 bit ints, but the IdWorker is
pumping out java Longs which are signed.
This approach feels wrong to me. The red flag is the duplication of
data within the payload: in 30+ years of professional development,
I've never seen that work out well.
The root of the problem is that you've chosen to deliver data in a
format (JSON) that can't support integers with a value
I did some investigation into the snowflake algorithm recently and yes, it's
safe for 64bit signed longs. Even if Twitter moved away from using
scala/java longs internally (which are definitely signed), you'd still have
something like 65 years from now before the algorithm rolled past the 2^63-1
I wouldn't blame this on JSON, because it's not JSON that has the
problems, but JavaScript. All of my Objective-C apps that communicate
use JSON as well, and they don't have the limitation. The issue does not
apply to XML either - there's no type specification in XML.
As far as I know, this issue
With all due respect, the root of the problem is that computer
scientists think in terms of abstract machines with infinitely-wide
registers, infinitely many addressable RAM cells, etc., and business
people think in terms of human populations and their tweet rates
growing geometrically for
You normally don't need all 53 bits - but as snowflake simply skips a
few IDs every now and then, you get a lot more IDs.
Chance of it actually reaching 53 bits? I'd say that it happens at the
end of November... Friday the 26th?
Tom
On 10/19/10 8:04 PM, M. Edward (Ed) Borasky wrote:
With all
This still gives you some API legacy that you need to maintain, but
it's a much cleaner approach than what is currently being proposed.
-ch
On Oct 19, 10:39 am, Tom van der Woerdt i...@tvdw.eu wrote:
I wouldn't blame this on JSON, because it's not JSON that has the
problems, but JavaScript.
It's kind of cool that Craig has been programming professionally
longer than some people on this list have even been alive!
I've thought about the problem, and at least came up with a catchy
name for it to go with the crises of yore: Bitpocalypse.
Since Twittelator uses the JSON library YAJL,
Hey everyone,
Thank you to all of you for your questions, patience and contributions to
this thread. Hearing your views and knowing how you use the API helps us
provide more information where there wasn't enough, and clarify details
where there was ambiguity.
I've collated the questions i've
Thanks to @gotwalt for spotting the missing commas.
Fixed JSON sample ...
[
{
coordinates: null,
truncated: false,
created_at: Thu Oct 14 22:20:15 + 2010,
favorited: false,
entities: {
urls: [
],
hashtags: [
],
user_mentions: [
{
37 matches
Mail list logo
