Comments inline.
On Aug 21, 2008, at 4:57 PM, Chad Walters wrote:
Looking at things from the standpoint of efficiency also argues for
a limited set of modifiers, since we need to actually pass this
information on the wire.
In THRIFT-110, I suggest that we reserve 2 bits at the top of the
type byte for modifiers. This allows for 3 different modifiers for
any given primary type (plus a default).
I'm fine with suggesting we keep the list short, but isn't saying we
keep 2 bits free an implementation concern for protocols?
For integer types, I suggested that we add "fixed" and
"zipper"/"zigzag". The default assumption would be that a variable
encoding could be used. "fixed" would mean that variable encoding
is unlikely to be helpful, and "zipper"/"zigzag" would suggest a
variable encoding suitable for fields likely to contain a mix of
negative and positive values.
I presume this would mean that the default modifier should be
interpreted as "variable", and that protocols can feel free to
compress all ints by default.
For string types, I suggested that we add "extern". Strings marked
with this modifier could be stored in some separate indexed table
and referenced by index when passed through the protocol.
Does this sound like a good starting point? Are there other
modifiers we'd like to add?
Chad
On 8/21/08 4:41 PM, "Mark Slee" <[EMAIL PROTECTED]> wrote:
I'd propose starting with a fixed set of type hints that we agree
upon.
We could certainly implement a dynamic any-string goes type of thing.
But here's my concern with that:
1/ If we agree upon a fixed set of modifiers, we're more likely to get
them implemented across more languages since deciding on the set will
involve getting some basic agreement amongst the Thrift community.
We're
likely to retain more overall compatibility.
2/ If the modifiers are dynamic, it implies that people would make
their
own local changes, and write their own custom Protocol
implementations.
We're more likely to duplicate this work or disagree on things. This
will make custom protocol submissions back into the trunk more
difficult, and the core project becomes more likely to lose quality
work.
So, while a dynamic implementation is arguably more elegant/
flexible, I
think having the community agree upon a reasonable set of type
annotations is most healthy for the coherence of Thrift in the long
run.
Cheers,
Mark
-----Original Message-----
From: Bryan Duxbury [mailto:[EMAIL PROTECTED]
Sent: Thursday, August 21, 2008 3:40 PM
To: thrift-dev@incubator.apache.org
Subject: Re: Adding a "vint" type to Thrift
I'm not opposed to using a type modifier approach instead of a
separate
type. Will we really be adding "arbitrary" annotations to the objects,
or will they be of a specific nature? That is, will we have modifiers
like "fixed", "variable", "extern", etc. be compiler tokens, or
will you
just be allowed to add any string you want as a modifier before the
type? The latter would certainly be more flexible, but would be more
complicated as well.
-Bryan
On Aug 21, 2008, at 2:07 PM, David Reiss wrote:
First, I don't understand why it would be limited to 64-bit. What if
I have an integer that I want represented as 32 bits in memory but
serialized with a variable-length encoding.
Second, I wonder if creating a new type is the best solution. I've
been thinking for a while that it might be useful to add arbitrary
annotations to types. We already have a few special cases of this
(cpp_type in C++, slist in Erlang), and vint would be a third. I
think it might be time to bite the bullet and implement
annotations so
we can stop adding new syntactic constructs for each minor extension
to the data model.
--David
Bryan Duxbury wrote:
All,
We've been following the discussion on THRIFT-110. I think as a
first
step we should add a "vint" (variable integer) to Thrift.
My general proposal is that we add vint as a new type. Protocols
that
don't do variable length integers can treat vints as i64s. Protocols
that do implement variable length integers can choose encodings that
make sense for them.
We'd have to change the compiler and all the generators and
libraries
to recognize vint, if for no other reason than to treat it as an
i64.
However, this would make implementing other protocols that included
variable length integers very easy.
Are there any objections to this proposal? If not, I'll make an
issue, and Rapleaf is willing to commit time to this enhancement.
-Bryan
