[
https://issues.apache.org/jira/browse/AVRO-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723297#action_12723297
]
Scott Carey commented on AVRO-49:
---------------------------------
On average, this will waste one bit per int if all numbers are positive.
Performance should be slightly lower as well due to the extra bit shift
(perhaps imperceptibly so).
I do find that "id" types are extremely common, although a third common type is
one that is positive for almost all values, and occasionally contains small
negative numbers (most often, -1).
However perhaps the most common case will be the length fields in all "string"
and "bytes" fields, which by definition are always positive or zero, but are
encoded as if they could be negative.
Although not optimal, a client can 'pack' their positive number data into the
current format if space is a huge concern: If even, bit shift to the right one
and set the sign bit indicating negative on write, and on read multiply
negative numbers by two and flip the sign.
The length fields of strings and bytes can even do this -- although its
somewhat of a waste to rotate the int one way in the client and then back again
in the low level encode.
Perhaps the best thing is to do the reverse and pack unsigned internally, and
have the "int" and "long" types pack the sign bit?
The other variable length type I can think of is a Numeric data of arbitrary,
exact precision (Think BigDecimal)
It is possible to pack something like that into the bytes type though.
> Efficient positive varints
> --------------------------
>
> Key: AVRO-49
> URL: https://issues.apache.org/jira/browse/AVRO-49
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Ismael Juma
>
> Hi,
> Avro looks like an interesting project, so I was looking at the
> specification. I noticed that varints use zig-zag encoding which is a good
> signed representation. However, id-like values are very common and they are
> often always positive. I think it would be nice to support them as
> efficiently as possible too (like Protocol Buffers, for example). Would
> something like this be considered at all?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.