On 08/16/2010 02:46 PM, Patrick Linehan wrote:
does anyone have any suggestions for dealing with large lists/arrays of
primitive values in avro?
in my case (numerical algorithms), my naive mapping of a vector type
(mathematical vectors, not java Vectors) to an avro specific type
generates a GenericArray<Double>. needless to say, i would prefer to
avoid the cost of boxing up all the individual floating point numbers.
is it possible to coerce avro into using raw java primitive arrays, e.g.
"double[]"?
It should be possible to subclass things to effect this. In particular:
- extend GenericData, overriding isArray()
- extend GenericDatumReader, overriding readArray()
- extend GenericDatumWriter, overriding writeArray()
or s/Generic/Specific/ if you're using generated classes.
Note that the reflect implementation already supports java primitive
arrays, but values are boxed on read and write. However if you write
type-specific loops for int, float, long and double arrays you should be
able to avoid any boxing.
Please tell me how this works for you.
Thanks,
Doug
