[
https://issues.apache.org/jira/browse/AVRO-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943156#comment-13943156
]
Tom White commented on AVRO-1402:
---------------------------------
> (Doug) For greatest interoperability we might use strings instead of bytes,
> with scientific notation.
Hive would probably prefer a binary representation for performance, but perhaps
others can comment on that.
Thanks for the pointers on supporting subtypes more fully - I can create
another patch for those. Regarding the incompatibility, I was thinking of an
application that processes Avro files that it didn't generate - if it upgraded
to the new version of Avro and read a file with a decimal subtype it would
receive a BigDecimal when it was only expecting a ByteBuffer.
> (Jarcec) Databases do not allows different rows for the same column to have
> different scale or precision.
The scale and precision specified in the column definition are maximums. My
understanding is that decimals with smaller scale and precision may be stored
in such columns. For example in Hive if I have a DECIMAL(5,3) column then 12.4
is stored as [124, 1] as the unscaled int-scale pair (so precision 3, scale 1),
and not as [12400, 3] So in effect there are per-record precision and scale
values, unless I'm missing something.
> (Jarcec) I'm wondering if it would make sense to store the precision in the
> Avro metadata as well?
Applications could set scale and precision attributes as metadata to describe
the (maximum) scale and precision of the decimals written to that field.
Would that be sufficient? I'm not sure if we want to define and implement a
policy for how to handle decimals that don't fit a max precision/scale in Avro.
> Support for DECIMAL primitive type
> ----------------------------------
>
> Key: AVRO-1402
> URL: https://issues.apache.org/jira/browse/AVRO-1402
> Project: Avro
> Issue Type: New Feature
> Affects Versions: 1.7.5
> Reporter: Mariano Dominguez
> Priority: Minor
> Labels: Hive
> Attachments: AVRO-1402.patch
>
>
> Currently, Avro does not seem to support a DECIMAL type or equivalent.
> http://avro.apache.org/docs/1.7.5/spec.html#schema_primitive
> Adding DECIMAL support would be particularly interesting when converting
> types from Avro to Hive, since DECIMAL is already a supported data type in
> Hive (0.11.0).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
