[
https://issues.apache.org/jira/browse/AVRO-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201661#comment-16201661
]
liviu commented on AVRO-2088:
-----------------------------
Hi [~zi],
Sorry for this late answer; you were right, this is a DS bug/lack of
functionality.. even if you pass to DS a avsc with DECIMAL logical datatype the
DS will export the fields as BYTES but in schema is putting decimal logical
type instead of bytes , so the resulting avro file is wrong (not matching with
the source one)
I repeted the test, and the source value "3.12" value was exported by DS as
"3.12" and interpreted by Hive as 8586652.66
Thanks,
Liviu
> Decimal logicalType values serialized in hexidecimal vs decimal
> ---------------------------------------------------------------
>
> Key: AVRO-2088
> URL: https://issues.apache.org/jira/browse/AVRO-2088
> Project: Avro
> Issue Type: Task
> Reporter: liviu
>
> We use this schema for AVRO file:
> {code:java}
> "name":"col1",
> "type":["null",
> {
> "type":"bytes",
> "logicalType":"decimal",
> "precision":19,
> "scale":2
> }
> ]
> {code}
> - if we save data in avro using sqoop or hive (external table), the values
> are saved in hexadecimal format (ex. for 3.12 value is:
> {color:#d04437}*{"col1":{"bytes":"\u00018"}}*{color}
> - if we save the data in that avro file using datastage , the values are
> saved in decimal format (ex. for 3.12 the saved value is:
> {color:#d04437}*{"col1":{"bytes":"3.12"}}*{color}
> The questions are:
> 1). why there is this differences, in one case the data is serialised using
> hexidecimal and the other case using decimal?
> 2). are these differences caused by Avro serialization encoding used (for one
> case is used binary encoding, for the other case is used json encoding)?
> 3). how can we control how the values are serialized (ex. we want to have
> them as "3.12" instead of "\u00018")
> Thanks,
> Liviu
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
