Jingwei Zhang created HUDI-1607:
-----------------------------------
Summary: Decimal handling bug in SparkAvroPostProcessor
Key: HUDI-1607
URL: https://issues.apache.org/jira/browse/HUDI-1607
Project: Apache Hudi
Issue Type: Bug
Reporter: Jingwei Zhang
This issue related to [#[Hudi-1343]|[https://github.com/apache/hudi/pull/2192].]
I think the purpose of Hudi-1343 was to bridge the different between avro
1.8.2(used by hudi) and avro 1.9.2(used by upstream system) thru internal
Struct type. In particular, the incompatible form to express nullable type
between those two versions.
It was all good until I hit the type Decimal. Since it can either be FIXED or
BYTES, if an avro schema contains decimal type with BYTES as its literal type,
after this two way conversion its literal type become FIXED instead. This will
cause an exception to be thrown in AvroConversionHelper as the data underneath
is ByteBuffer rather than GenericFixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
