[
https://issues.apache.org/jira/browse/HUDI-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-6569:
----------------------------
Description:
When the input records have a enum field in the Avro schema, the write fails
due to:
{code:java}
29542 [Executor task launch worker for task 2.0 in stage 35.0 (TID 56)] ERROR
org.apache.hudi.io.HoodieCreateHandle [] - Error writing record
HoodieRecord{key=HoodieKey { recordKey=09959033-0742-4638-aca4-44162efa4b27
partitionPath=2015/03/17}, currentLocation='null', newLocation='null'}
org.apache.avro.AvroRuntimeException: Unknown schema type: ENUM
at
org.apache.hudi.avro.HoodieAvroUtils.rewritePrimaryType(HoodieAvroUtils.java:948)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchemaInternal(HoodieAvroUtils.java:895)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:830)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchemaInternal(HoodieAvroUtils.java:851)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:830)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:800)
~[classes/:?]
at
org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:784)
~[classes/:?]
at
org.apache.hudi.common.model.HoodieAvroRecord.prependMetaFields(HoodieAvroRecord.java:132)
~[classes/:?]
at
org.apache.hudi.io.HoodieCreateHandle.doWrite(HoodieCreateHandle.java:144)
~[classes/:?]
at org.apache.hudi.io.HoodieWriteHandle.write(HoodieWriteHandle.java:182)
~[classes/:?]
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:98)
~[classes/:?]
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:42)
~[classes/:?]
at
org.apache.hudi.common.util.queue.SimpleExecutor.execute(SimpleExecutor.java:68)
~[classes/:?]
at
org.apache.hudi.execution.SparkLazyInsertIterable.computeNext(SparkLazyInsertIterable.java:80)
~[classes/:?]
at
org.apache.hudi.execution.SparkLazyInsertIterable.computeNext(SparkLazyInsertIterable.java:39)
~[classes/:?]
at
org.apache.hudi.client.utils.LazyIterableIterator.next(LazyIterableIterator.java:119)
~[classes/:?] {code}
> Fix Hudi write failure for Avro enum type
> -----------------------------------------
>
> Key: HUDI-6569
> URL: https://issues.apache.org/jira/browse/HUDI-6569
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Priority: Major
>
> When the input records have a enum field in the Avro schema, the write fails
> due to:
> {code:java}
> 29542 [Executor task launch worker for task 2.0 in stage 35.0 (TID 56)] ERROR
> org.apache.hudi.io.HoodieCreateHandle [] - Error writing record
> HoodieRecord{key=HoodieKey { recordKey=09959033-0742-4638-aca4-44162efa4b27
> partitionPath=2015/03/17}, currentLocation='null', newLocation='null'}
> org.apache.avro.AvroRuntimeException: Unknown schema type: ENUM
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewritePrimaryType(HoodieAvroUtils.java:948)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchemaInternal(HoodieAvroUtils.java:895)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:830)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchemaInternal(HoodieAvroUtils.java:851)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:830)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:800)
> ~[classes/:?]
> at
> org.apache.hudi.avro.HoodieAvroUtils.rewriteRecordWithNewSchema(HoodieAvroUtils.java:784)
> ~[classes/:?]
> at
> org.apache.hudi.common.model.HoodieAvroRecord.prependMetaFields(HoodieAvroRecord.java:132)
> ~[classes/:?]
> at
> org.apache.hudi.io.HoodieCreateHandle.doWrite(HoodieCreateHandle.java:144)
> ~[classes/:?]
> at org.apache.hudi.io.HoodieWriteHandle.write(HoodieWriteHandle.java:182)
> ~[classes/:?]
> at
> org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:98)
> ~[classes/:?]
> at
> org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:42)
> ~[classes/:?]
> at
> org.apache.hudi.common.util.queue.SimpleExecutor.execute(SimpleExecutor.java:68)
> ~[classes/:?]
> at
> org.apache.hudi.execution.SparkLazyInsertIterable.computeNext(SparkLazyInsertIterable.java:80)
> ~[classes/:?]
> at
> org.apache.hudi.execution.SparkLazyInsertIterable.computeNext(SparkLazyInsertIterable.java:39)
> ~[classes/:?]
> at
> org.apache.hudi.client.utils.LazyIterableIterator.next(LazyIterableIterator.java:119)
> ~[classes/:?] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
