Wenning Ding created HUDI-1375:
----------------------------------
Summary: Fix bug for HoodieAvroUtils.removeMetadataFields
Key: HUDI-1375
URL: https://issues.apache.org/jira/browse/HUDI-1375
Project: Apache Hudi
Issue Type: Bug
Reporter: Wenning Ding
There's a code bug in HoodieAvroUtils.removeMetadataFields().
Reproduction steps:
{{}}
{code:java}
import org.apache.hudi.avro.HoodieAvroUtils
import org.apache.avro.Schema
val EXAMPLE_SCHEMA = """{"type": "record","name": "testrec","fields": [
{"name": "timestamp","type": "double"},{"name": "_row_key", "type":
"string"},{"name": "non_pii_col", "type": "string"},{"name": "pii_col", "type":
"string", "column_category": "user_profile"}]}"""
val schema =
HoodieAvroUtils.addMetadataFields(newSchema.Parser().parse(EXAMPLE_SCHEMA))
HoodieAvroUtils.removeMetadataFields(schema)
{code}
{{}}
Then it would throw an exception:
{{}}
{code:java}
org.apache.avro.AvroRuntimeException: Field already used: timestamp type:DOUBLE
pos:5 at org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647) at
org.apache.hudi.avro.HoodieAvroUtils.removeMetadataFields(HoodieAvroUtils.java:209)
... 49 elided {code}
{{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
