[
https://issues.apache.org/jira/browse/HUDI-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udit Mehrotra resolved HUDI-1375.
---------------------------------
Resolution: Fixed
> Fix bug for HoodieAvroUtils.removeMetadataFields
> ------------------------------------------------
>
> Key: HUDI-1375
> URL: https://issues.apache.org/jira/browse/HUDI-1375
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Wenning Ding
> Assignee: Wenning Ding
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.6.1, 0.7.0
>
>
> There's a code bug in HoodieAvroUtils.removeMetadataFields().
> Reproduction steps:
>
> {{}}
> {code:java}
> import org.apache.hudi.avro.HoodieAvroUtils
> import org.apache.avro.Schema
> val EXAMPLE_SCHEMA = """{"type": "record","name": "testrec","fields": [
> {"name": "timestamp","type": "double"},{"name": "_row_key", "type":
> "string"},{"name": "non_pii_col", "type": "string"},{"name": "pii_col",
> "type": "string", "column_category": "user_profile"}]}"""
> val schema =
> HoodieAvroUtils.addMetadataFields(newSchema.Parser().parse(EXAMPLE_SCHEMA))
> HoodieAvroUtils.removeMetadataFields(schema)
> {code}
> {{}}
> Then it would throw an exception:
>
> {{}}
> {code:java}
> org.apache.avro.AvroRuntimeException: Field already used: timestamp
> type:DOUBLE pos:5 at
> org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647) at
> org.apache.hudi.avro.HoodieAvroUtils.removeMetadataFields(HoodieAvroUtils.java:209)
> ... 49 elided {code}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
