Prashant Wason created HUDI-741:
-----------------------------------
Summary: Fix Hoodie's schema evolution checks
Key: HUDI-741
URL: https://issues.apache.org/jira/browse/HUDI-741
Project: Apache Hudi (incubating)
Issue Type: Bug
Reporter: Prashant Wason
HUDI requires a Schema to be specified in HoodieWriteConfig and is used by the
HoodieWriteClient to create the records. The schema is also saved in the data
files (parquet format) and log files (avro format).
Since a schema is required each time new data is ingested into a HUDI dataset,
schema can be evolved over time. But HUDI should ensure that the evolved schema
is compatible with the older schema.
HUDI specific validation of schema evolution should ensure that a newer schema
can be used for the dataset by checking that the data written using the old
schema can be read using the new schema.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
