umehrot2 opened a new pull request #1559: URL: https://github.com/apache/incubator-hudi/pull/1559
*Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull request Support to store and read schema from **HoodieCommitMetadata** has been added in Hudi recently. This PR integrated Hive Sync to read schema from the metadata, and fallback to reading from files only if the schema from metadata is unavailable (for backwards compatibility). This is a pre-requisite to https://jira.apache.org/jira/browse/HUDI-620 where we want to store the merged bootstrap schema in commit metadata and obtain it during hive-sync. This avoids us having to look up the indexes for external files and merging the schemas just for hive-sync. In addition, this PR does a fix in HoodieWrite client to fallback on reading schema from the files in case schema from metadata is not available (for validating the schema). ## Brief change log - Fix in HoodieWriteClient to fallback on reading schema from files - Hive sync changes and unit tests to read schema from HoodieCommitMetadata ## Verify this pull request This change added unit tests. ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
