alexeykudinkin opened a new pull request #4060: URL: https://github.com/apache/hudi/pull/4060
## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the purpose of the pull request Addressing issues discovered during our extensive preparatory testing (summarized in [HUDI-2814](https://issues.apache.org/jira/browse/HUDI-2814)) ## Brief change log NOTE: This change is stacked onto https://github.com/apache/hudi/pull/4026 - [critical] Fixed index new/original table merging sequence to always prefer values from the new index-table - [critical] Fixed `DataSkippingUtils` to (conservatively) interrupt pruning in case data filter contains non-indexed column references - [critical] Fixed Z-index to properly handle changes of the list of clustered columns - [critical] Fixed incorrect data-type conversions (for ex, Decimal to Double) - [critical] Fixed race-condition in Parquet's `DateStringifier` class sharing `SimpleDataFormat` object which is NOT thread-safe (and have been cause for tests flakiness) - [major] Properly handle exceptions origination during pruning in `HoodieFileIndex` - [major] Adding tests for most of the Z-indexing critical flows (Z-index table creation, merging, (basic) data-skipping) ## Verify this pull request This change added tests and can be verified as follows: - *Added functional tests for critical aspects of the Z-indexing workflow (index table creation, merging, (basic) data-skipping)* ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ] Commit message is descriptive of the change - [ ] CI is green - [ ] Necessary doc changes done or have another open PR - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
