yihua opened a new pull request, #8156: URL: https://github.com/apache/hudi/pull/8156
### Change Logs In the following scenario, before this fix, the validation job fires a false alarm complaining that the partition list returned by the file system and the metadata table: - Commit C1 creates the partition, the partition metadata is written, and C1 fails during writing data files. Next time, C2 adds new data to the same partition after C1 is rolled back. In this case, the partition metadata still has C1 as the created commit time, since Hudi does not rewrite the partition metadata in C2. This PR fixes the validation of partition listing in metadata table validator for this case, by checking if the created commit time is the same as or before the latest commit. ### Impact Fixes the false validation failure in the scenario where the commit creating the partitions fails and is rolled back later. ### Risk level low ### Documentation Update N/A ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
