wombatu-kun opened a new pull request, #11656: URL: https://github.com/apache/hudi/pull/11656
### Change Logs From previous PR: HoodieFileGroupReader failed if preCombine and partition fields are the same with IllegalArgumentException: Field: ts does not exist in the table schema. precombineField is required but it was filtered from dataSchema as other partition fields. To fix this I made HoodieFileGroupReaderBasedParquetFileFormat do not filter partitionColumn from dataSchema if it is the same as preCombine field. --------------- But I did it wrong, as you can see from this discussion: https://github.com/apache/hudi/pull/11473/files#r1681098941 Fixed 2 things: - filtering condition was wrong during evaluation of dataSchema; - options did not contain precombineField. With this PR I fixed it. ### Impact precombineField and partition field may be the same, and it works with local spark and on cluster. ### Risk level (write none, low medium or high below) none ### Documentation Update none - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
