cshuo opened a new pull request, #14337: URL: https://github.com/apache/hudi/pull/14337
### Describe the issue this Pull Request addresses Fix https://github.com/apache/hudi/issues/14297, fix duplicate field exception in hive query with where clause. ### Summary and Changelog The READ_COLUMN_NAMES_CONF_STR includes all columns from the query, including those used in the WHERE clause, so any column referenced in the filter (non-partition) will appear twice if already present in the project schema, so we should deduplicate the read columns before construct file group reader. ### Impact bugfix ### Risk Level low ### Documentation Update <!-- Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none". - The config description must be updated if new configs are added or the default value of the configs are changed. - Any new feature or user-facing change requires updating the Hudi website. Please follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website. --> ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
