TheR1sing3un opened a new pull request, #12684: URL: https://github.com/apache/hudi/pull/12684
In many wide table scenarios, there may be thousands of columns in a table, and there are multiple tasks processing different columns. When reading, the downstream only cares about some dimension columns. But now the payload mode does not support columns trimming, resulting in great performance regression when perform snapshot read on file slices with log files exist. This is because all columns of the base file are read, even though most of the columns are not needed by the user ### Change Logs 1. support projection push down for payload mode _Describe context and summary for this change. Highlight if any code was copied._ ### Impact Improve mor read performance ### Risk level (write none, low medium or high below) medium ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable - [x] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
