yihua opened a new pull request, #12932: URL: https://github.com/apache/hudi/pull/12932
### Change Logs This PR fixes the unnecessary scanning of the target table in MERGE INTO statement in Spark by only using the source table as the input if the target table has a record key (before this change the the source and target table is left joined, causing all partitions in the target table to be scanned unnecessarily, if the source table only contains data targeting a handful of partitions). For primary-keyless table the logic is not changed, i.e., the source and target table is left joined to get the meta column values for prepped upsert in MERGE INTO. ### Impact Improves performance of MERGE INTO on Spark ### Risk level low. Performance improvement only. Existing tests have guarded the correctness of MERGE INTO statement. ### Documentation Update N/A ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
