XiaoHongbo-Hope opened a new pull request, #9240: URL: https://github.com/apache/paimon/pull/9240
## Purpose `TableUpdate.update_by_predicate` currently reads every matched row into one Arrow table before writing updates. Memory therefore grows with the full predicate result, even though data-evolution updates are written one logical file group at a time. This PR processes the existing plan one complete file group at a time: - unpack packed splits by overlapping row-id range; - preserve indexed row ranges and deletion-file metadata; - read only `_ROW_ID` for literal assignments; - write each group once and accumulate the normal `CommitMessage` list. The public API and atomic commit semantics do not change. Peak data memory is bounded by the largest logical file group instead of all matched rows. ## Tests - verifies multiple file groups packed in one split are processed separately; - covers a group containing an existing data-evolution delta; - runs existing predicate update and global-index cases in batch and stream modes. 110 related tests passed locally. Flake8 and `git diff --check` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
