steFaiz commented on PR #7324: URL: https://github.com/apache/paimon/pull/7324#issuecomment-3982705383
> I mean you can use configure scanned snapshot id to `commit.strict-mode.last-safe-snapshot`, this snapshot id should be used by MERGE INTO source. > > And you don't need to introduce `MergeIntoCommitterOperator`, just use `CommitterOperator`, `commit.strict-mode.last-safe-snapshot` will solve your `FilterAndCommit` problem. @JingsongLi Thanks for your explaination! I've set the `commit.strict-mode.last-safe-snapshot` as the latestSnapshotId in the action's constructor. I introduce the new committer operator wrapping a `CommitterOperator` to check the received committables, so that we could throw and error or drop partition index if merge into action modified some global-indexed columns. An alternative approach is to introduce a new operator before committer e.g. `UpdateCheckerOperator`. Which one do you prefer, I'm willing to modify my code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
