liucao-dd opened a new pull request, #56705: URL: https://github.com/apache/spark/pull/56705
### What changes were proposed in this pull request? This PR adds row-level DSv2 execution support for `INSERT INTO ... REPLACE USING`. It introduces a `RowLevelOperation.Command.REPLACE` path that rewrites scoped replace operations onto the existing row-level write machinery: - copy-on-write tables use `ReplaceData` - merge-on-read tables use `WriteDelta` - replace write metrics are reported through `ReplaceSummary` - existing row-level connectors are protected by a new `SupportsRowLevelReplace` opt-in marker The PR also updates the in-memory DSv2 test fixtures to cover positive and negative opt-in behavior, including copy, ALTER, and transactional wrapper paths. ### Why are the changes needed? `INSERT INTO ... REPLACE USING` needs a row-level execution path so DSv2 tables can replace source-defined scopes without relying on table-wide overwrite behavior. The explicit `SupportsRowLevelReplace` opt-in prevents existing row-level connectors that only implemented DELETE, UPDATE, or MERGE from unexpectedly receiving the new `REPLACE` command. ### Does this PR introduce _any_ user-facing change? Yes. DSv2 connectors that implement `SupportsRowLevelReplace` can support `INSERT INTO ... REPLACE USING (cols)` through row-level operations. Row-level tables that do not opt in fail analysis with an unsupported table operation error instead of invoking the connector's row-level operation builder with an unexpected command. ### How was this patch tested? Tested with: ``` export JAVA_HOME=/Library/Java/JavaVirtualMachines/amazon-corretto-17.jdk/Contents/Home export SPARK_LOCAL_IP=127.0.0.1 build/sbt 'catalyst/Test/compile' 'sql/testOnly org.apache.spark.sql.connector.GroupBasedReplaceUsingTableSuite org.apache.spark.sql.connector.DeltaBasedReplaceUsingTableSuite org.apache.spark.sql.connector.ReplaceUsingUnsupportedTableSuite' ``` Also ran representative row-level regression suites during development: ``` build/sbt 'sql/testOnly *DeltaBasedReplaceUsingTableSuite *ReplaceUsingUnsupportedTableSuite *GroupBasedDeleteFromTableSuite *DeltaBasedMergeIntoTableSuite' ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor (GPT-5.5) Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
