JustinBinber opened a new pull request, #11339: URL: https://github.com/apache/paimon/pull/11339
Fixes #11338 ### Purpose An UPDATE + INSERT MERGE immediately after `ALTER TABLE ADD COLUMNS` can silently drop values for the new columns on inserted rows. The target scan must stay on one data snapshot for conflict detection. Its table copy was also replacing the current logical schema with the older snapshot schema, and the INSERT writer inherited that schema. Keep the current schema when pinning the data snapshot in the Spark MERGE command and when copying write options for the partial-column writer. This keeps the snapshot used for reads and conflict checks while allowing matched UPDATE and unmatched INSERT to write the new column. ### Tests - New regression creates a data snapshot, adds a nullable column, then uses one MERGE to write that column on matched and unmatched rows. It also checks an untouched row and runs with join reuse enabled and disabled. - The Spark 3.5.9 / Scala 2.12 / JDK 8 Maven reactor passed all four tests in `DataEvolutionUpdateSnapshotTest` and `DataEvolutionMergeJoinReuseConflictTest`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
