sshivampeta opened a new pull request, #16289: URL: https://github.com/apache/iceberg/pull/16289
Summary Fixes https://github.com/apache/iceberg/issues/16232. This change prevents REPLACE TABLE transactions from silently overwriting concurrent committed table changes. If table metadata changes after a replace transaction starts, the replace transaction now fails with CommitFailedException instead of committing stale metadata. Problem Before this change, replaceTransaction() could overwrite concurrent commits without conflict detection. This affected concurrent: schema updates table property updates data appends snapshot expiration another replace transaction The root problem was that replace transactions refreshed the underlying table state but still committed the stale replacement metadata built when the transaction started. In addition, replace-table update requirements skipped several optimistic concurrency checks. This allowed committed changes to disappear silently. Changes Updated BaseTransaction.commitReplaceTransaction to fail if the table metadata changed since the replace transaction started. Updated UpdateRequirements.forReplaceTable to use the same optimistic concurrency checks as ordinary table updates. Added regression coverage in TestReplaceTableSafety. Updated catalog tests to expect concurrent replace conflicts instead of last-writer-wins behavior. Updated Hive replace transaction tests to preserve concurrent changes and expect conflict failures. Updated Spark encryption replace test expectations for concurrent replace behavior. Why this fixes the issue A replace transaction now compares the original base metadata with the latest table metadata after refresh. If another writer committed in between, the replace transaction fails before committing stale metadata. This prevents stale replace metadata from overwriting committed schema, property, snapshot, or data changes. For REST-style commits, UpdateRequirements.forReplaceTable now emits normal optimistic concurrency requirements, so server-side validation can also reject conflicting replace commits. Testing Ran: ./gradlew :iceberg-core:test --tests "org.apache.iceberg.TestReplaceTableSafety" --tests "org.apache.iceberg.TestUpdateRequirements" ./gradlew :iceberg-hive-metastore:test --tests "org.apache.iceberg.hive.TestHiveCreateReplaceTable" ./gradlew :iceberg-hive-metastore:test ./gradlew :iceberg-spark:iceberg-spark-4.1_2.13:checkstyleTest ./gradlew :iceberg-azure:integrationTest :iceberg-aws:integrationTest --continue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
