ArnavBalyan commented on code in PR #9363:
URL: https://github.com/apache/paimon/pull/9363#discussion_r3838304348
##########
paimon-core/src/main/java/org/apache/paimon/operation/commit/DataEvolutionConflictDetection.java:
##########
@@ -356,8 +368,34 @@ private Optional<RuntimeException>
checkForRowIdFromSnapshot(
return Optional.empty();
}
+ // Fail closed when the latest snapshot ID is less than the base
snapshot ID.
+ // This indicates a rollback has deleted newer snapshots, and the
staged update
+ // is based on a snapshot lineage that no longer exists.
+ if (latestSnapshot.id() < rowIdCheckFromSnapshot) {
+ return Optional.of(
+ new RuntimeException(
+
ErrorMessages.DATA_EVOLUTION_SNAPSHOT_LINEAGE_CONFLICT_MESSAGE));
+ }
+
+ // Detect equal snapshot IDs with different snapshot UUIDs (ABA
problem).
+ // A rollback can delete a snapshot and a new commit can reuse the
same numeric ID.
+ // If the base snapshot UUID differs from the current snapshot UUID at
that ID,
+ // the staged update is based on a different snapshot lineage.
+ Snapshot baseSnapshot =
snapshotManager.snapshot(rowIdCheckFromSnapshot);
+ if (baseSnapshot == null) {
+ return Optional.of(
+ new RuntimeException(
+
ErrorMessages.DATA_EVOLUTION_SNAPSHOT_LINEAGE_CONFLICT_MESSAGE));
+ }
+ if (baseSnapshotUuid != null
Review Comment:
Can we propagate the base snapshot UUID. Spark and flink are calling the 1
argument overload method, never passing the uuid.
The base snapshot UUID will always be null and this comparison will be
always skipped.
Can we capture the readSnapshot's uuid, and pass to the overload? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
