ArnavBalyan commented on code in PR #9363:
URL: https://github.com/apache/paimon/pull/9363#discussion_r3838304348
##########
paimon-core/src/main/java/org/apache/paimon/operation/commit/DataEvolutionConflictDetection.java:
##########
@@ -356,8 +368,34 @@ private Optional<RuntimeException>
checkForRowIdFromSnapshot(
return Optional.empty();
}
+ // Fail closed when the latest snapshot ID is less than the base
snapshot ID.
+ // This indicates a rollback has deleted newer snapshots, and the
staged update
+ // is based on a snapshot lineage that no longer exists.
+ if (latestSnapshot.id() < rowIdCheckFromSnapshot) {
+ return Optional.of(
+ new RuntimeException(
+
ErrorMessages.DATA_EVOLUTION_SNAPSHOT_LINEAGE_CONFLICT_MESSAGE));
+ }
+
+ // Detect equal snapshot IDs with different snapshot UUIDs (ABA
problem).
+ // A rollback can delete a snapshot and a new commit can reuse the
same numeric ID.
+ // If the base snapshot UUID differs from the current snapshot UUID at
that ID,
+ // the staged update is based on a different snapshot lineage.
+ Snapshot baseSnapshot =
snapshotManager.snapshot(rowIdCheckFromSnapshot);
+ if (baseSnapshot == null) {
+ return Optional.of(
+ new RuntimeException(
+
ErrorMessages.DATA_EVOLUTION_SNAPSHOT_LINEAGE_CONFLICT_MESSAGE));
+ }
+ if (baseSnapshotUuid != null
Review Comment:
Looks like Spark and Flink are using the single argument offloading causing
the method to never get invoked. Would be better to capture and pass the UUID
to the overloaded method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
