JingsongLi commented on code in PR #11017:
URL: https://github.com/apache/paimon/pull/11017#discussion_r4561893355


##########
paimon-core/src/main/java/org/apache/paimon/operation/FileStoreCommitImpl.java:
##########
@@ -1817,6 +1946,24 @@ private Pair<Boolean, CommitCallback.Context> 
rollbackToAsLatestOnceWithoutLock(
         // the snapshots between the target and the previous latest, breaking 
the global uniqueness
         // of _ROW_ID. Keep the larger of the previous latest and the target 
nextRowId.
         Long nextRowId = maxNextRowId(latest.nextRowId(), 
targetSnapshot.nextRowId());
+        Map<String, String> rollbackProperties =
+                SequenceSnapshotProperties.mergeGuardMaxSequenceNumber(
+                        targetSnapshot.properties(), latest, deltaFiles, 
false);
+        rollbackProperties =
+                DeleteOrderingGuardGeneration.merge(
+                        rollbackProperties,
+                        latest,
+                        options.deleteOrderingGuardsEnabled(),
+                        true,
+                        null,
+                        partitionType);
+        rollbackProperties =
+                DeleteOrderingGuardPartitions.merge(
+                        rollbackProperties,
+                        targetSnapshot,

Review Comment:
   [P1] Validate restored guard policy before publishing rollback-as-latest
   
   This path restores the target guard metadata/index manifest without checking 
it against the current persisted ordering policy. I followed the documented 
migration sequence: capture an ascending delete/version4, append hidden 
version2 and tag it, truncate the entire table, persist descending order, 
reopen and successfully write/read new data. `rollbackToAsLatest` of the old 
guarded target then returns true and advances the head, but a fresh public read 
of the latest table fails with `Delete ordering guard ordering/deletion policy 
changed since schema 0; an ordering migration is required.` Ordinary commits 
validate the segment headers; this bypass publishes an incompatible state 
first. Reject an incompatible target before publication, or atomically restore 
a compatible effective policy with its guards.



##########
paimon-core/src/main/java/org/apache/paimon/table/AbstractFileStoreTable.java:
##########
@@ -747,6 +756,35 @@ public void rollbackTo(String tagName) {
         }
     }
 
+    private void validateDestructiveRollback() {
+        SnapshotManager snapshotManager = snapshotManager();
+        Snapshot latest;
+        try {
+            latest = snapshotManager.latestSnapshot();
+        } catch (RuntimeException e) {
+            if (!(e.getCause() instanceof FileNotFoundException)) {
+                throw e;
+            }
+            // Rollback can recover from a latest hint pointing to a missing 
snapshot. Check the
+            // latest existing snapshot so that recovery still respects 
retained guard history.
+            try {
+                latest =
+                        snapshotManager.safelyGetAllSnapshots().stream()
+                                .max(Comparator.comparingLong(Snapshot::id))
+                                .orElse(null);
+            } catch (IOException ex) {
+                throw new UncheckedIOException(ex);
+            }
+        }
+        if (coreOptions().deleteOrderingGuardsEnabled()
+                || DeleteOrderingGuardGeneration.isTracked(latest)
+                || DeleteOrderingGuardFile.mayContainGuards(latest)) {
+            throw new UnsupportedOperationException(

Review Comment:
   [P1] Apply destructive-history fencing to fastForward as well
   
   The new rejection protects `rollbackTo`, but public `fastForward` still 
copies an old branch's generation onto main through FileSystemBranchManager. I 
reproduced this sequentially, with the writer already closed: commit key1, 
create an old-history branch, prepare key2 under generation0, truncate main, 
and verify that the exact prepared committable is rejected with `generation 
changed`. After `table.fastForward("old-history")`, retrying that same 
committable succeeds and a real Java read returns keys1/2. The branch 
legitimately restores key1, but must not revive the separately fenced key2 
message. Reject guarded fast-forward too, or publish the branch contents with a 
fresh destination generation that cannot rewind; no concurrent writer is 
required for this failure.



##########
paimon-core/src/main/java/org/apache/paimon/table/source/snapshot/SnapshotReaderImpl.java:
##########
@@ -530,12 +614,33 @@ private Plan toIncrementalPlan(
             @Nullable Snapshot afterSnapshot,
             @Nullable Long afterWatermark,
             Map<BinaryRow, Map<Integer, List<ManifestEntry>>> afterFiles) {
+        // Diff must resolve the complete logical state at each endpoint. 
Removing a shared
+        // data file can hide a before-image needed to compare the visible 
rows after guard
+        // resolution.
+        boolean resolveGuards =
+                !isStreaming || 
DeleteOrderingGuardFile.mayContainGuards(afterSnapshot);
+        Map<Pair<BinaryRow, Integer>, List<IndexFileMeta>> beforeGuards =
+                resolveGuards ? incrementalGuards(beforeSnapshot.get()) : 
Collections.emptyMap();

Review Comment:
   [P1] Avoid requiring expired predecessor metadata for current APPEND deltas
   
   This unconditionally resolves `beforeSnapshot` whenever the after-snapshot 
carries guard history, even when an APPEND delta has no before files. With a 
full-compacted delete guard, I appended a stale key1/version2 plus fresh 
key2/version5, expired all predecessors while retaining current snapshot3, and 
verified that its data/guard files and batch read remain valid. A public 
file-monitor stream restored at3 then fails here loading expired snapshot2; the 
identical unguarded control succeeds. Flink's default PK lookup 
FILE_MONITOR/AllDelta path also calls this code. Current-delta consumption 
should not require a deleted predecessor solely to discover an empty before 
side. Avoid that load when unnecessary, or preserve the required predecessor 
guard references independently of expired snapshot metadata.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to