SaketaChalamchala opened a new pull request, #12014: URL: https://github.com/apache/ozone/pull/12014
Found by TLA+ model checking and code review of the OM optimized snapshot diff paths under [HDDS-15926](https://issues.apache.org/jira/browse/HDDS-15926), on commit d3b75c75bf9ad475551202477c3a21a3f4b968b7. The patch is generated with Specula (Claude Opus 5.5). Should be merged after https://github.com/apache/ozone/pull/12013 ## What changes were proposed in this pull request? FSO snapshot diff dependency ordering (`SnapDiffDependencyGraph`) incorrectly applied a reoccupied-path skip when adding to-snapshot path-prefix edges. That skip broke ordering in two ways: 1. Wrong “ordered” reports that do not replay — it dropped required edges for `RENAME`(target) under a reoccupied ancestor. **Example A: Wrong order (skip too aggressive for RENAME)** **Snapshot 1 → 2:** ``` mkdir a create a/a snapshot 1 mv a b mkdir a mv b/a a/a # file ends at a/a in snapshot 2 snapshot 2 ``` **Report entries:** ``` RENAME a/a -> a/a RENAME a -> b CREATE a ``` This leaves S2 with `b/a` instead of `a/a, `b` **After this patch:** the graph detects a cycle (true constraint conflict). With HDDS-17738, ordered jobs should fall back to an unordered report instead of `FAILED`. 3. False cycles — it kept spurious edges from to-snapshot ancestor ops to `MODIFY` entries (and missed ancestor-freed paths), so jobs failed even when a valid order existed. **Example B: False cycle (skip too weak for MODIFY + nested recreate)** **Snapshot 1 → 2:** ``` mkdir -p a/a create a/a/a snapshot 1 mv a b mkdir -p a/a overwrite b/a/a # MODIFY at from-snapshot path a/a/a snapshot 2 ``` **Report entries (valid replay order exists):** ``` MODIFY a/a/a RENAME a -> b CREATE a CREATE a/a ``` `MODIFY a/a/a` is on the old tree (path in snapshot 1). It must run before `RENAME a -> b`. It must not depend on `CREATE a`, `/ CREATE a/a` on the new tree. The old logic still added (or failed to skip) a `CREATE a` → `MODIFY a/a/a` style edge when the path was freed via an ancestor rename (a, not a/a/a itself). That closed a false cycle → ordered job `FAILED`, while unordered worked. **After this patch:** no to-snapshot edges for `MODIFY`; expected order: `MODIFY a/a/a, RENAME a -> b, CREATE a, CREATE a/a` The fix aligns edge rules with the path namespace contract: `MODIFY` never gets to-snapshot prefix edges, and `CREATE` / `RENAME(target)` always keep those edges. The reoccupied-path skip is removed (`buildReoccupiedPaths`, `shouldSkipReoccupiedPathToSnapshotEdge`). 1. addPathPrefixToSnapshotEdges — skip `MODIFY` (and `DELETE`) entirely; only `CREATE` and `RENAME`(target) get to-snapshot prefix edges. 2. Remove `buildReoccupiedPaths`, `shouldSkipReoccupiedPathToSnapshotEdge`, and `isStrictPathPrefix`. 3. Always add `CREATE`/`RENAME(target) `ancestor → descendant edges (no skip). Rationale in one line: `MODIFY` is from-snapshot only for hierarchy; `CREATE`/`RENAME(target)` are to-snapshot only. Mixing `MODIFY` into to-snapshot prefix edges (with heuristics) caused both bugs. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17739 ## How was this patch tested? - Unit: `TestSnapDiffDependencyGraph` - New: `testRenameSourceReuseWithDescendantRenameIntoNewOccupierDetectsCycle`, `testRenameSourceReuseByNestedCreateWithDescendantModify`, `testDeleteAndRecreateNestedPathWithDescendantModifyAndRename` - Updated: `testDeleteAndRecreateWithOldTreeChildModifyAndRename` now expects a cycle (cross-parent rename into new occupier) - Existing reoccupied-path `MODIFY` cases updated / still pass with the new edge rules -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
