[
https://issues.apache.org/jira/browse/HDDS-17745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17745:
------------------------------
Attachment: TestBugMC5EmptyDirectoryDelta.java
> Optimized FSO snapshot diff drops every file change below the bucket root
> when no directory row changed
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-17745
> URL: https://issues.apache.org/jira/browse/HDDS-17745
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 2.3.0
> Reporter: Siyao Meng
> Priority: Major
> Attachments: MC-5-run-directory-stage-when-only-files-changed.patch,
> TestBugMC5EmptyDirectoryDelta.java
>
>
> h3. Mechanism
> {{SnapshotDiffManager.scanTablesForDiff}} returns early when a table has no
> delta SST files, before it picks a reader. For an FSO bucket the directory
> stage is also what builds the path edges that {{SnapDiffPathResolver}} uses
> to resolve file entries. Creating or deleting a file under an existing
> directory writes nothing to {{directoryTable}}, so when only files changed
> the directory delta is empty, the directory stage is skipped, every file
> entry whose parent is not the bucket resolves to null, and
> {{MergeJoinSnapDiffWriter}} drops it with a DEBUG log. The job ends DONE.
> {{DagDiffSequentialReader.scanDirectoryTables}} already handles an empty
> delta by scanning both directory tables; the early return keeps it from
> running.
> h3. Trigger
> FSO bucket, {{ozone.om.snapshot.diff.optimized.enabled=true}}, native rocks
> tools library loaded (the default,
> {{ozone.om.snapshot.load.native.lib=true}}):
> # Create {{/a/old}}, take snapshot snap1.
> # Create {{/a/new}}, delete {{/a/old}}, take snapshot snap2.
> # Request the snapshot diff snap1 to snap2, ordered or unordered.
> The diff path does not matter once the native library is loaded: the DAG
> reader, a forced full diff and the automatic full diff fallback all build the
> directory delta only from SST files the two snapshots do not share, and all
> pass through the same early return. A forced full diff with the native
> library was run and gives the same empty report; the automatic fallback uses
> the same code (by reading). Without the native library the optimized path
> runs only with a forced full diff, and the non native delta computation then
> adds every SST file of the from snapshot to the delta, so the directory delta
> is never empty and the bug does not show.
> h3. Impact
> Both reports are empty instead of {{[CREATE a/new, DELETE a/old]}}. Any diff
> in which files were only created, overwritten or deleted, with no directory
> row written (no directory created, renamed or deleted, and no file renamed,
> since a file rename updates the modification time of both parent
> directories), loses every change below the bucket root. The report is
> persisted and returned for every later read of the pair.
> h3. Reproduction
> [^TestBugMC5EmptyDirectoryDelta.java] ({{hadoop-ozone/integration-test}},
> needs {{-Drocks_tools_native}}) runs the steps above through the FileSystem
> and snapshot APIs on a mini cluster with the native library, on the default
> DAG path (the forced full diff variant was checked separately). On the
> unfixed code it fails with {{Expecting actual: [] to contain exactly in any
> order: [+ ./a/new, - ./a/old]}}; with the patch it passes.
> h3. Patch
> [^MC-5-run-directory-stage-when-only-files-changed.patch], against
> d3b75c75bf9ad475551202477c3a21a3f4b968b7. It also applies to master at
> bbc8505e00d.
> The early return now skips the directory stage only when the earlier file
> stage put nothing into the new and old lists
> ({{SnapDiffJobStore.hasListEntries}}, from the counters the store already
> keeps). The directory stage then runs with an empty delta and builds the
> edges. A diff with no changes at all keeps the fast path. A diff in which
> only files changed now scans both directory tables of the bucket, which a
> diff with one directory change already does.
> Covered by {{testOptimizedDiffWithUnchangedDirectoryTable}} in the existing
> {{TestOptimizedSnapshotDiff}} (native), for both report orderings. Without
> the change both reports are empty. With it {{TestOptimizedSnapshotDiff}}, the
> snapshot diff unit suites and {{TestSnapshotDiffManager}} (191 tests) pass,
> and checkstyle is clean.
> Found by TLA+ model checking and code review of the OM optimized snapshot
> diff paths under HDDS-15926, on commit
> d3b75c75bf9ad475551202477c3a21a3f4b968b7. Checked against HDDS issues and
> apache/ozone pull requests for duplicates before filing. The attached patch
> is a proposal for review. Generated with Specula (Claude Opus 5.5).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
