Siyao Meng created HDDS-17744:
---------------------------------
Summary: Optimized snapshot diff omits a file or directory moved
into a directory that is deleted before its children are purged
Key: HDDS-17744
URL: https://issues.apache.org/jira/browse/HDDS-17744
Project: Apache Ozone
Issue Type: Bug
Reporter: Siyao Meng
Attachments: MC-1-report-move-into-deleted-dir-as-delete.patch,
TestBugMC1RenameIntoDeletedDir.java
h3. Mechanism
A file or directory moved under a directory that is then deleted keeps its row
in {{fileTable}} or {{directoryTable}} until the directory deep clean purges
the children, so a snapshot taken in between holds it under a parent that no
longer exists in {{directoryTable}}. The merge join sees the same objectId on
both sides and classifies it as RENAME. In
{{MergeJoinSnapDiffWriter.resolveReportEntries}} the target path cannot be
resolved, so the entry becomes null, and {{writeReportEntriesForRows}} drops it
with a DEBUG log. Nothing else in the report removes it. The legacy diff
({{SnapshotDiffManager.generateDiffReport}}) handles the same shape in its
branch for an old path that resolves and a new path that does not, and reports
a DELETE of the old path.
h3. Trigger
FSO bucket, {{ozone.om.snapshot.diff.optimized.enabled=true}} (default false;
the optimized diff was added by HDDS-15391 and is not in any release yet):
# Create file {{/a}} and directory {{/b}}, take snapshot snap1.
# {{mv /a /b/a}}, then {{rm -r /b}}.
# Take snapshot snap2 before the directory deleting service has purged the
children of {{b}} (it runs on an interval).
# Request the snapshot diff snap1 to snap2, ordered or unordered.
h3. Impact
The DONE report is {{[DELETE b]}} instead of {{[DELETE a, DELETE b]}}.
Replaying it onto snap1 keeps {{/a}}, which does not exist in snap2. The report
is persisted and returned for every later read of the pair. The legacy diff of
an identical pair on the same OM reports both deletes. If the moved object is a
directory, its whole subtree stays on replay: moving a directory into a trash
directory and then deleting the trash directory recursively is a common pattern.
h3. Reproduction
[^TestBugMC1RenameIntoDeletedDir.java] ({{hadoop-ozone/integration-test}}) runs
the steps above through the FileSystem and snapshot APIs on a mini cluster,
with forced full diff and native diff disabled so the native library is not
needed. The base class keeps the KeyManager background services stopped, which
holds the window open. It first computes the legacy report of an identical pair
(both deletes), then asserts the optimized unordered and ordered reports. On
the unfixed code it fails with {{Expecting actual: [- ./b] ... could not find
the following elements: [- ./a]}}; with the patch it passes.
h3. Patch
[^MC-1-report-move-into-deleted-dir-as-delete.patch], against
d3b75c75bf9ad475551202477c3a21a3f4b968b7. It also applies to master at
bbc8505e00d.
In the RENAME branch of {{MergeJoinSnapDiffWriter.resolveReportEntries}}, when
the source path resolves and the target path does not, the writer now emits
DELETE of the source path, the same rule as the legacy diff. Entries where
neither path resolves are still dropped. The dependency ordered report needs no
extra handling, since the graph takes the entry type from the report entry.
Covered by {{testRenameIntoDeletedDirectoryReportsDeleteOfSource}} in the
existing {{TestMergeJoinSnapDiffWriter}}, for both report orderings. Without
the change it fails because only {{DELETE b}} is reported. With it the snapshot
diff unit suites and {{TestSnapshotDiffManager}} (193 tests) and
{{TestOptimizedSnapshotDiff}} with the native library pass, and checkstyle is
clean.
One previously correct report changes: if the source's parent is deleted as
well ({{mv /c/a /b/a}}, {{rm -r /b}}, {{rm -r /c}}), the report gains a
redundant but harmless {{DELETE c/a}} next to {{DELETE c}}, ordered before it;
the legacy diff emits it too.
Not changed: if the moved file was also modified, a MODIFY of the old path is
still emitted next to the DELETE (the legacy diff emits only the DELETE).
Found by TLA+ model checking and code review of the OM optimized snapshot diff
paths under HDDS-15926, on commit d3b75c75bf9ad475551202477c3a21a3f4b968b7.
Checked against HDDS issues and apache/ozone pull requests for duplicates
before filing. The attached patch is a proposal for review. Generated with
Specula (Claude Opus 5.5).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
