smengcl commented on code in PR #12013:
URL: https://github.com/apache/ozone/pull/12013#discussion_r4534448067
##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/diff/TestMergeJoinSnapDiffWriter.java:
##########
@@ -315,6 +318,51 @@ void testDependencyOrderingFallsBackWhenOverLimit() throws
Exception {
}
}
+ @Test
+ void testDependencyOrderingFallsBackOnCycle() throws Exception {
+ byte[] sig = signature("s");
+ SnapDiffJobStore store = spy(newStore(true));
+ try (SnapDiffJobStore ignored = store) {
+ // Files a and b swap names: each RENAME frees the path the other one
takes.
+ store.putNewList(301L, entry(BUCKET_OBJECT_ID, "b", false, sig));
+ store.putOldList(301L, entry(BUCKET_OBJECT_ID, "a", false, sig));
+ store.putNewList(302L, entry(BUCKET_OBJECT_ID, "a", false, sig));
+ store.putOldList(302L, entry(BUCKET_OBJECT_ID, "b", false, sig));
+ store.flushWrites();
+
+ Pair<Long, String> reportMeta =
writeReport(mockManagerPassthroughDeletes(), store,
+ BUCKET_OBJECT_ID, true, true, null);
+ // Successful write with unordered outcome is what SnapshotDiffManager
marks DONE with.
+ assertEquals(2L, reportMeta.getKey().longValue());
+ assertFalse(store.isReportDependencyOrdered());
+ assertEquals("Report is dependency unordered.",
+ "Report is dependency " + (store.isReportDependencyOrdered() ?
"ordered" : "unordered") + ".");
Review Comment:
This one builds the reason string inside the test, so it passes whenever the
`assertFalse` above passes and does not touch `SnapshotDiffManager`. Could we
drop it? `assertFalse(store.isReportDependencyOrdered())` already checks the
outcome, and the reason text itself comes from `updateJobStatusToDone`.
##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/diff/TestMergeJoinSnapDiffWriter.java:
##########
@@ -315,6 +318,51 @@ void testDependencyOrderingFallsBackWhenOverLimit() throws
Exception {
}
}
+ @Test
+ void testDependencyOrderingFallsBackOnCycle() throws Exception {
+ byte[] sig = signature("s");
+ SnapDiffJobStore store = spy(newStore(true));
+ try (SnapDiffJobStore ignored = store) {
+ // Files a and b swap names: each RENAME frees the path the other one
takes.
+ store.putNewList(301L, entry(BUCKET_OBJECT_ID, "b", false, sig));
+ store.putOldList(301L, entry(BUCKET_OBJECT_ID, "a", false, sig));
+ store.putNewList(302L, entry(BUCKET_OBJECT_ID, "a", false, sig));
+ store.putOldList(302L, entry(BUCKET_OBJECT_ID, "b", false, sig));
+ store.flushWrites();
+
+ Pair<Long, String> reportMeta =
writeReport(mockManagerPassthroughDeletes(), store,
+ BUCKET_OBJECT_ID, true, true, null);
+ // Successful write with unordered outcome is what SnapshotDiffManager
marks DONE with.
+ assertEquals(2L, reportMeta.getKey().longValue());
+ assertFalse(store.isReportDependencyOrdered());
+ assertEquals("Report is dependency unordered.",
+ "Report is dependency " + (store.isReportDependencyOrdered() ?
"ordered" : "unordered") + ".");
+
+ List<DiffReportEntry> entries = store.readReportEntriesForTest();
+
assertThat(entries).extracting(DiffReportEntry::getType).containsExactly(RENAME,
RENAME);
+ assertEquals("a", new String(entries.get(0).getSourcePath(),
StandardCharsets.UTF_8));
+ assertEquals("b", new String(entries.get(0).getTargetPath(),
StandardCharsets.UTF_8));
+ assertEquals("b", new String(entries.get(1).getSourcePath(),
StandardCharsets.UTF_8));
+ assertEquals("a", new String(entries.get(1).getTargetPath(),
StandardCharsets.UTF_8));
+
+ try (SnapDiffJobStore unorderedStore = newStore(true)) {
+ unorderedStore.putNewList(301L, entry(BUCKET_OBJECT_ID, "b", false,
sig));
+ unorderedStore.putOldList(301L, entry(BUCKET_OBJECT_ID, "a", false,
sig));
+ unorderedStore.putNewList(302L, entry(BUCKET_OBJECT_ID, "a", false,
sig));
+ unorderedStore.putOldList(302L, entry(BUCKET_OBJECT_ID, "b", false,
sig));
+ unorderedStore.flushWrites();
+ List<DiffReportEntry> explicitUnordered =
+ runWriteReport(unorderedStore, BUCKET_OBJECT_ID, true, false);
+ for (int i = 0; i < entries.size(); i++) {
+ assertArrayEquals(explicitUnordered.get(i).getSourcePath(),
entries.get(i).getSourcePath());
+ assertArrayEquals(explicitUnordered.get(i).getTargetPath(),
entries.get(i).getTargetPath());
+ }
Review Comment:
nit: the path checks above already pin the order (a -> b, then b -> a). If
we keep the comparison with the unordered run, `assertEquals(explicitUnordered,
entries)` might be simpler, since `DiffReportEntry` implements `equals` over
the type and both paths. It would also catch a size mismatch, which this loop
over `entries` does not. The four put calls repeated for `unorderedStore` could
go into a small helper shared by both stores.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]