yunlou11 commented on code in PR #5730:
URL: https://github.com/apache/paimon/pull/5730#discussion_r2165274862
##########
paimon-core/src/test/java/org/apache/paimon/mergetree/compact/LookupChangelogMergeFunctionWrapperTest.java:
##########
@@ -547,4 +551,80 @@ public void testKeepLowestHighLevel() {
kv = result.result();
assertThat(kv.value().getInt(0)).isEqualTo(3);
}
+
+ @Test
+ public void testWithDeletionVectorsMaintainer() {
+ IndexFileHandler indexFileHandler =
+ new IndexFileHandler(
+ null,
+ null,
+ null,
+ null,
+ new DeletionVectorsIndexFile(null, null, null, true));
+ DeletionVectorsMaintainer dvMaintainer =
+ DeletionVectorsMaintainer.factory(indexFileHandler).create();
+ Map<InternalRow, LookupLevels.PositionedKeyValue> lookup = new
HashMap<>();
+ LookupChangelogMergeFunctionWrapper function =
+ new LookupChangelogMergeFunctionWrapper(
+
LookupMergeFunction.wrap(DeduplicateMergeFunction.factory()),
+ lookup::get,
+ EQUALISER,
+ LookupStrategy.from(false, true, true, true),
+ dvMaintainer,
+ null);
+
+ // With level-0, with level-x in lookup, with level-x deleted in
dvMaintainer
+ function.reset();
+ LookupLevels.PositionedKeyValue positionedKv =
+ new LookupLevels.PositionedKeyValue(
+ new KeyValue().replace(row(1), 1, INSERT,
row(2)).setLevel(5), "f1", 1);
+ lookup.put(row(1), positionedKv);
+ // Simulation scenario: (file:"f1", pos: 1) has been deleted
+ dvMaintainer.notifyNewDeletion("f1", 1);
+ function.add(new KeyValue().replace(row(1), 2, INSERT,
row(2)).setLevel(0));
+ // notifyNewDeletion will return 'false' because (file:"f1", pos: 1)
has been deleted
+ ChangelogResult result = function.getResult();
+ assertThat(result).isNotNull();
+ List<KeyValue> changelogs = result.changelogs();
+ assertThat(changelogs).hasSize(1);
+ assertThat(changelogs.get(0).valueKind()).isEqualTo(INSERT);
+ assertThat(changelogs.get(0).value().getInt(0)).isEqualTo(2);
+ KeyValue kv = result.result();
+ assertThat(kv).isNotNull();
+ assertThat(kv.value().getInt(0)).isEqualTo(2);
+
+ // With level-0, with level-x in lookup, without level-x deleted in
dvMaintainer
+ function.reset();
+ lookup.clear();
+ positionedKv =
+ new LookupLevels.PositionedKeyValue(
+ new KeyValue().replace(row(1), 1, INSERT,
row(2)).setLevel(5), "f2", 1);
+ lookup.put(row(1), positionedKv);
+ // Simulation scenario: (file:"f2", pos: 2) has been deleted but not
(file:"f2", pos: 1)
+ dvMaintainer.notifyNewDeletion("f2", 2);
+ function.add(new KeyValue().replace(row(1), 2, INSERT,
row(2)).setLevel(0));
+ // notifyNewDeletion will return 'true' because (file:"f2", pos: 1)
has not been deleted
+ result = function.getResult();
Review Comment:
NotifyNewDeletion will be called inside "function.getResult()" . Because
(file:"f2", pos: 1) has not been deleted, "modified" of NotifyNewDeletion will
return true (like HashSet.add()). So 'highLevel' will not be null , at the same
time, "...row-deduplicate=true" and the records value are the same, So
changelog will not be generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@paimon.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
