vinothchandar commented on a change in pull request #3644:
URL: https://github.com/apache/hudi/pull/3644#discussion_r772800849
##########
File path:
hudi-client/hudi-java-client/src/test/java/org/apache/hudi/table/action/commit/TestJavaCopyOnWriteActionExecutor.java
##########
@@ -480,4 +478,76 @@ public void testBulkInsertRecords(String bulkInsertMode)
throws Exception {
records1.addAll(records2);
return records1;
}
+
+ @Test
+ public void testHoodieConcatHandle() throws Exception {
+ HoodieWriteConfig config =
makeHoodieClientConfigBuilder().withMergeAllowDuplicateOnInserts(true).build();
+ String firstCommitTime = makeNewCommitTime();
+ HoodieJavaWriteClient writeClient = getHoodieWriteClient(config);
+ metaClient = HoodieTableMetaClient.reload(metaClient);
+ BaseFileUtils fileUtils = BaseFileUtils.getInstance(metaClient);
+
+ String partitionPath = "2021/09/11";
+
+ // Get some records belong to the same partition (2021/09/11)
+ String insertRecordStr1 = "{\"_row_key\":\"1\","
+ + "\"time\":\"2021-09-11T16:16:41.415Z\",\"number\":1}";
+ String insertRecordStr2 = "{\"_row_key\":\"2\","
+ + "\"time\":\"2021-09-11T16:16:41.415Z\",\"number\":2}";
+ List<HoodieRecord> records1 = new ArrayList<>();
+ RawTripTestPayload insertRow1 = new RawTripTestPayload(insertRecordStr1);
+ RawTripTestPayload insertRow2 = new RawTripTestPayload(insertRecordStr2);
+ records1.add(new HoodieRecord(new HoodieKey(insertRow1.getRowKey(),
insertRow1.getPartitionPath()), insertRow1));
+ records1.add(new HoodieRecord(new HoodieKey(insertRow2.getRowKey(),
insertRow2.getPartitionPath()), insertRow2));
+
+ // First insert
+ writeClient.startCommitWithTime(firstCommitTime);
+ writeClient.insert(records1, firstCommitTime);
+
+ FileStatus[] allFiles = getIncrementalFiles(partitionPath, "0", -1);
+ assertEquals(1, allFiles.length);
+
+ // Read out the bloom filter and make sure filter can answer record exist
or not
+ Path filePath = allFiles[0].getPath();
+ BloomFilter filter = fileUtils.readBloomFilterFromMetadata(hadoopConf,
filePath);
+ for (HoodieRecord record : records1) {
+ assertTrue(filter.mightContain(record.getRecordKey()));
+ }
+
+ insertRecordStr1 = "{\"_row_key\":\"1\","
+ + "\"time\":\"2021-09-11T16:39:41.415Z\",\"number\":3}";
+ insertRecordStr2 = "{\"_row_key\":\"2\","
+ + "\"time\":\"2021-09-11T16:39:41.415Z\",\"number\":4}";
+
+ List<HoodieRecord> records2 = new ArrayList<>();
+ insertRow1 = new RawTripTestPayload(insertRecordStr1);
+ insertRow2 = new RawTripTestPayload(insertRecordStr2);
+ records2.add(new HoodieRecord(new HoodieKey(insertRow1.getRowKey(),
insertRow1.getPartitionPath()), insertRow1));
+ records2.add(new HoodieRecord(new HoodieKey(insertRow2.getRowKey(),
insertRow2.getPartitionPath()), insertRow2));
+
+ Thread.sleep(1000);
Review comment:
please avoid relying on sleep in tests. You can generate the commit
times in the test, for greater control? Also could we move this test as a
separate one for `HoodieConcatHandle`?
##########
File path:
hudi-client/hudi-java-client/src/test/java/org/apache/hudi/table/action/commit/TestJavaCopyOnWriteActionExecutor.java
##########
@@ -480,4 +478,76 @@ public void testBulkInsertRecords(String bulkInsertMode)
throws Exception {
records1.addAll(records2);
return records1;
}
+
+ @Test
+ public void testHoodieConcatHandle() throws Exception {
Review comment:
Wondeing if this can be rewritten more nicely using the TestTable
abstraction. cc @xushiyan .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
