SteNicholas commented on a change in pull request #2111:
URL: https://github.com/apache/hudi/pull/2111#discussion_r501739210
##########
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/action/commit/TestUpsertPartitioner.java
##########
@@ -90,14 +88,33 @@ private UpsertPartitioner getUpsertPartitioner(int
smallFileSize, int numInserts
List<HoodieRecord> records = new ArrayList<>();
records.addAll(insertRecords);
records.addAll(updateRecords);
- WorkloadProfile profile = new
WorkloadProfile(buildProfile(jsc.parallelize(records)));
+ WorkloadProfile profile = new
WorkloadProfile(buildProfile(jsc.parallelize(records)),
WriteOperationType.UPSERT);
UpsertPartitioner partitioner = new UpsertPartitioner(profile, context,
table, config);
assertEquals(0, partitioner.getPartition(
new Tuple2<>(updateRecords.get(0).getKey(),
Option.ofNullable(updateRecords.get(0).getCurrentLocation()))),
"Update record should have gone to the 1 update partition");
return partitioner;
}
+ private UpsertPartitioner getInsertPartitioner(int smallFileSize, int
numInserts, int fileSize, String testPartitionPath,
+ boolean autoSplitInserts) throws Exception {
+ HoodieWriteConfig config = makeHoodieClientConfigBuilder()
+
.withCompactionConfig(HoodieCompactionConfig.newBuilder().compactionSmallFileSize(smallFileSize)
+
.insertSplitSize(100).autoTuneInsertSplits(autoSplitInserts).build())
Review comment:
@leesf I don't think `100` should be passed via method parameter. You
could refer to the method `getUpsertPartitioner`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
