xushiyan commented on a change in pull request #2143:
URL: https://github.com/apache/hudi/pull/2143#discussion_r502124874
##########
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieWriteableTestTable.java
##########
@@ -128,4 +148,37 @@ public HoodieWriteableTestTable withInserts(String
partition, String fileId, Hoo
return this;
}
+
+ public HoodieWriteableTestTable withLogAppends(HoodieRecord... records)
throws Exception {
+ return withLogAppends(Arrays.asList(records));
+ }
+
+ public HoodieWriteableTestTable withLogAppends(List<HoodieRecord> records)
throws Exception {
+ for (List<HoodieRecord> groupedRecords: records.stream()
+
.collect(Collectors.groupingBy(HoodieRecord::getCurrentLocation)).values()) {
+ appendRecordsToLogFile(groupedRecords);
+ }
+ return this;
+ }
+
+ private void appendRecordsToLogFile(List<HoodieRecord> groupedRecords)
throws Exception {
+ String partitionPath = groupedRecords.get(0).getPartitionPath();
+ HoodieRecordLocation location = groupedRecords.get(0).getCurrentLocation();
+ try (HoodieLogFormat.Writer logWriter =
HoodieLogFormat.newWriterBuilder().onParentPath(new Path(basePath,
partitionPath))
+
.withFileExtension(HoodieLogFile.DELTA_EXTENSION).withFileId(location.getFileId())
+ .overBaseCommit(location.getInstantTime()).withFs(fs).build()) {
+ Map<HoodieLogBlock.HeaderMetadataType, String> header = new HashMap<>();
+ header.put(HoodieLogBlock.HeaderMetadataType.INSTANT_TIME,
location.getInstantTime());
+ header.put(HoodieLogBlock.HeaderMetadataType.SCHEMA, schema.toString());
+ logWriter.appendBlock(new
HoodieAvroDataBlock(groupedRecords.stream().map(r -> {
+ try {
+ GenericRecord val = (GenericRecord)
r.getData().getInsertValue(schema).get();
+ HoodieAvroUtils.addHoodieKeyToRecord(val, r.getRecordKey(),
r.getPartitionPath(), "");
+ return (IndexedRecord) val;
+ } catch (IOException e) {
Review comment:
@yanghua ok fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
