YannByron commented on code in PR #6476:
URL: https://github.com/apache/hudi/pull/6476#discussion_r970473374
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -273,6 +283,33 @@ protected HoodieFileWriter createNewFileWriter(String
instantTime, Path path, Ho
return HoodieFileWriterFactory.getFileWriter(instantTime, path,
hoodieTable, config, schema, taskContextSupplier);
}
+ protected HoodieLogFormat.Writer createLogWriter(
+ Option<FileSlice> fileSlice, String baseCommitTime) throws IOException {
+ int logVersion = HoodieLogFile.LOGFILE_BASE_VERSION;
+ long logFileSize = 0L;
+ String logWriteToken = writeToken;
+ if (fileSlice.isPresent()) {
+ Option<HoodieLogFile> latestLogFileOpt =
fileSlice.get().getLatestLogFile();
+ if (latestLogFileOpt.isPresent()) {
+ HoodieLogFile latestLogFile = latestLogFileOpt.get();
+ logVersion = latestLogFile.getLogVersion();
+ logFileSize = latestLogFile.getFileSize();
+ logWriteToken =
FSUtils.getWriteTokenFromLogPath(latestLogFile.getPath());
+ }
+ }
+ return HoodieLogFormat.newWriterBuilder()
+
.onParentPath(FSUtils.getPartitionPath(hoodieTable.getMetaClient().getBasePath(),
partitionPath))
+ .withFileId(fileId)
+ .overBaseCommit(baseCommitTime)
+ .withLogVersion(logVersion)
+ .withFileSize(logFileSize)
+ .withSizeThreshold(config.getLogFileMaxSize())
+ .withFs(fs)
+ .withRolloverLogWriteToken(writeToken)
+ .withLogWriteToken(logWriteToken)
+ .withFileExtension(HoodieLogFile.DELTA_EXTENSION).build();
Review Comment:
the cdc log file with `CDCBlock` is a normal log file, same with others. But
when read mor in non-cdc mode, we will skip the cdc block directly only with
the cost of reading the first few bytes from InputStream of the log files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
