danny0405 commented on code in PR #9776:
URL: https://github.com/apache/hudi/pull/9776#discussion_r1343421100
##########
hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java:
##########
@@ -544,15 +543,40 @@ public static Option<Pair<Integer, String>>
getLatestLogVersion(FileSystem fs, P
}
/**
- * computes the next log version for the specified fileId in the partition
path.
+ * Get the latest log version for the fileId in the partition path.
+ */
+ public static int getLatestLogVersion(FileSystem fs, Path partitionPath,
final String fileId,
+ final String
logFileExtension, final String deltaCommitTime, final String writeToken) throws
IOException {
+ int version = HoodieLogFile.LOGFILE_BASE_VERSION;
+ String logFileName = makeLogFileName(fileId, logFileExtension,
deltaCommitTime, version, writeToken);
+ while (fileExists(fs, new Path(partitionPath, logFileName))) {
Review Comment:
You are right, actually we already do that, in the `LogFormatWriter` we
would check the file existence anyway when creating a new file, and here we
also do a check for generating the next version number, in `LogFormatWriter`,
can maintain the latest used version number there, so each time we start from
the latest number.
You mean to remove the file existence check totally, I'm fine with that. Do
you think in `HoodieLogFormatWriter.getOutputStream`, we should also remove the
existence check for creating new files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
