wulei0302 commented on code in PR #6963:
URL: https://github.com/apache/hudi/pull/6963#discussion_r1021110814
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java:
##########
@@ -164,20 +166,31 @@ private void init(HoodieRecord record) {
String baseInstantTime;
String baseFile = "";
List<String> logFiles = new ArrayList<>();
+ Option<HoodieInstant> maxCompleteInstant =
hoodieTable.getMetaClient().getActiveTimeline().getWriteTimeline()
+ .filterMorCompactionInstants().lastInstant();
+ if (maxCompleteInstant.isPresent()) {
+ if (fileSlice.isPresent()) {
+ baseInstantTime = fileSlice.get().getBaseInstantTime();
+ } else {
+ baseInstantTime = instantTime;
+ // Handle log file only case. This is necessary for the concurrent
clustering and writer case (e.g., consistent hashing bucket index).
+ // NOTE: flink engine use instantTime to mark operation type, check
BaseFlinkCommitActionExecutor::execute
+ if (record.getCurrentLocation() != null &&
HoodieInstantTimeGenerator.isValidInstantTime(record.getCurrentLocation().getInstantTime()))
{
+ baseInstantTime = record.getCurrentLocation().getInstantTime();
+ }
+ }
+ } else {
+ String instantTime = HoodieActiveTimeline.createNewInstantTime();
+ baseInstantTime = instantTime.substring(0, instantTime.length() - 9) +
String.format("%09d", 0);
Review Comment:
What if the instant time for concurrent writing just spans 24:00. For
example,
instant_ 1 is 20221113235900000,
instant_ 2 is 20221114000100000,
and then instant_ 2 Finish first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
