wombatu-kun commented on code in PR #19023:
URL: https://github.com/apache/hudi/pull/19023#discussion_r3432506681
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -547,6 +549,11 @@ private boolean recommitInstant(HoodieTimeline
completedTimeline, long checkpoin
if (writeClient.getConfig().getFailedWritesCleanPolicy().isLazy()) {
writeClient.getHeartbeatClient().start(instant);
}
+ // Initialize the transaction state so that OCC conflict resolution uses
the correct
+ // baseline: the last completed instant before this inflight instant was
created.
Review Comment:
This is still open for the restoreEvents path. A recommitted inflight
instant does not retain an old completion time: transitionStateToComplete ->
createCompleteFileInMetaPath in ActiveTimelineV2 stamps a fresh wall-clock
completion time at recovery, so a sibling A recommitted before B always has
completionTime(A) greater than B's requestedTime. The new two-arg
getLastCompletedTxnInstantAndMetadata then drops A through its completionTime <
currentInstantTime filter, so A is not selected as B's baseline; the baseline
falls back to a pre-restart instant (or empty), and
SimpleConcurrentFileWritesConflictResolutionStrategy's
findInstantsAfter(baseline.requestedTime()) still leaves A in B's candidate
set. With both writes touching the same file groups, 2+ buffered inflight
instants throw the same false conflict; the per-iteration reload makes A
visible but does not change baseline selection.
testCommittingMultipleInstantsWithOCC exercises commitInstants
(checkpointComplete), not restoreEvents, so this path stays uncovered. A
restoreEvents test with two buffered inflight instants under OCC would confirm
it. The recommit baseline likely needs to pick the max-requestedTime completed
instant below currentInstantTime without the completionTime upper bound (the
lastInstant semantics the commitInstants preTxn path already uses), so the
just-recommitted sibling is chosen as the baseline and then excluded by
findInstantsAfter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
