danny0405 commented on code in PR #20412:
URL: https://github.com/apache/hudi/pull/20412#discussion_r4373968980
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -533,7 +568,7 @@ class HoodieSparkSqlWriterInternal {
// scalastyle:on
val writeConfig = client.getConfig
- instantTime = client.startCommit(commitActionType)
+ instantTime = recordStartedInstant(client,
client.startCommit(commitActionType))
Review Comment:
[P2] Handle failures inside startCommit after the heartbeat starts
The outer cleanup now covers failures after startCommit returns, but
argument evaluation means recordStartedInstant is never called if startCommit
itself throws. BaseHoodieWriteClient.startCommit starts the LAZY-policy
heartbeat before createNewInstant/createRequestedCommitWithReplaceMetadata, and
its finally only ends the transaction. If writing the requested instant fails
(or succeeds on storage but reports an error), startedInstant remains None and
releaseStartedInstant does nothing. With ignoreFailedBatch=true, the live
client is retained, so that heartbeat keeps running; a persisted requested
instant remains protected from lazy rollback. Could startCommit clean up its
own heartbeat on exceptional exit, or otherwise expose the allocated instant
before that fallible storage operation? Please add failure injection after
heartbeat startup and before startCommit returns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
