danny0405 commented on PR #19046: URL: https://github.com/apache/hudi/pull/19046#issuecomment-5351517103
> @danny0405 What are your findings around the OCC/correctness questions we discussed.. The current conflict resolution happens right before the Hudi commit, current Flink try to commit to Hudi timeline after receiving a checkpoint ack msg(which indicates the checkpoint is already successful to Flink signt of view), let's say it is `ckp_n`, when the exception throws because of the confclit resolution, the commit procedure aborted and trigger the job to restart, but the job by default restore from the last successful checkpoint `ckp_n` which is actually invalid to Hudi. I have fire a similar discuss here before for the rocksDB statebackend solution: https://github.com/apache/hudi/discussions/18296, and there are some explanaitons and also proposals in the discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
