zhengchenyu commented on code in PR #4899:
URL: https://github.com/apache/hive/pull/4899#discussion_r2297765793
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezProcessor.java:
##########
@@ -293,6 +293,13 @@ protected void initializeAndRunProcessor(Map<String,
LogicalInput> inputs,
rproc.run();
perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.TEZ_RUN_PROCESSOR);
+
+ // Try to call canCommit to AM. If there is no other speculative attempt
execute canCommit, then continue.
+ // If there are other speculative attempt execute canCommit first, then
wait until the attempt is killed
+ // or the committed task fails.
+ while (!this.processorContext.canCommit()) {
Review Comment:
@deniskuzZ Sorry, I don't understand you. Do you means replace sleep with
callback and do not block?
I try to enable speculative execution. The purpose of this is to avoid
simultaneous commits. If a commit occurs simultaneously, the slower attempt
will sleep, and then when the faster attempt completes, the slower sleep
attempt will be killed, thus exiting the sleep loop. If the slower attempt does
not block here, the slower attempt and the faster attempt may commit the file
in the close method at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
