[GitHub] [spark] cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487462638 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487441589 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-483644141 I've added it to the PR description. let me quote it here > This fix covers the corner case, because: 1. If `DAGScheduler` gets the task completion event from zombie TSM before submitting the new stage attempt, then `DAGScheduler` knows that this partition is completed, and it will exclude this partition when creating task set for the new stage attempt. See `DAGScheduler.submitMissingTasks` 2. If `DAGScheduler` gets the task completion event from zombie TSM after submitting the new stage attempt, then the active TSM is already created. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-483642412 I've updated the PR description, to make it clearer about what the problem is and why propose this fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
