prakharjain09 commented on pull request #28619:
URL: https://github.com/apache/spark/pull/28619#issuecomment-644163889
@attilapiros Thanks for the review.
> One more question: the checkAndSubmitSpeculatableTask checks whether the
number of running copies from the task is one:
spark/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
Line 1007 in 4affa58
if (!successful(index) && copiesRunning(index) == 1 &&
> It could happen that we already have two running copies of the task but
both are running on decommissioning executors (i.o on the same host). I am not
sure whether this is worth to be considered (I mean here or maybe in a followup
PR). What do you think?
Yes it is possible that two copies can be running on decommissioning
executors and so we will end up not speculating for it because of above checks.
I felt it might be a corner case which we can take as separate/follow-up PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
