Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/22221#discussion_r213071006
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl(
var reason: Option[ExecutorLossReason] = None
synchronized {
try {
- taskIdToTaskSetManager.get(tid) match {
+ Option(taskIdToTaskSetManager.get(tid)) match {
--- End diff --
yes as far as I can see its safe. If the get happened before its removed
it calculates the accumulators, if its after its removed it just gets an empty
array back. This isn't any different then when it was synchronized. There is
nothing in the statusUpdate between the get and call to cleanupTaskState where
it removes that I see depends on accumulators or anything else.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
