Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22221#discussion_r213005226
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl(
var reason: Option[ExecutorLossReason] = None
synchronized {
try {
- taskIdToTaskSetManager.get(tid) match {
+ Option(taskIdToTaskSetManager.get(tid)) match {
--- End diff --
`ConcurrentHashMap` make each operation like `get()`, `remove()`, and
others. Thus, I reviewed places more than one operations are within a
`synchronized`. The place is here.
When we apply this PR, the `get` in `accumUpdatesWithTaskIds` can be
executed between `get()` and `remove()`. My question is like a confirmation
whether it is safe or not.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
