Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22371#discussion_r216167004
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -76,6 +79,13 @@ private[spark] class IndexShuffleBlockResolver(
logWarning(s"Error deleting index ${file.getPath()}")
}
}
+
+ // This should be called when we unregister shuffle from
ShuffleManager, so it's safe to set
+ // null for given shuffleId.
+ val locks = shuffleIdToLocks.get(shuffleId)
--- End diff --
Just `shuffleIdLocks.remove(shuffleId)` ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
