Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22883#discussion_r229805894
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -1105,6 +1095,15 @@ private[spark] class AppStatusListener(
cleanupCachedQuantiles(key)
}
+
+ // Delete tasks for all stages in one pass, as deleting them for each
stage individually is slow
+ val tasks = kvstore.view(classOf[TaskDataWrapper]).asScala
+ val keys = stages.map(s => (s.info.stageId, s.info.attemptId)).toSet
--- End diff --
nit: `.map { s => ... }`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
