Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22883#discussion_r229535706
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -1105,6 +1095,15 @@ private[spark] class AppStatusListener(
cleanupCachedQuantiles(key)
}
+
+ // Delete tasks for all stages in one pass, as deleting them for each
stage individually is slow
--- End diff --
Hi @patrickbrownsync , can you explain more about why deleting them in the
loop above is slower? I don't quite understand it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
