Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/19751#discussion_r156716551
--- Diff: core/src/main/scala/org/apache/spark/status/KVUtils.scala ---
@@ -68,6 +69,19 @@ private[spark] object KVUtils extends Logging {
db
}
+ /** Turns a KVStoreView into a Scala sequence, applying a filter. */
+ def viewToSeq[T](
--- End diff --
looks like you don't actually need a sequence at all in any of the call
sites, you could just use an iterator. I'm thinking about the price of that
say if you're cleaning up 100k tasks repeatedly.
This does give you a nice spot to include `iter.close()`, but I think you
could change this to `foreachWithMaxFilterClose` or something to avoid ever
creating the list.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
