Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20481 Indices in the in-memory store are kinda dumb right now. It should be possible to do something smarter but that would increase memory usage. It would also be good to know what "large cluster" means. Large clusters shouldn't affect the UI responsiveness. Large apps might; but I tried apps with 100k+ tasks on each stage and things seemed fine.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org