Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20481
Indices in the in-memory store are kinda dumb right now. It should be
possible to do something smarter but that would increase memory usage.
It would also be good to know what "large cluster" means. Large clusters
shouldn't affect the UI responsiveness. Large apps might; but I tried apps with
100k+ tasks on each stage and things seemed fine.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
