[
https://issues.apache.org/jira/browse/SPARK-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell updated SPARK-8667:
-----------------------------------
Component/s: Web UI
> Improve Spark UI behavior at scale
> ----------------------------------
>
> Key: SPARK-8667
> URL: https://issues.apache.org/jira/browse/SPARK-8667
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Reporter: Patrick Wendell
> Assignee: Shixiong Zhu
>
> This is a parent ticket and we can create child tickets when solving specific
> issues. The main problem I would like to solve is the fact that the Spark UI
> has issues at very large scale.
> The worst issue is when there is a stage page with more than a few thousand
> tasks. In this case:
> 1. The page itself is very slow to load and becomes unresponsive with huge
> number of tasks.
> 2. The Scala XML output can become so large that it crashes the driver
> program due to OOM for a page with a huge number of tasks.
> I am not sure if (1) is caused by javascript slowness, or maybe just the raw
> amount of data sent over the wire. If it is the latter, it might be possible
> to add compression to the HTTP payload to help improve load time.
> It would be nice to reproduce+investigate these issues further and create
> specific sub tasks to improve them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
