[
https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-30664:
----------------------------------
Affects Version/s: (was: 3.0.0)
3.1.0
> Add more metrics to the all stages page
> ---------------------------------------
>
> Key: SPARK-30664
> URL: https://issues.apache.org/jira/browse/SPARK-30664
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 3.1.0
> Reporter: Enrico Minack
> Priority: Minor
> Attachments: image-2020-01-28-16-12-49-807.png,
> image-2020-01-28-16-13-36-174.png, image-2020-01-28-16-15-20-258.png
>
>
> The web UI page for individual stages has many useful metrics to diagnose
> poorly performing stages, e.g. spilled bytes or GC time. Identifying those
> stages among hundreds or thousands of stages is cumbersome, as you have to
> click through all stages on the all stages page. The all stages page should
> host more metrics from the individual stages page like
> - Peak Execution Memory
> - Spill (Memory)
> - Spill (Disk)
> - GC Time
> These additional metrics would make the page more complex, so showing them
> should be optional. The individual stages page hides some metrics under
> !image-2020-01-28-16-12-49-807.png! . Those new metrics on the all stages
> page should also be made optional in the same way.
> !image-2020-01-28-16-13-36-174.png!
> Existing metrics like
> - Input
> - Output
> - Shuffle Read
> - Shuffle Write
> could be made optional as well and active by default. Then users can remove
> them if they want but get the same view as now by default.
> The table extends as additional metrics get checked / unchecked:
> !image-2020-01-28-16-15-20-258.png!
> Sorting the table by metrics allows to find the stages with highest GC time
> or spilled bytes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
