[
https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enrico Minack updated SPARK-30664:
----------------------------------
Attachment: Show Additional Metrics.png
> Add more metrics to the all stages page
> ---------------------------------------
>
> Key: SPARK-30664
> URL: https://issues.apache.org/jira/browse/SPARK-30664
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 3.0.0
> Reporter: Enrico Minack
> Priority: Minor
> Attachments: Show Additional Metrics.png
>
>
> The web UI page for individual stages has many useful metrics to diagnose
> poorly performing stages, e.g. spilled bytes or GC time. Identifying those
> stages among hundreds or thousands of stages is cumbersome, as you have to
> click through all stages on the all stages page. The all stages page should
> host more metrics from the individual stages page like
> - Peak Execution Memory
> - Spill (Memory)
> - Spill (Disk)
> - GC Time
> These additional metrics would make the page more complex, so showing them
> should be optional. The individual stages page hides some metrics under "Show
> Additional Metrics". Those new metrics on the all stages page should also be
> made optional in the same way.
> Existing metrics like
> - Input
> - Output
> - Shuffle Read
> - Shuffle Write
> could be made optional as well and active by default. Then users can remove
> them if they want but get the same view as now by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
