[
https://issues.apache.org/jira/browse/SPARK-44862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766633#comment-17766633
]
Ruifeng Zheng commented on SPARK-44862:
---------------------------------------
I think the added metric should be included
[here|https://spark.apache.org/docs/latest/monitoring.html#component-instance--master]
!image-2023-09-19-11-21-55-416.png!
> Adding metric tracking number of jobs running on a cluster
> ----------------------------------------------------------
>
> Key: SPARK-44862
> URL: https://issues.apache.org/jira/browse/SPARK-44862
> Project: Spark
> Issue Type: New Feature
> Components: Web UI
> Affects Versions: 4.0.0
> Reporter: Ruifeng Zheng
> Assignee: Kent Yao
> Priority: Major
> Attachments: image-2023-09-19-11-21-55-416.png
>
>
> We frequently come across issues where in there are considerable number of
> jobs/notebooks running on one interactive cluster. This leads to either
> degraded performance due to cluster being overloaded or leads to driver OOM.
> It would be helpful to have a metric identifying number of notebooks/jobs
> running on a cluster as this would be important datapoint for customers to
> balance out their workloads.
> We propose to add a metric which keeps track of number of jobs running on a
> cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
