[
https://issues.apache.org/jira/browse/SPARK-44862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759459#comment-17759459
]
Kent Yao commented on SPARK-44862:
----------------------------------
There is a gauge named activeJobs in the DAGSchedulerSource. Is it satisfied?
> Adding metric tracking number of jobs running on a cluster
> ----------------------------------------------------------
>
> Key: SPARK-44862
> URL: https://issues.apache.org/jira/browse/SPARK-44862
> Project: Spark
> Issue Type: New Feature
> Components: Web UI
> Affects Versions: 4.0.0
> Reporter: Ruifeng Zheng
> Assignee: Kent Yao
> Priority: Major
>
> We frequently come across issues where in there are considerable number of
> jobs/notebooks running on one interactive cluster. This leads to either
> degraded performance due to cluster being overloaded or leads to driver OOM.
> It would be helpful to have a metric identifying number of notebooks/jobs
> running on a cluster as this would be important datapoint for customers to
> balance out their workloads.
> We propose to add a metric which keeps track of number of jobs running on a
> cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
