[
https://issues.apache.org/jira/browse/SPARK-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin reassigned SPARK-25285:
--------------------------------------
Assignee: Luca Canali
> Add executor task metrics to track the number of tasks started and of tasks
> successfully completed
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-25285
> URL: https://issues.apache.org/jira/browse/SPARK-25285
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Luca Canali
> Assignee: Luca Canali
> Priority: Minor
>
> The motivation for these additional metrics is to help in troubleshooting and
> monitoring task execution on a cluster. Currently available metrics include
> executor threadpool metrics for task completed and for active tasks. The
> addition of threadpool tasStarted metric will allow for example to collect
> info on the (approximate) number of failed tasks by computing the difference
> thread started – (active threads + completed tasks and/or successfully
> finished tasks).
> The proposed metric finishedTasks is also intended for this type of
> troubleshooting. The difference between finshedTasks and
> threadpool.completeTasks, is that the latter is a (dropwizard library) gauge
> taken from the threadpool, while the former is a (dropwizard) counter
> computed in the [[Executor]] class, when a task successfully finishes,
> together with several other task metrics counters.
> Note, there are similarities with some of the metrics introduced in
> SPARK-24398, however there are key differences, coming from the fact that
> this PR concerns the executor source, therefore providing metric values per
> executor + metric values do not require to pass through the listerner bus in
> this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
