[
https://issues.apache.org/jira/browse/SPARK-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell updated SPARK-1972:
-----------------------------------
Fix Version/s: (was: 1.2.0)
1.3.0
> Add support for setting and visualizing custom task-related metrics
> -------------------------------------------------------------------
>
> Key: SPARK-1972
> URL: https://issues.apache.org/jira/browse/SPARK-1972
> Project: Spark
> Issue Type: Improvement
> Affects Versions: 0.9.1
> Reporter: Kalpit Shah
> Fix For: 1.3.0
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Various RDDs may want to set/track custom metrics for improved monitoring and
> performance tuning. For e.g.:
> 1. A Task involving a JdbcRDD may want to track some metric related to JDBC
> execution.
> 2. A Task involving a user-defined RDD may want to track some metric specific
> to user's application.
> We currently use TaskMetrics for tracking task-related metrics, which
> provides no way of tracking custom task-related metrics. It is not good to
> introduce a new field in TaskMetric everytime we want to track a custom
> metric. That approach would be cumbersome and ugly. Besides, some of these
> custom metrics may only make sense for a specific RDD-subclass. Therefore, we
> need TaskMetrics to provide a generic way to allow RDD-subclasses to track
> custom metrics when computing partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
