[
https://issues.apache.org/jira/browse/SPARK-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291007#comment-14291007
]
Apache Spark commented on SPARK-5401:
-------------------------------------
User 'ryan-williams' has created a pull request for this issue:
https://github.com/apache/spark/pull/4194
> Executor ID should be set before MetricsSystem is created
> ---------------------------------------------------------
>
> Key: SPARK-5401
> URL: https://issues.apache.org/jira/browse/SPARK-5401
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Ryan Williams
>
> MetricsSystem construction [attempts to namespace metrics from each executor
> using that executor's
> ID|https://github.com/apache/spark/blob/0d1e67ee9b29b51bccfc8a319afe9f9b4581afc8/core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala#L131].
> The ID is [currently set at Executor construction
> time|https://github.com/apache/spark/blob/0d1e67ee9b29b51bccfc8a319afe9f9b4581afc8/core/src/main/scala/org/apache/spark/executor/Executor.scala#L76-L79]
> (uncoincidentally, just before the {{ExecutorSource}} is registered), but
> this is after the {{MetricsSystem}} has been initialized (which [happens
> during {{SparkEnv}}
> construction|https://github.com/apache/spark/blob/0d1e67ee9b29b51bccfc8a319afe9f9b4581afc8/core/src/main/scala/org/apache/spark/SparkEnv.scala#L323-L332],
> which itself happens during {{ExecutorBackend}} construction, *before*
> {{Executor}} construction).
> I noticed this problem because I wasn't seeing any JVM metrics from my
> executors in a Graphite dashboard I've set up; turns out all the executors
> (and the driver) were namespacing their metrics under "<driver>", and
> Graphite responds to such a situation by only taking the last value it
> receives for each "metric" within a configurable time window (e.g. 10s). I
> was seeing per-executor metrics, properly namespaced with each executor's ID,
> from {{ExecutorSource}}, which as I mentioned above is registered after the
> executor ID is set.
> I have a one-line fix for this that I will submit shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
