GitHub user LucaCanali opened a pull request:
https://github.com/apache/spark/pull/22279
[SPARK-25277][YARN] YARN applicationMaster metrics should not register
static metrics
## What changes were proposed in this pull request?
YARN applicationMaster metrics registration introduced in SPARK-24594
causes further registration of static metrics (Codegenerator and
HiveExternalCatalog) and of JVM metrics, which I believe do not belong in this
context.
This looks like an unintended side effect of using the start method of
[[MetricsSystem]].
A possible solution proposed here, is to introduce startNoRegisterSources
to avoid these additional registrations of static sources and of JVM sources in
the case of YARN applicationMaster metrics (this could be useful for other
metrics that may be added in the future).
## How was this patch tested?
Manually tested on a YARN cluster,
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/LucaCanali/spark
YarnMetricsRemoveExtraSourceRegistration
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22279.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22279
----
commit a99075873fa1519fe07344eb90d5c8af02b9d7e5
Author: LucaCanali <luca.canali@...>
Date: 2018-08-30T07:07:47Z
YARN applicationMaster metrics should not register static metrics
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
