[ https://issues.apache.org/jira/browse/SPARK-27641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-27641: ------------------------------------ Assignee: Apache Spark > Unregistering a single Metrics Source with no metrics leads to removing all > the metrics from other sources with the same name > ----------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-27641 > URL: https://issues.apache.org/jira/browse/SPARK-27641 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.2.3, 2.3.3, 2.4.2 > Reporter: Sergey Zhemzhitsky > Assignee: Apache Spark > Priority: Major > > Currently Spark allows registering multiple Metric Sources with the same > source name like the following > {code:scala} > val acc1 = sc.longAccumulator > LongAccumulatorSource.register(sc, {"acc1" -> acc1}) > val acc2 = sc.longAccumulator > LongAccumulatorSource.register(sc, {"acc2" -> acc2}) > {code} > In that case there are two metric sources registered and both of these > sources have the same name - > [AccumulatorSource|https://github.com/apache/spark/blob/6ef45301a46c47c12fbc74bb9ceaffea685ed944/core/src/main/scala/org/apache/spark/metrics/source/AccumulatorSource.scala#L47] > If you try to unregister the source with no accumulators and metrics > registered like the following > {code:scala} > SparkEnv.get.metricsSystem.removeSource(new LongAccumulatorSource) > {code} > ... then all the metrics for all the sources with the same name will be > unregistered because of the > [following|https://github.com/apache/spark/blob/6ef45301a46c47c12fbc74bb9ceaffea685ed944/core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala#L171] > snippet which removes all matching records which start with the > corresponding prefix which includes the source name, but does not include > metric name to be removed. > {code:scala} > def removeSource(source: Source) { > sources -= source > val regName = buildRegistryName(source) > registry.removeMatching((name: String, _: Metric) => > name.startsWith(regName)) > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org