[
https://issues.apache.org/jira/browse/SPARK-36360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Neumann updated SPARK-36360:
-----------------------------------
Description:
The StreamingSource includes the appName in its sourceName. This is not desired
for people using a custom namespace for metrics reporting using
{{spark.metrics.namespace}} configuration property as the {{spark.app.name}}
cannot be excluded in the name of the metric. Using a metrics namespace results
in a duplicated indicator for {{spark.app.name}}.
All other metrics do not include the appName in their sourceName, e.g.
ExecutorMetricsSource. If desired you can configure that with a metrics
namespace. The same should be done in StreamingSource.
was:
The StreamingSource includes the appName in its sourceName. This is not desired
for people using a custom namespace for metrics reporting using
{{spark.metrics.namespace}} configuration property as the {{spark.app.name}}
cannot be excluded in the name of the metric. Using a metrics namespace results
in a duplicated indicator for {{spark.app.name}}.
All other metrics do not include the appName in their sourceName, e.g.
ExecutorMetricsSource. If desired you can configure that with a metrics
namespace.
> StreamingSource duplicates appName
> ----------------------------------
>
> Key: SPARK-36360
> URL: https://issues.apache.org/jira/browse/SPARK-36360
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.1.2
> Reporter: Marcel Neumann
> Priority: Blocker
>
> The StreamingSource includes the appName in its sourceName. This is not
> desired for people using a custom namespace for metrics reporting using
> {{spark.metrics.namespace}} configuration property as the {{spark.app.name}}
> cannot be excluded in the name of the metric. Using a metrics namespace
> results in a duplicated indicator for {{spark.app.name}}.
> All other metrics do not include the appName in their sourceName, e.g.
> ExecutorMetricsSource. If desired you can configure that with a metrics
> namespace. The same should be done in StreamingSource.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
