Janos Matyas created SPARK-22343:
------------------------------------
Summary: Add support for publishing Spark metrics into Prometheus
Key: SPARK-22343
URL: https://issues.apache.org/jira/browse/SPARK-22343
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 2.2.0
Reporter: Janos Matyas
I've created a PR for supporting Prometheus a metric sink for Spark - in the
https://github.com/apache-spark-on-k8s/spark fork. Based on the maintainers of
the project I should create an issue here as well, in order to be tracked
upstream as well. See below the original text of the PR:
_
Publishing Spark metrics into Prometheus - as discussed earlier in #384.
Implemented a metrics sink that publishes Spark metrics into Prometheus via
[Prometheus Pushgateway](https://prometheus.io/docs/instrumenting/pushing/).
Metrics data published by Spark is based on
[Dropwizard](http://metrics.dropwizard.io/). The format of Spark metrics is not
supported natively by Prometheus thus these are converted using
[DropwizardExports](https://prometheus.io/client_java/io/prometheus/client/dropwizard/DropwizardExports.html)
prior pushing metrics to the pushgateway.
Also the default Prometheus pushgateway client API implementation does not
support metrics timestamp thus the client API has been ehanced to enrich
metrics data with timestamp. _
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
