nkarpov commented on issue #25232: [SPARK-28475][CORE] Add regex MetricFilter to GraphiteSink URL: https://github.com/apache/spark/pull/25232#issuecomment-513996865 Hi @dongjoon-hyun! Setup: start a graphite server & add host, port etc. in `metrics.properties` For each scenario below step through w/ debugger to confirm MetricFilter is registered and corresponding metrics are posted to graphite server: 1. Did not provide `driver.sink.graphite.regex` in `metrics.properties` -> successfully defaults to MetricFilter.ALL & all (driver) metrics reported 2. Did provide `driver.sink.graphite.regex=streaming` in `metrics.properties`, enable streaming metrics configuration (`spark.conf.set("spark.sql.streaming.metricsEnabled", true)`) & start a MemoryStream to console -> successfully registers anonymous MetricFilter in this PR and reports the 6 metrics that are expected to match regex expression `streaming` I couldn't find an existing testing suite for the GraphiteSink so I verified manually as above. The change seemed minor enough for that to be OK but let me know if a more robust suite should be added. Also, if we're happy with the naming convention here, I will add documentation in the following files as part of the PR https://github.com/apache/spark/blob/master/docs/monitoring.md & https://github.com/apache/spark/blob/master/conf/metrics.properties.template
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
