HeartSaVioR commented on a change in pull request #23798: [SPARK-26890][DOC] Add list of available Dropwizard metrics in Spark and add additional configuration details to the monitoring documentation URL: https://github.com/apache/spark/pull/23798#discussion_r259687249
########## File path: docs/monitoring.md ########## @@ -684,9 +688,256 @@ code in your Spark package**_. For sbt users, set the the `-Pspark-ganglia-lgpl` profile. In addition to modifying the cluster's Spark build user applications will need to link to the `spark-ganglia-lgpl` artifact. -The syntax of the metrics configuration file is defined in an example configuration file, +The syntax of the metrics configuration file and the parameters available for each sink are defined +in an example configuration file, `$SPARK_HOME/conf/metrics.properties.template`. +When using configuration parameters instead of the metrics configuration file, the relevant +parameter names have the following form: +`spark.metrics.conf.[instance|*].sink.[sink_name].[parameter_name]`. +This example shows a list of configuration parameters for a Graphite sink: +``` +"spark.metrics.conf.*.sink.graphite.class"="org.apache.spark.metrics.sink.GraphiteSink" +"spark.metrics.conf.*.sink.graphite.host"="graphiteEndPoint_hostName>" +"spark.metrics.conf.*.sink.graphite.port"=<graphite_listening_port> +"spark.metrics.conf.*.sink.graphite.period"=10 +"spark.metrics.conf.*.sink.graphite.unit"=seconds +"spark.metrics.conf.*.sink.graphite.prefix"="optional_prefix" +``` + +Default values of the Spark metrics configuration are as follows: +``` +"*.sink.servlet.class" = "org.apache.spark.metrics.sink.MetricsServlet" +"*.sink.servlet.path" = "/metrics/json" +"master.sink.servlet.path" = "/metrics/master/json" +"applications.sink.servlet.path" = "/metrics/applications/json" +``` + +Additional sources can be configured using the metrics configuration file or the configuration Review comment: If this can also done with adding them into `metrics.properties`, maybe better to add them to `metrics.properties.template`. Just my 2 cents. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
