xkrogen commented on a change in pull request #31936:
URL: https://github.com/apache/spark/pull/31936#discussion_r600691511
##########
File path:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
##########
@@ -75,6 +76,15 @@
* is because an application running on the same Yarn cluster may choose to
not use the external
* shuffle service, in which case its setting of `spark.authenticate` should
be independent of
* the service's.
+ *
+ * The shuffle service will produce metrics via the YARN NodeManager's {@code
metrics2} system
+ * under a namespace specified by the {@value
SPARK_SHUFFLE_SERVICE_METRICS_NAMESPACE_KEY} config.
+ *
+ * By default, all configurations for the shuffle service will be taken
directly from the
+ * Hadoop {@link Configuration} passed by the YARN NodeManager. It is also
possible to configure
+ * the shuffle service by placing a resource named
+ * {@value SHUFFLE_SERVICE_CONF_OVERLAY_RESOURCE_NAME} into the classpath,
which should be an
Review comment:
This will work on YARN 2.7 and older as well. Made this more explicit
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
