sunchao commented on a change in pull request #34635:
URL: https://github.com/apache/spark/pull/34635#discussion_r753456919
##########
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
##########
@@ -78,6 +78,20 @@ package object config extends Logging {
.booleanConf
.createWithDefault(false)
+ private[spark] val AM_SEND_TOKEN_CONF =
+ ConfigBuilder("spark.yarn.am.sendTokenConf")
+ .doc("This config is only supported for Hadoop 3.x profile. The value of
this config is a " +
+ "regex expression used to grep a list of config entries from the job's
configuration " +
+ "file (e.g., hdfs-site.xml) and send to RM, which uses them when
renewing delegation " +
+ "tokens. A typical use case of this feature is to support delegation
tokens in an " +
+ "environment where a YARN cluster needs to talk to multiple downstream
HDFS clusters, " +
+ "where the YARN RM may not have configs (e.g., dfs.nameservices,
dfs.ha.namenodes.*, " +
+ "dfs.namenode.rpc-address.*) to connect to these clusters. This config
is very similar " +
+ "to 'mapreduce.job.send-token-conf'. Please check YARN-5910 for more
details.")
Review comment:
Good suggestion. Will add.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
