dongjoon-hyun commented on a change in pull request #34635:
URL: https://github.com/apache/spark/pull/34635#discussion_r751875313
##########
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
##########
@@ -78,6 +78,20 @@ package object config extends Logging {
.booleanConf
.createWithDefault(false)
+ private[spark] val AM_SEND_TOKEN_CONF =
+ ConfigBuilder("spark.yarn.am.sendTokenConf")
+ .doc("The value of this config is a regex expression used to grep a
list of " +
+ "config entries from the job's configuration file (e.g.,
hdfs-site.xml) and send to " +
+ "RM, which uses them when renewing delegation tokens. A typical use
case of " +
+ "this feature is to support delegation tokens in an environment
where a YARN cluster " +
+ "needs to talk to multiple downstream HDFS clusters, where the YARN
RM may not have " +
+ "configs (e.g., dfs.nameservices, dfs.ha.namenodes.*,
dfs.namenode.rpc-address.*)" +
+ "to connect to these clusters. This config is very similar to " +
+ "'mapreduce.job.send-token-conf'. Please check YARN-5910 for more
details.")
Review comment:
We had better mention explicitly that this config is ignored in Hadoop
2.7 because we still have Hadoop 2.7 distribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
