[ https://issues.apache.org/jira/browse/YARN-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junfan Zhang updated YARN-9746: ------------------------------- Attachment: YARN-9746-01.patch > Rm should only rewrite partial jobConf passed by app when supporting > multi-cluster token renew > ---------------------------------------------------------------------------------------------- > > Key: YARN-9746 > URL: https://issues.apache.org/jira/browse/YARN-9746 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Junfan Zhang > Priority: Major > Attachments: YARN-9746-01.patch > > > This issue links to YARN-5910. > When to support multi-cluster delegation token renew, the path of YARN-5910 > works in most scenarios. > But when intergrating with Oozie, we encounter some problems. In Oozie having > multi delegation tokens including HDFS_DELEGATION_TOKEN(another cluster HA > token) and MR_DELEGATION_TOKEN(Oozie mr launcher token), to support renew > another cluster's token, YARN-5910 was patched and related config was set. > The config is as follows > {code:xml} > <property> > <name>mapreduce.job.send-token-conf</name> > > <value>dfs.namenode.kerberos.principal|dfs.nameservices|^dfs.namenode.rpc-address.*$|^dfs.ha.namenodes.*$|^dfs.client.failover.proxy.provider.*$</value> > </property> > <property> > <name>dfs.nameservices</name> > > <value>hadoop-clusterA-ns01,hadoop-clusterA-ns02,hadoop-clusterA-ns03,hadoop-clusterA-ns04,hadoop-clusterB-ns01,hadoop-clusterB-ns02,hadoop-clusterB-ns03,hadoop-clusterB-ns04</value> > </property> > <property> > <name>dfs.ha.namenodes.hadoop-clusterB-ns01</name> > <value>nn1,nn2</value> > </property> > <property> > > <name>dfs.namenode.rpc-address.hadoop-clusterB-ns01.nn1</name> > <value>namenode01-clusterB.qiyi.hadoop:8020</value> > </property> > <property> > > <name>dfs.namenode.rpc-address.hadoop-clusterB-ns01.nn2</name> > <value>namenode02-clusterB.qiyi.hadoop:8020</value> > </property> > <property> > > <name>dfs.client.failover.proxy.provider.hadoop-clusterB-ns01</name> > > <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> > </property> > {code} > However, the MR_DELEGATION_TOKEN could‘t be renewed, because of lacking some > config. Although we can set the required configurations through the app, this > is not a good idea. So i think rm should only rewrite the jobConf passed by > app to solve the above situation. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org