[ https://issues.apache.org/jira/browse/YARN-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361017#comment-16361017 ]
Daryn Sharp commented on YARN-7922: ----------------------------------- This shouldn't be able to happen. Distributed shell gets the renewer from {{YarnClientUtils.getRmPrincipal}} which calls {{SecurityUtil.getServerPrincipal}} to substitute _HOST. Yet somehow the substitution did not occur. The most conceivable, yet unlikely, way I see this failing is the principal has more than 3 components, ie. <REALM> contains another / or @, which would cause the substitution to short-out. > Yarn dont resolve rm/_HOST to hostname > -------------------------------------- > > Key: YARN-7922 > URL: https://issues.apache.org/jira/browse/YARN-7922 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn > Affects Versions: 2.7.3 > Reporter: Berry Österlund > Priority: Minor > > The normal auth_to_local usually removes everything after the / in the > username of the Kerberos principle. That, together with the _HOST setting in > the configuration files specifying the Kerberos principles is usually what is > required to convert rm/_HOST@<REALM> to user yarn. > In our environment, we cant use the default rules in auth_to_local. We have > to specify each and every host and only convert those specifically. In other > words, we don’t have the DEFAULT rule in auth_to_local. Ideally, the config > for us would be the following > {code:java} > RULE:[1:$1@$0](rm@<REALM>)s/.*/invalid_user/ > RULE:[2:$1/$2@$0](rm/rm1_host.fulldomain@<REALM>)s/.*/yarn/ > RULE:[2:$1/$2@$0](rm/rm2_host.fulldomain@<REALM>)s/.*/yarn/ > {code} > But if we use only that configuration, the servicecheck in Ambari failes with > the following exception. > {code:java} > org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit > application_1518422080198_0002 to YARN : Failed to renew token: Kind: > HDFS_DELEGATION_TOKEN, Service: ha-hdfs:devhadoop, Ident: > (HDFS_DELEGATION_TOKEN token 11096 for ambari-qa) > at > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:272) > at > org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:708) > at > org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:215) > {code} > > Inside the RM’s logfile, I can find the following. > {code:java} > Caused by: org.apache.hadoop.security.AccessControlException: yarn tries to > renew a token with renewer rm/_HOST@<REALM> > {code} > Adding the following rule to auth_to_local solves the problem > RULE:[2:$1/$2@$0](rm/_HOST@<REALM>)s/.*/yarn/ > The client used to test this is executed with the following command > yarn org.apache.hadoop.yarn.applications.distributedshell.Client > -shell_command ls -num_containers 1 -jar > /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar > -timeout 300000 --queue <YARN_QUEUE> -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org