Daryn Sharp commented on YARN-7922:

This shouldn't be able to happen.  Distributed shell gets the renewer from 
{{YarnClientUtils.getRmPrincipal}} which calls 
{{SecurityUtil.getServerPrincipal}} to substitute _HOST.  Yet somehow the 
substitution did not occur.

The most conceivable, yet unlikely, way I see this failing is the principal has 
more than 3 components, ie. <REALM> contains another / or @, which would cause 
the substitution to short-out.

> Yarn dont resolve rm/_HOST to hostname
> --------------------------------------
>                 Key: YARN-7922
>                 URL: https://issues.apache.org/jira/browse/YARN-7922
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 2.7.3
>            Reporter: Berry Österlund
>            Priority: Minor
> The normal auth_to_local usually removes everything after the / in the 
> username of the Kerberos principle. That, together with the _HOST setting in 
> the configuration files specifying the Kerberos principles is usually what is 
> required to convert rm/_HOST@<REALM> to user yarn.
> In our environment, we cant use the default rules in auth_to_local. We have 
> to specify each and every host and only convert those specifically. In other 
> words, we don’t have the DEFAULT rule in auth_to_local. Ideally, the config 
> for us would be the following
> {code:java}
> RULE:[1:$1@$0](rm@<REALM>)s/.*/invalid_user/
> RULE:[2:$1/$2@$0](rm/rm1_host.fulldomain@<REALM>)s/.*/yarn/
> RULE:[2:$1/$2@$0](rm/rm2_host.fulldomain@<REALM>)s/.*/yarn/
> {code}
> But if we use only that configuration, the servicecheck in Ambari failes with 
> the following exception.
> {code:java}
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit 
> application_1518422080198_0002 to YARN : Failed to renew token: Kind: 
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:devhadoop, Ident: 
> (HDFS_DELEGATION_TOKEN token 11096 for ambari-qa)
> at 
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:272)
> at 
> org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:708)
> at 
> org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:215)
> {code}
> Inside the RM’s logfile, I can find the following.
> {code:java}
> Caused by: org.apache.hadoop.security.AccessControlException: yarn tries to 
> renew a token with renewer rm/_HOST@<REALM>
> {code}
> Adding the following rule to auth_to_local solves the problem
>  RULE:[2:$1/$2@$0](rm/_HOST@<REALM>)s/.*/yarn/
> The client used to test this is executed with the following command
>  yarn org.apache.hadoop.yarn.applications.distributedshell.Client 
> -shell_command ls -num_containers 1 -jar 
> /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar
>  -timeout 300000 --queue <YARN_QUEUE>

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to