[
https://issues.apache.org/jira/browse/YARN-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136795#comment-17136795
]
Eric Yang commented on YARN-10310:
----------------------------------
Trunk code without patch 001 produces:
Launching application using hdfs/[email protected] principal:
{code}
$ kinit hdfs/[email protected]
Password for hdfs/[email protected]:
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/yarn app -launch rr sleeper
2020-06-16 09:08:28,553 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:08:29,325 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:08:29,329 INFO client.ApiServiceClient: Loading service
definition from local FS:
/usr/local/hadoop-3.4.0-SNAPSHOT/share/hadoop/yarn/yarn-service-examples/sleeper/sleeper.json
2020-06-16 09:08:45,835 INFO client.ApiServiceClient: Application ID:
application_1592323643465_0001
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/hdfs dfs -ls .yarn/services/rr
Found 3 items
drwxr-x--- - hdfs supergroup 0 2020-06-16 09:08
.yarn/services/rr/conf
drwxr-xr-x - hdfs supergroup 0 2020-06-16 09:08 .yarn/services/rr/lib
-rw-rw-rw- 1 hdfs supergroup 831 2020-06-16 09:08
.yarn/services/rr/rr.json
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/hdfs dfs -rmr .yarn/services/rr
rmr: DEPRECATED: Please use '-rm -r' instead.
Deleted .yarn/services/rr
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/yarn app -launch rr sleeper
2020-06-16 09:10:18,754 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:10:19,206 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:10:19,209 INFO client.ApiServiceClient: Loading service
definition from local FS:
/usr/local/hadoop-3.4.0-SNAPSHOT/share/hadoop/yarn/yarn-service-examples/sleeper/sleeper.json
2020-06-16 09:10:21,421 ERROR client.ApiServiceClient: Service name rr is
already taken.
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/hdfs dfs -ls .yarn/services
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ klist
Ticket cache: FILE:/tmp/krb5cc_123
Default principal: hdfs/[email protected]
Valid starting Expires Service principal
06/16/2020 09:08:15 06/17/2020 09:08:15 krbtgt/[email protected]
{code}
Launching application using hdfs principal while service file is already
deleted from hdfs:
{code}
$ kinit
Password for [email protected]:
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/yarn app -launch rr sleeper
2020-06-16 09:20:05,737 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:20:06,405 INFO client.DefaultNoHARMFailoverProxyProvider:
Connecting to ResourceManager at kerberos.example.com/192.168.1.9:8032
2020-06-16 09:20:06,409 INFO client.ApiServiceClient: Loading service
definition from local FS:
/usr/local/hadoop-3.4.0-SNAPSHOT/share/hadoop/yarn/yarn-service-examples/sleeper/sleeper.json
2020-06-16 09:20:10,082 ERROR client.ApiServiceClient: Service name rr is
already taken.
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$ ./bin/hdfs dfs -ls .yarn/services
[hdfs@kerberos hadoop-3.4.0-SNAPSHOT]$
{code}
If the application is running, verifyNoLiveAppInRM does throw exception. I can
not reproduce the claimed issue. I suspect that verifyNoLiveAppInRM did not
throw exception due to cluster configuration issues.
We should not use getShortUserName() api on the client side. The client must
pass the full principal name to server, and only server resolves the short name
when necessary.
Please check in core-site.xml, the following properties have been configured:
{code}
<property>
<name>hadoop.http.authentication.type</name>
<value>kerberos</value>
</property>
<property>
<name>hadoop.http.filter.initializers</name>
<value>org.apache.hadoop.security.AuthenticationFilterInitializer</value>
</property>
{code}
If they are not configured correctly, you may be accessing ServiceClient
insecurely which result in the errors that you were seeing.
> YARN Service - User is able to launch a service with same name
> --------------------------------------------------------------
>
> Key: YARN-10310
> URL: https://issues.apache.org/jira/browse/YARN-10310
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bilwa S T
> Assignee: Bilwa S T
> Priority: Major
> Attachments: YARN-10310.001.patch
>
>
> As ServiceClient uses UserGroupInformation.getCurrentUser().getUserName() to
> get user whereas ClientRMService#submitApplication uses
> UserGroupInformation.getCurrentUser().getShortUserName() to set application
> username.
> In case of user with name hdfs/[email protected]. below condition fails
> ClientRMService#getApplications()
> {code:java}
> if (users != null && !users.isEmpty() &&
> !users.contains(application.getUser())) {
> continue;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
