[
https://issues.apache.org/jira/browse/YARN-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278327#comment-17278327
]
Gézapeti edited comment on YARN-10601 at 2/3/21, 7:57 PM:
----------------------------------------------------------
I think we haven't added the doAs block as it looked like the job submission
used the tokens we've added into the job config and everything we've checked
showed that the application is run as the user we've intended. We have missed
similar issues due to our test setup unfortunately: OOZIE-3478
was (Author: gezapeti):
I think we haven't added the doAs block as it looked like the job submission
used the token we've added into the job config and everything we've checked
showed that the application is run as the user we've intended. We have missed
similar issues due to our test setup unfortunately: OOZIE-3478
> The Yarn client should use the UGI who created the Yarn client for obtaining
> a delegation token for the remote log dir
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-10601
> URL: https://issues.apache.org/jira/browse/YARN-10601
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Affects Versions: 3.3.0, 3.4.0
> Reporter: Daniel Fritsi
> Priority: Critical
>
> It seems there was a bug introduced in YARN-10333 in this section of
> *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*:
> {code:java}
> Path remoteRootLogDir = fileController.getRemoteRootLogDir();
> FileSystem fs = remoteRootLogDir.getFileSystem(conf);
> final org.apache.hadoop.security.token.Token<?>[] finalTokens =
> fs.addDelegationTokens(masterPrincipal, credentials);
> {code}
> *{color:#0747A6}{{remoteRootLogDir.getFileSystem}}{color}* simply does this:
> {code:java}
> public FileSystem getFileSystem(Configuration conf) throws IOException {
> return FileSystem.get(this.toUri(), conf);
> }
> {code}
> As far as I know it's customary to create a YarnClient instance via
> *{color:#0747A6}{{YarnClient.createYarnClient()}}{color}* in a
> UserGroupInformation.doAs block if you would like to use it with a different
> user then the current one. E.g.:
> {code:java}
> YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() {
> @Override
> public YarnClient run() throws Exception {
> YarnClient yarnClient = YarnClient.createYarnClient();
> yarnClient.init(conf);
> yarnClient.start();
> return yarnClient;
> }
> });
> {code}
> If this statement is correct then I think YarnClient should save the
> *{color:#0747A6}{{UserGroupInformation.getCurrentUser()}}{color}* when the
> YarnClient is being created and the
> *{color:#0747A6}{{remoteRootLogDir.getFileSystem(conf)}}{color}* call should
> be made inside an ugi.doAs block with that saved user.
> A more concrete example:
> {code:java}
> public YarnClient createYarnClient(UserGroupInformation ugi, Configuration
> conf) throws Exception {
> return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> {
> // Her I am the submitterUser (see below)
> YarnClient yarnClient = YarnClient.createYarnClient();
> yarnClient.init(conf);
> yarnClient.start();
> return yarnClient;
> });
> }
> public void run() {
> // Here I am the serviceUser
> // ...
> Configuration conf = ...
> // ...
> UserGroupInformation ugi = getSubmitterUser();
> // ...
> YarnClient yarnClient = createYarnClient(ugi);
> // ...
> ApplicationSubmissionContext context = ...
> // ...
> yarnClient.submitApplication(context);
> }
> {code}
> As you can see *{color:#0747A6}{{submitApplication}}{color}* is not invoked
> inside an ugi.doAs block and submitApplication is the one who will eventually
> invoke *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*. That's
> why we need to save the UGI during the YarnClient creation and create the
> FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn
> will try to get a delegation token with an incorrect user (serviceUser)
> instead of the submitterUser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
