Daniel Fritsi created YARN-10601:
------------------------------------
Summary: The Yarn client should use the UGI who created the Yarn
client for obtaining a delegation token for the remote log dir
Key: YARN-10601
URL: https://issues.apache.org/jira/browse/YARN-10601
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation
Affects Versions: 3.3.0
Reporter: Daniel Fritsi
It seems there was a bug introduced in YARN-10333 in this section of
*{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*:
{code:java}
Path remoteRootLogDir = fileController.getRemoteRootLogDir();
FileSystem fs = remoteRootLogDir.getFileSystem(conf);
final org.apache.hadoop.security.token.Token<?>[] finalTokens =
fs.addDelegationTokens(masterPrincipal, credentials);
{code}
*{color:#0747A6}{{remoteRootLogDir.getFileSystem}}{color}* simply does this:
{code:java}
public FileSystem getFileSystem(Configuration conf) throws IOException {
return FileSystem.get(this.toUri(), conf);
}
{code}
As far as I know it's customary to create a YarnClient instance via
*{color:#0747A6}{{YarnClient.createYarnClient()}}{color}* in a
UserGroupInformation.doAs block if you would like to use it with a different
user then the current one. E.g.:
{code:java}
YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() {
@Override
public YarnClient run() throws Exception {
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start();
return yarnClient;
}
});
{code}
If this statement is correct then I think YarnClient should save the
*{color:#0747A6}{{UserGroupInformation.getCurrentUser()}}{color}* when the
YarnClient is being created and the
*{color:#0747A6}{{remoteRootLogDir.getFileSystem(conf)}}{color}* call should be
made inside an ugi.doAs block with that saved user.
A more concrete example:
{code:java}
public YarnClient createYarnClient(UserGroupInformation ugi, Configuration
conf) throws Exception {
return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> {
// Her I am the submitterUser (see below)
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start();
return yarnClient;
});
}
public void run() {
// Here I am the serviceUser
// ...
Configuration conf = ...
// ...
UserGroupInformation ugi = getSubmitterUser();
// ...
YarnClient yarnClient = createYarnClient(ugi);
// ...
ApplicationSubmissionContext context = ...
// ...
yarnClient.submitApplication(context);
}
{code}
As you can see *{color:#0747A6}{{submitApplication}}{color}* is not invoked
inside an ugi.doAs block and submitApplication is the one who will eventually
invoke *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*. That's why
we need to save the UGI during the YarnClient creation and create the
FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn
will try to get a delegation token with an incorrect user (serviceUser) instead
of the submitterUser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
