[ https://issues.apache.org/jira/browse/YARN-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Fritsi updated YARN-10601: --------------------------------- Affects Version/s: 3.4.0 > The Yarn client should use the UGI who created the Yarn client for obtaining > a delegation token for the remote log dir > ---------------------------------------------------------------------------------------------------------------------- > > Key: YARN-10601 > URL: https://issues.apache.org/jira/browse/YARN-10601 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation > Affects Versions: 3.3.0, 3.4.0 > Reporter: Daniel Fritsi > Priority: Critical > > It seems there was a bug introduced in YARN-10333 in this section of > *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*: > {code:java} > Path remoteRootLogDir = fileController.getRemoteRootLogDir(); > FileSystem fs = remoteRootLogDir.getFileSystem(conf); > final org.apache.hadoop.security.token.Token<?>[] finalTokens = > fs.addDelegationTokens(masterPrincipal, credentials); > {code} > *{color:#0747A6}{{remoteRootLogDir.getFileSystem}}{color}* simply does this: > {code:java} > public FileSystem getFileSystem(Configuration conf) throws IOException { > return FileSystem.get(this.toUri(), conf); > } > {code} > As far as I know it's customary to create a YarnClient instance via > *{color:#0747A6}{{YarnClient.createYarnClient()}}{color}* in a > UserGroupInformation.doAs block if you would like to use it with a different > user then the current one. E.g.: > {code:java} > YarnClient yarnClient = ugi.doAs(new PrivilegedExceptionAction<YarnClient>() { > @Override > public YarnClient run() throws Exception { > YarnClient yarnClient = YarnClient.createYarnClient(); > yarnClient.init(conf); > yarnClient.start(); > return yarnClient; > } > }); > {code} > If this statement is correct then I think YarnClient should save the > *{color:#0747A6}{{UserGroupInformation.getCurrentUser()}}{color}* when the > YarnClient is being created and the > *{color:#0747A6}{{remoteRootLogDir.getFileSystem(conf)}}{color}* call should > be made inside an ugi.doAs block with that saved user. > A more concrete example: > {code:java} > public YarnClient createYarnClient(UserGroupInformation ugi, Configuration > conf) throws Exception { > return ugi.doAs((PrivilegedExceptionAction<YarnClient>) () -> { > // Her I am the submitterUser (see below) > YarnClient yarnClient = YarnClient.createYarnClient(); > yarnClient.init(conf); > yarnClient.start(); > return yarnClient; > }); > } > public void run() { > // Here I am the serviceUser > // ... > Configuration conf = ... > // ... > UserGroupInformation ugi = getSubmitterUser(); > // ... > YarnClient yarnClient = createYarnClient(ugi); > // ... > ApplicationSubmissionContext context = ... > // ... > yarnClient.submitApplication(context); > } > {code} > As you can see *{color:#0747A6}{{submitApplication}}{color}* is not invoked > inside an ugi.doAs block and submitApplication is the one who will eventually > invoke *{color:#0747A6}{{addLogAggregationDelegationToken}}{color}*. That's > why we need to save the UGI during the YarnClient creation and create the > FileSystem instance inside an ugi.doAs with that saved user. Otherwise Yarn > will try to get a delegation token with an incorrect user (serviceUser) > instead of the submitterUser. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org