> On aug. 24, 2017, 12:19 de, Peter Bacsko wrote: > > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > > Lines 1094 (patched) > > <https://reviews.apache.org/r/61734/diff/1/?file=1799993#file1799993line1094> > > > > I would rewrite this part a bit. > > > > If we're running on a secure cluster, then let's always create a > > Credentials object and call this method. By doing so we don't need this > > null check. > > András Piros wrote: > Moved the check to `checkCredentials()` and called from > `submitLauncher()` just before the `acquireHDFSDelegationToken()` call. We > cannot always create a `Credentials` because we need `credentialsProperties` > for that.
I've taken a deeper look into submitLauncher(). It's a bit more complicated than I thought. There is a property called "oozie.credentials.skip". If that's set to true, then delegation tokens won't be fetched and no credPropertiesMap will be created. But in that case, HDFS tokens cannot be added either. I assume that regardless of the "skip" property, we always have to add the HDFS token in a secure cluster to prevent lower-level issues but we can ask Robert about that. - Peter ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61734/#review183689 ----------------------------------------------------------- On aug. 24, 2017, 11:53 de, András Piros wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61734/ > ----------------------------------------------------------- > > (Updated aug. 24, 2017, 11:53 de) > > > Review request for oozie, Peter Cseh, Peter Bacsko, and Robert Kanter. > > > Repository: oozie-git > > > Description > ------- > > In a secure environment, when both HDFS HA and log aggregation are turned on, > `JavaActionExecutor` is not able to call `YarnClient#submitApplication` since > `HDFS_DELEGATION_TOKEN` is missing. > > In those cases we need to get `HDFS_DELEGATION_TOKEN` from YARN: > > * get server principal / YARN renewer via > `HadoopAccessorService#getServerPrincipal` > * get `HDFS_DELEGATION_TOKEN` via `DFSClient#getDelegationToken` > * add `HDFS_DELEGATION_TOKEN` to `Credentials` > > > Diffs > ----- > > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > dc17950418ada9b4848ed5597702a29e7124ab38 > core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java > b507c79d44aa2095caf532a5f4d19d20c1a30630 > > > Diff: https://reviews.apache.org/r/61734/diff/2/ > > > Testing > ------- > > `TestJavaActionExecutor`, `TestHadoopAccessorService` > > > Thanks, > > András Piros > >
