> On Aug. 24, 2017, 12:19 a.m., Peter Bacsko wrote: > > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > > Lines 1094 (patched) > > <https://reviews.apache.org/r/61734/diff/1/?file=1799993#file1799993line1094> > > > > I would rewrite this part a bit. > > > > If we're running on a secure cluster, then let's always create a > > Credentials object and call this method. By doing so we don't need this > > null check. > > András Piros wrote: > Moved the check to `checkCredentials()` and called from > `submitLauncher()` just before the `acquireHDFSDelegationToken()` call. We > cannot always create a `Credentials` because we need `credentialsProperties` > for that. > > Peter Bacsko wrote: > I've taken a deeper look into submitLauncher(). It's a bit more > complicated than I thought. > > There is a property called "oozie.credentials.skip". If that's set to > true, then delegation tokens won't be fetched and no credPropertiesMap will > be created. But in that case, HDFS tokens cannot be added either. > > I assume that regardless of the "skip" property, we always have to add > the HDFS token in a secure cluster to prevent lower-level issues but we can > ask Robert about that.
Based on [Robert Kanter's comment](https://issues.apache.org/jira/browse/OOZIE-3035?focusedCommentId=16144101&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16144101) implementing `JavaActionExecutor#setCredentialPropertyToActionConf` in a way we also check for `UserGroupInformation#isSecurityEnabled` where we return `null` when `oozie.credentials.skip` is set in either `actionConf` or `wfJobConf`. Taking also the liberty to refactor that method for easier readability. - András ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61734/#review183689 ----------------------------------------------------------- On Aug. 29, 2017, 8:44 a.m., András Piros wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61734/ > ----------------------------------------------------------- > > (Updated Aug. 29, 2017, 8:44 a.m.) > > > Review request for oozie, Peter Cseh, Peter Bacsko, and Robert Kanter. > > > Repository: oozie-git > > > Description > ------- > > In a secure environment, when both HDFS HA and log aggregation are turned on, > `JavaActionExecutor` is not able to call `YarnClient#submitApplication` since > `HDFS_DELEGATION_TOKEN` is missing. > > In those cases we need to get `HDFS_DELEGATION_TOKEN` from YARN: > > * get server principal / YARN renewer via > `HadoopAccessorService#getServerPrincipal` > * get `HDFS_DELEGATION_TOKEN` via `DFSClient#getDelegationToken` > * add `HDFS_DELEGATION_TOKEN` to `Credentials` > > > Diffs > ----- > > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > dc17950418ada9b4848ed5597702a29e7124ab38 > core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java > b507c79d44aa2095caf532a5f4d19d20c1a30630 > > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java > c51c64a18c74c73aa9907027992033a9975bf4d7 > > > Diff: https://reviews.apache.org/r/61734/diff/3/ > > > Testing > ------- > > `TestJavaActionExecutor`, `TestHadoopAccessorService` > > > Thanks, > > András Piros > >
