[jira] [Commented] (HADOOP-14350) Relative path for Kerberos keytab is not working on IBM JDK

2017-05-15 Thread Wen Yuan Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011660#comment-16011660
 ] 

Wen Yuan Chen commented on HADOOP-14350:


Actually we are running Spark job on yarn application. Spark will copy the 
keytab to its working directory and pass the relative path to the login method. 
We are not able to configure Spark to use absolute path.  

> Relative path for Kerberos keytab is not working on IBM JDK
> ---
>
> Key: HADOOP-14350
> URL: https://issues.apache.org/jira/browse/HADOOP-14350
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.7.3
> Environment: IBM JDK
>Reporter: Wen Yuan Chen
>
> For the sample code below:
> public class TestKrb {
>   public static void main(String[] args) throws IOException {
> String user = args[0], path = args[1];
> UserGroupInformation ugi = 
> UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
> System.out.println("Login successfully");
>   }
> }
> When I use IBM JDK and pass a relative path for the Kerberos keytab, it will 
> throw error messages.  According to the debug log, it always tries to read 
> the keytab from the root path.  See the debug logs below:
> 2017-04-19 02:29:13,982 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful 
> kerberos logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,990 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos 
> logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,991 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
> 2017-04-19 02:29:13,992 DEBUG 
> [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and 
> group related metrics
> [KRB_DBG_CFG] Config:main:   Java config file: 
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main:   Loaded from Java config
> 2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] -  Creating 
> new Groups object
> 2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Trying to load the custom-built native-hadoop library...
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: 
> hadoop (Not found in java.library.path)
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
> 2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] - 
> Unable to load native-hadoop library for your platform... using builtin-java 
> classes where applicable
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling 
> back to shell based
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group 
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited 
> with exit code 0
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group 
> mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; 
> cacheTimeout=30; warningDeltaMs=5000
> IBMJGSSProvider Build-Level: -20161128
> [JGSS_DBG_CRED]  main JAAS config: principal=job/analytics
> [JGSS_DBG_CRED]  main JAAS config: credsType=initiate and accept
> [JGSS_DBG_CRED]  main config: useDefaultCcache=false
> [JGSS_DBG_CRED]  main config: useCcache=null
> [JGSS_DBG_CRED]  main config: useDefaultKeytab=false
> [JGSS_DBG_CRED]  main config: useKeytab=//job.keytab
> [JGSS_DBG_CRED]  main JAAS config: forwardable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: renewable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: proxiable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: tryFirstPass=false

[jira] [Commented] (HADOOP-14350) Relative path for Kerberos keytab is not working on IBM JDK

2017-05-07 Thread Wen Yuan Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000305#comment-16000305
 ] 

Wen Yuan Chen commented on HADOOP-14350:


Can anyone help on this issue?

> Relative path for Kerberos keytab is not working on IBM JDK
> ---
>
> Key: HADOOP-14350
> URL: https://issues.apache.org/jira/browse/HADOOP-14350
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.7.3
>Reporter: Wen Yuan Chen
>
> For the sample code below:
> public class TestKrb {
>   public static void main(String[] args) throws IOException {
> String user = args[0], path = args[1];
> UserGroupInformation ugi = 
> UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
> System.out.println("Login successfully");
>   }
> }
> When I use IBM JDK and pass a relative path for the Kerberos keytab, it will 
> throw error messages.  According to the debug log, it always tries to read 
> the keytab from the root path.  See the debug logs below:
> 2017-04-19 02:29:13,982 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful 
> kerberos logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,990 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos 
> logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,991 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
> 2017-04-19 02:29:13,992 DEBUG 
> [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and 
> group related metrics
> [KRB_DBG_CFG] Config:main:   Java config file: 
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main:   Loaded from Java config
> 2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] -  Creating 
> new Groups object
> 2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Trying to load the custom-built native-hadoop library...
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: 
> hadoop (Not found in java.library.path)
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
> 2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] - 
> Unable to load native-hadoop library for your platform... using builtin-java 
> classes where applicable
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling 
> back to shell based
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group 
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited 
> with exit code 0
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group 
> mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; 
> cacheTimeout=30; warningDeltaMs=5000
> IBMJGSSProvider Build-Level: -20161128
> [JGSS_DBG_CRED]  main JAAS config: principal=job/analytics
> [JGSS_DBG_CRED]  main JAAS config: credsType=initiate and accept
> [JGSS_DBG_CRED]  main config: useDefaultCcache=false
> [JGSS_DBG_CRED]  main config: useCcache=null
> [JGSS_DBG_CRED]  main config: useDefaultKeytab=false
> [JGSS_DBG_CRED]  main config: useKeytab=//job.keytab
> [JGSS_DBG_CRED]  main JAAS config: forwardable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: renewable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: proxiable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: tryFirstPass=false (default)
> [JGSS_DBG_CRED]  main JAAS config: useFirstPass=false (default)
> [JGSS_DBG_CRED]  main JAAS config: moduleBanner=false (default)
> [JGSS_DBG_CRED]  main JAAS config: interactive login? no
>

[jira] [Commented] (HADOOP-14350) Relative path for Kerberos keytab is not working on IBM JDK

2017-04-25 Thread Wen Yuan Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984142#comment-15984142
 ] 

Wen Yuan Chen commented on HADOOP-14350:


I try Hadoop 2.8, but also get the same error messages.

> Relative path for Kerberos keytab is not working on IBM JDK
> ---
>
> Key: HADOOP-14350
> URL: https://issues.apache.org/jira/browse/HADOOP-14350
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.7.3
>Reporter: Wen Yuan Chen
>
> For the sample code below:
> public class TestKrb {
>   public static void main(String[] args) throws IOException {
> String user = args[0], path = args[1];
> UserGroupInformation ugi = 
> UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
> System.out.println("Login successfully");
>   }
> }
> When I use IBM JDK and pass a relative path for the Kerberos keytab, it will 
> throw error messages.  According to the debug log, it always tries to read 
> the keytab from the root path.  See the debug logs below:
> 2017-04-19 02:29:13,982 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful 
> kerberos logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,990 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos 
> logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,991 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
> 2017-04-19 02:29:13,992 DEBUG 
> [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and 
> group related metrics
> [KRB_DBG_CFG] Config:main:   Java config file: 
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main:   Loaded from Java config
> 2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] -  Creating 
> new Groups object
> 2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Trying to load the custom-built native-hadoop library...
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: 
> hadoop (Not found in java.library.path)
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
> 2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] - 
> Unable to load native-hadoop library for your platform... using builtin-java 
> classes where applicable
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling 
> back to shell based
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group 
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited 
> with exit code 0
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group 
> mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; 
> cacheTimeout=30; warningDeltaMs=5000
> IBMJGSSProvider Build-Level: -20161128
> [JGSS_DBG_CRED]  main JAAS config: principal=job/analytics
> [JGSS_DBG_CRED]  main JAAS config: credsType=initiate and accept
> [JGSS_DBG_CRED]  main config: useDefaultCcache=false
> [JGSS_DBG_CRED]  main config: useCcache=null
> [JGSS_DBG_CRED]  main config: useDefaultKeytab=false
> [JGSS_DBG_CRED]  main config: useKeytab=//job.keytab
> [JGSS_DBG_CRED]  main JAAS config: forwardable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: renewable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: proxiable=false (default)
> [JGSS_DBG_CRED]  main JAAS config: tryFirstPass=false (default)
> [JGSS_DBG_CRED]  main JAAS config: useFirstPass=false (default)
> [JGSS_DBG_CRED]  main JAAS config: moduleBanner=false (default)
> [JGSS_DBG_CRED]  main JAAS config: interactive

[jira] [Commented] (HADOOP-14350) Relative path for Kerberos keytab is not working on IBM JDK

2017-04-24 Thread Wen Yuan Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-14350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982317#comment-15982317
 ] 

Wen Yuan Chen commented on HADOOP-14350:


java version "1.8.0"
Java(TM) SE Runtime Environment (build pxa6480sr4fp1-20170215_01(SR4 FP1))
IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 
20170209_336038 (JIT enabled, AOT enabled)
J9VM - R28_20170209_0201_B336038
JIT  - tr.r14.java.green_20170125_131456
GC   - R28_20170209_0201_B336038_CMPRSS
J9CL - 20170209_336038)
JCL - 20170215_01 based on Oracle jdk8u121-b13

> Relative path for Kerberos keytab is not working on IBM JDK
> ---
>
> Key: HADOOP-14350
> URL: https://issues.apache.org/jira/browse/HADOOP-14350
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.7.3
>Reporter: Wen Yuan Chen
>Priority: Blocker
>
> For the sample code below:
> public class TestKrb {
>   public static void main(String[] args) throws IOException {
> String user = args[0], path = args[1];
> UserGroupInformation ugi = 
> UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
> System.out.println("Login successfully");
>   }
> }
> When I use IBM JDK and pass a relative path for the Kerberos keytab, it will 
> throw error messages.  According to the debug log, it always tries to read 
> the keytab from the root path.  See the debug logs below:
> 2017-04-19 02:29:13,982 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful 
> kerberos logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,990 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos 
> logins and latency (milliseconds)], valueName=Time)
> 2017-04-19 02:29:13,991 DEBUG 
> [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
> org.apache.hadoop.metrics2.lib.MutableRate 
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with 
> annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
> sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
> 2017-04-19 02:29:13,992 DEBUG 
> [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and 
> group related metrics
> [KRB_DBG_CFG] Config:main:   Java config file: 
> /opt/ibm/java/jre/lib/security/krb5.conf
> [KRB_DBG_CFG] Config:main:   Loaded from Java config
> 2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] -  Creating 
> new Groups object
> 2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Trying to load the custom-built native-hadoop library...
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: 
> hadoop (Not found in java.library.path)
> 2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
> java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
> 2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] - 
> Unable to load native-hadoop library for your platform... using builtin-java 
> classes where applicable
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling 
> back to shell based
> 2017-04-19 02:29:14,180 DEBUG 
> [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group 
> mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited 
> with exit code 0
> 2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group 
> mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; 
> cacheTimeout=30; warningDeltaMs=5000
> IBMJGSSProvider Build-Level: -20161128
> [JGSS_DBG_CRED]  main JAAS config: principal=job/analytics
> [JGSS_DBG_CRED]  main JAAS config: credsType=initiate and accept
> [JGSS_DBG_CRED]  main config: useDefaultCcache=false
> [JGSS_DBG_CRED]  main config: useCcache=null
> [JGSS_DBG_CRED]  main config: useDefaultKeytab=false
> [JGSS_DBG_CRED]  main config: useKeytab=//job.keytab
> [JGSS_DBG_CRED]  main JAAS config: forwardable=false (default)
>

[jira] [Created] (HADOOP-14350) Relative path for Kerberos keytab is not working on IBM JDK

2017-04-24 Thread Wen Yuan Chen (JIRA) 
Wen Yuan Chen created HADOOP-14350:
--

 Summary: Relative path for Kerberos keytab is not working on IBM 
JDK
 Key: HADOOP-14350
 URL: https://issues.apache.org/jira/browse/HADOOP-14350
 Project: Hadoop Common
  Issue Type: Bug
  Components: common, security
Affects Versions: 2.7.3
Reporter: Wen Yuan Chen
Priority: Blocker


For the sample code below:

public class TestKrb {
  public static void main(String[] args) throws IOException {
String user = args[0], path = args[1];
UserGroupInformation ugi = 
UserGroupInformation.loginUserFromKeytabAndReturnUGI(user, path);
System.out.println("Login successfully");
  }
}

When I use IBM JDK and pass a relative path for the Kerberos keytab, it will 
throw error messages.  According to the debug log, it always tries to read the 
keytab from the root path.  See the debug logs below:


2017-04-19 02:29:13,982 DEBUG 
[org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
org.apache.hadoop.metrics2.lib.MutableRate 
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with 
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful kerberos 
logins and latency (milliseconds)], valueName=Time)
2017-04-19 02:29:13,990 DEBUG 
[org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
org.apache.hadoop.metrics2.lib.MutableRate 
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with 
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos 
logins and latency (milliseconds)], valueName=Time)
2017-04-19 02:29:13,991 DEBUG 
[org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field 
org.apache.hadoop.metrics2.lib.MutableRate 
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with 
annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, 
sampleName=Ops, always=false, type=DEFAULT, value=[GetGroups], valueName=Time)
2017-04-19 02:29:13,992 DEBUG 
[org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and 
group related metrics
[KRB_DBG_CFG] Config:main:   Java config file: 
/opt/ibm/java/jre/lib/security/krb5.conf
[KRB_DBG_CFG] Config:main:   Loaded from Java config
2017-04-19 02:29:14,175 DEBUG [org.apache.hadoop.security.Groups] -  Creating 
new Groups object
2017-04-19 02:29:14,178 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
Trying to load the custom-built native-hadoop library...
2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: hadoop 
(Not found in java.library.path)
2017-04-19 02:29:14,179 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - 
java.library.path=/opt/ibm/java/jre/lib/amd64/compressedrefs:/opt/ibm/java/jre/lib/amd64:/usr/lib64:/usr/lib
2017-04-19 02:29:14,179 WARN [org.apache.hadoop.util.NativeCodeLoader] - Unable 
to load native-hadoop library for your platform... using builtin-java classes 
where applicable
2017-04-19 02:29:14,180 DEBUG 
[org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling 
back to shell based
2017-04-19 02:29:14,180 DEBUG 
[org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group 
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.util.Shell] - setsid exited 
with exit code 0
2017-04-19 02:29:14,334 DEBUG [org.apache.hadoop.security.Groups] - Group 
mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; 
cacheTimeout=30; warningDeltaMs=5000
IBMJGSSProvider Build-Level: -20161128
[JGSS_DBG_CRED]  main JAAS config: principal=job/analytics
[JGSS_DBG_CRED]  main JAAS config: credsType=initiate and accept
[JGSS_DBG_CRED]  main config: useDefaultCcache=false
[JGSS_DBG_CRED]  main config: useCcache=null
[JGSS_DBG_CRED]  main config: useDefaultKeytab=false
[JGSS_DBG_CRED]  main config: useKeytab=//job.keytab
[JGSS_DBG_CRED]  main JAAS config: forwardable=false (default)
[JGSS_DBG_CRED]  main JAAS config: renewable=false (default)
[JGSS_DBG_CRED]  main JAAS config: proxiable=false (default)
[JGSS_DBG_CRED]  main JAAS config: tryFirstPass=false (default)
[JGSS_DBG_CRED]  main JAAS config: useFirstPass=false (default)
[JGSS_DBG_CRED]  main JAAS config: moduleBanner=false (default)
[JGSS_DBG_CRED]  main JAAS config: interactive login? no
[JGSS_DBG_CRED]  main JAAS config: refreshKrb5Config = true
[KRB_DBG_CFG] Config:main:   Java config file: 
/opt/ibm/java/jre/lib/security/krb5.conf
[KRB_DBG_CFG] Config:main:   Loaded from Java config
[KRB_DBG_KDC] KdcComm:main:   >>> KdcAccessibility: reset
[KRB_DBG_KDC] KdcComm:main:   >>> KdcAccessibility: reset
[JGSS_DBG_CRED]  main Try keytab for principal=job/analytics

[jira] [Commented] (HADOOP-9969) TGT expiration doesn't trigger Kerberos relogin

2017-02-16 Thread Wen Yuan Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-9969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871297#comment-15871297
 ] 

Wen Yuan Chen commented on HADOOP-9969:
---

Any update on this issue? I meet the same issue on Hadoop 2.7.3 with IBM JDK 1.8

> TGT expiration doesn't trigger Kerberos relogin
> ---
>
> Key: HADOOP-9969
> URL: https://issues.apache.org/jira/browse/HADOOP-9969
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: ipc, security
>Affects Versions: 2.1.0-beta, 2.5.0, 2.5.2, 2.6.0, 2.6.1, 2.8.0, 2.7.1, 
> 2.6.2, 2.6.3
> Environment: IBM JDK7
>Reporter: Yu Gao
> Attachments: HADOOP-9969.patch, JobTracker.log
>
>
> In HADOOP-9698 & HADOOP-9850, RPC client and Sasl client have been changed to 
> respect the auth method advertised from server, instead of blindly attempting 
> the configured one at client side. However, when TGT has expired, an 
> exception will be thrown from SaslRpcClient#createSaslClient(SaslAuth 
> authType), and at this time the authMethod still holds the initial value 
> which is SIMPLE and never has a chance to be updated with the expected one 
> requested by server, so kerberos relogin will not happen.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org