cnauroth opened a new pull request, #7325: URL: https://github.com/apache/hadoop/pull/7325
### Description of PR Under simple auth, UGI identifies the current Hadoop user as the OS user. If the OS can't determine the user, then they receive a cryptic `KerberosAuthException`, even though Kerberos auth isn't really configured. I've seen this come up for Hadoop processes launched in Docker containers that accidentally specify a UID that's not actually present in the image. We can improve this error so that users have a better understanding of how to fix the problem. Here is how the Unix user identification flows through the JDK: https://github.com/openjdk/jdk11u-dev/blob/master/src/jdk.security.auth/share/classes/com/sun/security/auth/UnixPrincipal.java#L65 https://github.com/openjdk/jdk11u-dev/blob/master/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixLoginModule.java#L129 https://github.com/openjdk/jdk11u-dev/blob/master/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixSystem.java#L55 Ultimately, this is just delegating to native syscalls to `getuid` and `getpwuid`: https://github.com/openjdk/jdk11u-dev/blob/master/src/jdk.security.auth/unix/native/libjaas/Unix.c#L82 ### How was this patch tested? One easy way to reproduce this problem is to exec inside the dev container with a non-existent UID: ``` docker exec -it -u 123 690a92a9a5dc /bin/bash _ _ _ ______ | | | | | | | _ \ | |_| | __ _ __| | ___ ___ _ __ | | | |_____ __ | _ |/ _` |/ _` |/ _ \ / _ \| '_ \ | | | / _ \ \ / / | | | | (_| | (_| | (_) | (_) | |_) | | |/ / __/\ V / \_| |_/\__,_|\__,_|\___/ \___/| .__/ |___/ \___| \_(_) | | |_| This is the standard Hadoop Developer build environment. This has all the right tools installed required to build Hadoop from source. I have no name!@690a92a9a5dc:~/hadoop$ ``` Then, run UGI's main entry point: ``` I have no name!@690a92a9a5dc:/tmp/123/hadoop-3.4.1$ java -cp "$(bin/hadoop classpath)" org.apache.hadoop.security.UserGroupInformation Getting UGI for current user Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71) at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677) at java.base/java.security.AccessController.doPrivileged(AccessController.java:712) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2148) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2053) at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:733) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:683) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:590) at org.apache.hadoop.security.UserGroupInformation.main(UserGroupInformation.java:2302) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2064) at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:733) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:683) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:590) at org.apache.hadoop.security.UserGroupInformation.main(UserGroupInformation.java:2302) Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71) at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677) at java.base/java.security.AccessController.doPrivileged(AccessController.java:712) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2148) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2053) at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:733) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:683) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:590) at org.apache.hadoop.security.UserGroupInformation.main(UserGroupInformation.java:2302) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:850) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677) at java.base/java.security.AccessController.doPrivileged(AccessController.java:712) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2148) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2053) ... 4 more ``` This is problematic for two reasons: 1. The `NullPointerException` from the JDK doesn't point the user toward how to fix the problem. 1. The presence of `KerberosAuthException` actually misleads people into thinking it's a Kerberos problem, even though this is simple auth. After the patch, the error indicates a problem with the UID and avoids mentioning Kerberos. This will point users in the right direction of looking at problems with their OS user. ``` Exception in thread "main" java.io.IOException: Invalid UID, could not determine effective user at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2087) at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:734) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:684) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:591) at org.apache.hadoop.security.UserGroupInformation.main(UserGroupInformation.java:2326) Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71) at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677) at java.base/java.security.AccessController.doPrivileged(AccessController.java:712) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2172) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2067) at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:734) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:684) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:591) at org.apache.hadoop.security.UserGroupInformation.main(UserGroupInformation.java:2326) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:850) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677) at java.base/java.security.AccessController.doPrivileged(AccessController.java:712) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2172) at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2067) ... 4 more ``` ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
