[
https://issues.apache.org/jira/browse/HADOOP-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263363#comment-15263363
]
zhaojianbo commented on HADOOP-13069:
-------------------------------------
In our case, we want to proxy different users to use different resources(eg.
different resource pools in YARN, and different directories in HDFS with
different permission) in one process context. So we extend some methods int
class UserGroupInformation to make it accept a parameter
"dfs.client.config.proxyuser" in conf which specify the user name needed. The
code added like this:
{code:java}
Index:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
===================================================================
---
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
(revision 2303)
+++
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
(working copy)
@@ -18,6 +18,7 @@
package org.apache.hadoop.security;
import static
org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_USER_GROUP_METRICS_PERCENTILES_INTERVALS;
+import static
org.apache.hadoop.fs.CommonConfigurationKeys.DFS_CLIENT_CONFIG_PROXYUSER;
import static org.apache.hadoop.util.PlatformName.IBM_JAVA;
import java.io.File;
@@ -342,6 +343,7 @@
* Information about the logged in user.
*/
private static UserGroupInformation loginUser = null;
+ private static HashMap<String, UserGroupInformation> map = new
HashMap<String,UserGroupInformation>();
private static String keytabPrincipal = null;
private static String keytabFile = null;
@@ -650,6 +652,17 @@
}
}
+ public synchronized
+ static UserGroupInformation getCurrentUser(Configuration conf) throws
IOException {
+ AccessControlContext context = AccessController.getContext();
+ Subject subject = Subject.getSubject(context);
+ if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
+ return getLoginUser(conf);
+ } else {
+ return new UserGroupInformation(subject);
+ }
+ }
+
/**
* Find the most appropriate UserGroupInformation to use
*
@@ -776,6 +789,20 @@
return loginUser;
}
+ public synchronized
+ static UserGroupInformation getLoginUser(Configuration conf) throws
IOException {
+ String key = "";
+ synchronized (map){
+ key = conf.get(DFS_CLIENT_CONFIG_PROXYUSER,"");
+ if(map.containsKey(key)){
+ return map.get(key);
+ }
+ }
+ loginUserFromSubject(null,conf);
+ return map.get(key);
+ }
+
+
/**
* remove the login method that is followed by a space from the username
* e.g. "jack (auth:SIMPLE)" -> "jack"
@@ -841,6 +868,55 @@
}
}
+ public synchronized
+ static void loginUserFromSubject(Subject subject, Configuration conf) throws
IOException {
+ ensureInitialized();
+ UserGroupInformation tmpuser = null;
+ try {
+ if (subject == null) {
+ subject = new Subject();
+ }
+ LoginContext login =
+ newLoginContext(authenticationMethod.getLoginAppName(),
+ subject, new HadoopConfiguration());
+ login.login();
+ UserGroupInformation realUser = new UserGroupInformation(subject);
+ realUser.setLogin(login);
+ realUser.setAuthenticationMethod(authenticationMethod);
+ realUser = new UserGroupInformation(login.getSubject());
+ // If the HADOOP_PROXY_USER environment variable or property
+ // is specified, create a proxy user as the logged in user.
+ String proxyUser = conf.get(DFS_CLIENT_CONFIG_PROXYUSER, "");
+ if (proxyUser==null || proxyUser.isEmpty()) {
+ proxyUser = System.getenv(HADOOP_PROXY_USER);
+ if (proxyUser == null) {
+ proxyUser = System.getProperty(HADOOP_PROXY_USER);
+ }
+ }
+ tmpuser = proxyUser == null ? realUser : createProxyUser(proxyUser,
realUser);
+
+ String fileLocation = System.getenv(HADOOP_TOKEN_FILE_LOCATION);
+ if (fileLocation != null) {
+ // Load the token storage file and put all of the tokens into the
+ // user. Don't use the FileSystem API for reading since it has a lock
+ // cycle (HADOOP-9212).
+ Credentials cred = Credentials.readTokenStorageFile(
+ new File(fileLocation), conf);
+ tmpuser.addCredentials(cred);
+ }
+ tmpuser.spawnAutoRenewalThreadForUserCreds();
+ synchronized (map){
+ map.put(conf.get(DFS_CLIENT_CONFIG_PROXYUSER, ""), tmpuser);
+ }
+ } catch (LoginException le) {
+ LOG.debug("failure to login", le);
+ throw new IOException("failure to login", le);
+ }
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("UGI loginUser:" + tmpuser);
+ }
+ }
+
@InterfaceAudience.Private
@InterfaceStability.Unstable
@VisibleForTesting
Index:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
===================================================================
---
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
(revision 2303)
+++
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
(working copy)
@@ -281,4 +281,5 @@
public static final String NFS_EXPORTS_ALLOWED_HOSTS_SEPARATOR = ";";
public static final String NFS_EXPORTS_ALLOWED_HOSTS_KEY =
"nfs.exports.allowed.hosts";
public static final String NFS_EXPORTS_ALLOWED_HOSTS_KEY_DEFAULT = "* rw";
+ public static final String DFS_CLIENT_CONFIG_PROXYUSER =
"dfs.client.config.proxyuser";
}
{code}
And changing method getCurrentUser called into method getCurrentUser(conf) when
needing the feature.
> Making hadoop client proxy multi user in a process context
> ----------------------------------------------------------
>
> Key: HADOOP-13069
> URL: https://issues.apache.org/jira/browse/HADOOP-13069
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.6.0
> Reporter: zhaojianbo
>
> Now hadoop client can proxy an other user if setting HADOOP_PROXY_USER in the
> environment parameters. But the mechanism can proxy only one user in a
> process context. The code is like:
> {code:xml}
> public synchronized
> static UserGroupInformation getLoginUser() throws IOException {
> if (loginUser == null) {
> loginUserFromSubject(null);
> }
> return loginUser;
> }
> {code}
> The static variable loginUser cache the last login user.
> But in some cases, we want to proxy multi users in one process context. It
> seems to be that the current code is not support that.Is it?
> Correct me if I'm wrong.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
