[jira] [Updated] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunny Cheung updated HADOOP-11683: -- Assignee: roger mak (was: Sunny Cheung) Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681046#comment-14681046 ] Sunny Cheung commented on HADOOP-11683: --- Just reassigned this bug to [~roger.mak]. He is my colleague who implements this feature. Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360037#comment-14360037 ] Sunny Cheung commented on HADOOP-11683: --- bq. Be aware that HadoopKerberosName is now exposed to users in trunk. We should make sure that the solution here also works there. Yes, we are aware of this too. Just to confirm, since KerberosName and HadoopKerberosName are intended for HDFS and MapReduce projects only (as defined in LimitedPrivate), do we have the option to refactor these classes (and maybe provide an interface similar to GroupMappingServiceProvider)? Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358172#comment-14358172 ] Sunny Cheung commented on HADOOP-11683: --- bq. Would you contribute and do it yourself ? If so I can assign this to you. Yes, Centrify is absolutely willing to do this for the Hadoop project and donate code. Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352754#comment-14352754
]
Sunny Cheung commented on HADOOP-11683:
---
{quote}
I am assuming you are talking about :
john@example.com - user123
foo.sm...@example.com - user789
...
possibly some 200k such entries
{quote}
[~asuresh]: Yes, thanks.
bq. UserGroupsMappingProvider pluggable interface is a good example, which even
allows to query external LDAP server to perform user-groups mapping. We might
borrow similar idea from it for this.
[~drankye]: Thanks. Studying class GroupMappingServiceProvider and
CompositeGroupsMapping (for hadoop.security.group.mapping).
bq. To allow such an interface for the mapping would also allow to implement
the translation rules in modular approach, even not by user code.
What do we mean by modular approach vs. user code here?
bq. I understand the NameNode concern, yes it's possible to involve overhead
for NN if user provided plugin performs the mapping not fast every time. To
alleviate the pain, we could consider to support cache of the mapping results
in the framework.
Perhaps the plugin could forward requests to a local daemon with cache
capability (just like nscd for name service requests) if we have concern in
performance.
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350085#comment-14350085 ] Sunny Cheung commented on HADOOP-11683: --- It is worth mentioning that MIT Kerberos 1.12 added a plugin interface (called localauth) to control the relationship between Kerberos principals and local system accounts [1]. And a 3rd party software (SSSD) has leveraged this feature to support calls to getpwnam() passing in a Kerberos principal name to get normalized user profile back [2]. This implies that (to some degrees) arbitrary mapping of Kerberos principals to local system accounts is a common problem in authentication. References: [1] Local authorization interface (localauth) http://web.mit.edu/kerberos/krb5-1.12/doc/plugindev/localauth.html [2] Allow Kerberos Principals in getpwnam() calls https://fedorahosted.org/sssd/wiki/DesignDocs/NSSWithKerberosPrincipal Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
Sunny Cheung created HADOOP-11683: - Summary: Need a plugin API to translate long principal names to local OS user names arbitrarily Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350066#comment-14350066
]
Sunny Cheung commented on HADOOP-11683:
---
Our problem is that normal user principal names can be very different from
their Unix login. Some customers simply have arbitrary mapping between their
Kerberos principals and Unix user accounts. For example, one customer has over
200K users on AD with Kerberos principals in format first name.last
name@REALM (e.g. john@example.com). But their Unix names are in format
userID or just ID (e.g. user123456, 123456).
So, when Kerberos security is enabled on Hadoop clusters, how should we
configure to authenticate these users from Hadoop clients?
The current way is to use the hadoop.security.auth_to_local setting, e.g. from
core-site.xml:
property
namehadoop.security.auth_to_local/name
value
RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/mapred/
RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/hdfs/
RULE:[2:$1@$0](hm@.*EXAMPLE.COM)s/.*/hbase/
RULE:[2:$1@$0](rs@.*EXAMPLE.COM)s/.*/hbase/
DEFAULT/value
descriptionThe mapping from kerberos principal names
to local OS user names./description /property
These name translation rules can handle cases like mapping service accounts'
principals (e.g. nn/host@REALM or dn/host@REALM to hdfs). But that is not
scalable for normal users. There are just too many users to handle (as compared
to the finite amount of service accounts).
Therefore, we would like to ask if alternative name resolution plugin interface
can be supported by Hadoop. It could be similar to the way alternative
authentication plugin is supported for HTTP web-consoles [1]:
property
namehadoop.http.authentication.type/name
valueorg.my.subclass.of.AltKerberosAuthenticationHandler/value
/property
And the plugin interface can be as simple as this function (error handling
ignored here):
String auth_to_local (String krb5Principal) {
...
return unixName;
}
If this plugin interface is supported by Hadoop, then everyone can provide a
plugin to support arbitrary mapping. This will be extremely useful when
administrators need to tighten security on Hadoop with existing Kerberos
infrastructure.
References:
[1] Authentication for Hadoop HTTP web-consoles
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/HttpAuthentication.html
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
