[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752109#comment-15752109
]
Hadoop QA commented on HADOOP-11683:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HADOOP-11683 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-11683 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12766432/HADOOP-11683.003.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11284/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752105#comment-15752105 ] Graham Simpson commented on HADOOP-11683: - This is a darn useful feature for folks using Centrify or such products to integrate correctly with Active Directory. I'd argue this is a critical issue as correct mapping is essential to hadoop operation. You can use auth_to_local to workaround but of course this requires namenode restart which in a large enterprise cluster is not really an option. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497408#comment-15497408
]
Hadoop QA commented on HADOOP-11683:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HADOOP-11683 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-11683 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12766432/HADOOP-11683.003.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10528/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129507#comment-15129507 ] roger mak commented on HADOOP-11683: bq. This work keeps the behaviour and introduced pluggable provider mechanism but hasn't provided any plugin provider yet. You are correct. This work is to provide a configurable hook that will make it possible for other service providers to develop their plugin provider codes to do advanced name translation. How to implement the plugin provider codes is intentionally left to external service providers. Does it make sense? > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127540#comment-15127540 ] Junping Du commented on HADOOP-11683: - Move non-critical issue out of 2.6.4 to 2.6.5. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115110#comment-15115110
]
Kai Zheng commented on HADOOP-11683:
bq. I think it is important to recognize that principal -> username conversion
happens all over the stack.
Agree, this is similar to the user groups mapping behaviour. The configurations
and referenced providers introduced here should be the same on all the nodes.
bq. if a non-Java AM decides to provide user auth (think Slider), it doesn't
appear to have a way to access this functionality without using JNI.
I'm not sure I got this, but with the current codes, non-Java AMs are already
needing to access {{HadoopKerberosName}} or use the current mapping method via
the configuration {{auth_to_local}} I guess? This work keeps the behaviour and
introduced pluggable provider mechanism but hasn't provided any plugin provider
yet.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110802#comment-15110802 ] Allen Wittenauer commented on HADOOP-11683: --- I think it is important to recognize that principal -> username conversion happens all over the stack. For example, every single web UI is going to be needing this functionality. Unless I missed something, the way this code is written will require the mapping code+configuration to be present on every single node in a way that every single process is going to need access. To make matters worse, if a non-Java AM decides to provide user auth (think Slider), it doesn't appear to have a way to access this functionality without using JNI. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110122#comment-15110122
]
Kai Zheng commented on HADOOP-11683:
The patch looks good overall. Some comments.
1. Better to have an abstract like {{AbstractUserNameMappingProvider}} to
implement the new interface and extend {{Configured}}. Then all the providers
like {{CompositeUserNameMapping}} simply extend the abstract.
2. As I previously explained, it would be good to have a simple cache as
{{GroupMappingServiceProvider}} does, because, the convert from user name to
short name may be time consuming and involve a remote service call as you said.
The effort is small, the benefit is worth. Please note the mapping provider is
used in the core part and can be queried some times during a session. We should
try to avoid remote service call as possible.
3. The mapping provider instance should be created only when
{{setConfiguration}} is called, instead of every time a query or convert
happens, in {{HadoopKerberosName}}. Note the change in setConfiguration isn't
necessary.
4. We probably need to support refresh in server side as group mapping does,
it's useful when you change the mapping provider configurations but don't want
to restart your NN server. If so we can trigger it in
{{NameNodeRpcServer#refreshUserToGroupsMappings}}.
5. It maybe a mistake to have two test users of the same name.
{code}
private static TestUser joe = new TestUser("nob...@ad.com", "joe");
private static TestUser john = new TestUser("jackj...@bc.com", "john");
private static TestUser jack = new TestUser("jackj...@bc.com", "jack");
{code}
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109956#comment-15109956 ] Aaron T. Myers commented on HADOOP-11683: - Hey Roger, thanks a lot for taking up this effort. I took a quick look at the patch and it largely looks good to me. I haven't yet done a detailed code review, but I think the direction seems generally appropriate. One small thing I think the patch could definitely benefit from would be breaking out the documentation/example you have in there out of core-default.xml, and into some actual documentation that will end up published on the website. Putting lengthy docs explanations in an XML comment is not typically the way we document things. I can take a harder look at this in the coming days, but I think making that change would be a good start. To answer this question: bq. Just to confirm, since KerberosName and HadoopKerberosName are intended for HDFS and MapReduce projects only (as defined in LimitedPrivate), do we have the option to refactor these classes (and maybe provide an interface similar to GroupMappingServiceProvider)? Yes, that should be fine within our compatibility guidelines. Just be sure not to break HDFS/MR. [~aw] - do you have any more detailed comments on the latest patch? > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109975#comment-15109975 ] Kai Zheng commented on HADOOP-11683: In case it's desired, I'd like to do a careful review for this and hope it helps. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103975#comment-15103975 ] Junping Du commented on HADOOP-11683: - This sounds more like a feature instead of a bug. Will move it to next minor release 2.8.0 instead of 2.6.4 which is a maintenance release if nobody objects. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094384#comment-15094384 ] roger mak commented on HADOOP-11683: Hi all, please help me to know what I need to do to move on to have code review. As I mentioned earlier, the failed test cases have no relation with the code change. Thanks. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094661#comment-15094661
]
Hadoop QA commented on HADOOP-11683:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 42s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 33s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 8s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m 48s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 22m 30s
{color} | {color:red} root-jdk1.7.0_91 with JDK v1.7.0_91 generated 4 new
issues (was 724, now 724). {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 1s {color}
| {color:red} hadoop-common in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 0s
{color} | {color:green} hadoop-common in the patch passed with JDK v1.7.0_91.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 81m 50s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Timed out junit tests |
org.apache.hadoop.http.TestHttpServerLifecycle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12766432/HADOOP-11683.003.patch
|
| JIRA Issue | HADOOP-11683 |
| Optional Tests | asflicense
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15033723#comment-15033723 ] Junping Du commented on HADOOP-11683: - Move it to 2.6.4 as no update for a period of time. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15030115#comment-15030115 ] Junping Du commented on HADOOP-11683: - Hi, can we move this out of 2.6.3? Thanks! > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch, > HADOOP-11683.003.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14956155#comment-14956155
]
Hadoop QA commented on HADOOP-11683:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 21m 54s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 8s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 25s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 7s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 55s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 6m 41s | Tests failed in
hadoop-common. |
| | | 52m 47s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.ipc.TestRPC |
| | hadoop.security.ssl.TestReloadingX509TrustManager |
| | hadoop.metrics2.impl.TestGangliaMetrics |
| | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766432/HADOOP-11683.003.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 40cac59 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7806/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7806/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7806/console |
This message was automatically generated.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955997#comment-14955997 ] roger mak commented on HADOOP-11683: The failed unit test, hadoop.ipc.TestIPC, is unrelated to this patch. I am going to resubmit the patch for retry. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955812#comment-14955812
]
Hadoop QA commented on HADOOP-11683:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 20m 35s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 48s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 49s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 21s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 59s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 48s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 37s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 9m 15s | Tests failed in
hadoop-common. |
| | | 57m 41s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.ipc.TestIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766398/HADOOP-11683.003.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / aa299ec |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7805/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7805/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7805/console |
This message was automatically generated.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch,
> HADOOP-11683.003.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953702#comment-14953702 ] roger mak commented on HADOOP-11683: Update patch for trunk. > Need a plugin API to translate long principal names to local OS user names > arbitrarily > -- > > Key: HADOOP-11683 > URL: https://issues.apache.org/jira/browse/HADOOP-11683 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.6.0 >Reporter: Sunny Cheung >Assignee: roger mak > Attachments: HADOOP-11683.001.patch > > > We need a plugin API to translate long principal names (e.g. > john@example.com) to local OS user names (e.g. user123456) arbitrarily. > For some organizations the name translation is straightforward (e.g. > john@example.com to john_doe), and the hadoop.security.auth_to_local > configurable mapping is sufficient to resolve this (see HADOOP-6526). > However, in some other cases the name translation is arbitrary and cannot be > generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953794#comment-14953794
]
Hadoop QA commented on HADOOP-11683:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766168/HADOOP-11683.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 9849c8b |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7796/console |
This message was automatically generated.
> Need a plugin API to translate long principal names to local OS user names
> arbitrarily
> --
>
> Key: HADOOP-11683
> URL: https://issues.apache.org/jira/browse/HADOOP-11683
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.6.0
>Reporter: Sunny Cheung
>Assignee: roger mak
> Attachments: HADOOP-11683.001.patch, HADOOP-11683.002.patch
>
>
> We need a plugin API to translate long principal names (e.g.
> john@example.com) to local OS user names (e.g. user123456) arbitrarily.
> For some organizations the name translation is straightforward (e.g.
> john@example.com to john_doe), and the hadoop.security.auth_to_local
> configurable mapping is sufficient to resolve this (see HADOOP-6526).
> However, in some other cases the name translation is arbitrary and cannot be
> generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706974#comment-14706974 ] roger mak commented on HADOOP-11683: Hi, we are new to the contribution process. Just wonder if we are on the right track. The automated QA test returned with +1 test case failure which in our opinion is unrelated to our change. Do we need to do anything to restart the review process? or we just need to wait? Thanks in advance for any advice. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707030#comment-14707030 ] roger mak commented on HADOOP-11683: Xiaoyu Yao, thanks. Just re-submitted the patch. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706997#comment-14706997 ] Xiaoyu Yao commented on HADOOP-11683: - [~roger.mak], if you just want to restart a Jenkins test run, Cancel Patch and Submit Patch will do the trick. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697990#comment-14697990 ] roger mak commented on HADOOP-11683: The +1 failed test case, org.apache.hadoop.net.TestClusterTopology.testChooseRandom, has no relation with the change. It also passed in my local environment. The other 6 failed test cases are there before and also have no relation with the change: org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverFailBecomingStandbyAndFailFence org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailover org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverFailBecomingStandby org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverMultipleZKfcs org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverFailBecomingActive org.apache.hadoop.net.TestNetUtils.testNormalizeHostName Please reconsider the test result (or rerun the test). Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Affects Versions: 2.6.0 Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696404#comment-14696404
]
Hadoop QA commented on HADOOP-11683:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 0s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 44s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 43s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 5s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 21s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 53s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | common tests | 22m 16s | Tests failed in
hadoop-common. |
| | | 62m 3s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.net.TestNetUtils |
| | hadoop.net.TestClusterTopology |
| | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12749679/HADOOP-11683.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0a03054 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7465/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7465/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/7465/console |
This message was automatically generated.
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Affects Versions: 2.6.0
Reporter: Sunny Cheung
Assignee: roger mak
Attachments: HADOOP-11683.001.patch
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681046#comment-14681046 ] Sunny Cheung commented on HADOOP-11683: --- Just reassigned this bug to [~roger.mak]. He is my colleague who implements this feature. Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: roger mak Attachments: HADOOP-11683.001.patch We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360037#comment-14360037 ] Sunny Cheung commented on HADOOP-11683: --- bq. Be aware that HadoopKerberosName is now exposed to users in trunk. We should make sure that the solution here also works there. Yes, we are aware of this too. Just to confirm, since KerberosName and HadoopKerberosName are intended for HDFS and MapReduce projects only (as defined in LimitedPrivate), do we have the option to refactor these classes (and maybe provide an interface similar to GroupMappingServiceProvider)? Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358809#comment-14358809 ] Allen Wittenauer commented on HADOOP-11683: --- Just a pre-emptive comment: :) Be aware that HadoopKerberosName is now exposed to users in trunk. We should make sure that the solution here also works there. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358184#comment-14358184 ] Kai Zheng commented on HADOOP-11683: Thanks for your contribution. I just assigned it to yourself. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung Assignee: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358172#comment-14358172 ] Sunny Cheung commented on HADOOP-11683: --- bq. Would you contribute and do it yourself ? If so I can assign this to you. Yes, Centrify is absolutely willing to do this for the Hadoop project and donate code. Thanks. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354240#comment-14354240 ] Kai Zheng commented on HADOOP-11683: bq.What do we mean by modular approach vs. user code here ? I mean by having an interface here, we can have different modular implementation classes for each mechanism, instead of mixing all of mechanisms together in one *BIG* class. bq.Perhaps the plugin could forward requests to a local daemon with cache capability I'm not sure that would eliminate the necessity of caching results in Java side, particularly considering external daemon or service might not be connectable or reliable to NameNode. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352754#comment-14352754
]
Sunny Cheung commented on HADOOP-11683:
---
{quote}
I am assuming you are talking about :
john@example.com - user123
foo.sm...@example.com - user789
...
possibly some 200k such entries
{quote}
[~asuresh]: Yes, thanks.
bq. UserGroupsMappingProvider pluggable interface is a good example, which even
allows to query external LDAP server to perform user-groups mapping. We might
borrow similar idea from it for this.
[~drankye]: Thanks. Studying class GroupMappingServiceProvider and
CompositeGroupsMapping (for hadoop.security.group.mapping).
bq. To allow such an interface for the mapping would also allow to implement
the translation rules in modular approach, even not by user code.
What do we mean by modular approach vs. user code here?
bq. I understand the NameNode concern, yes it's possible to involve overhead
for NN if user provided plugin performs the mapping not fast every time. To
alleviate the pain, we could consider to support cache of the mapping results
in the framework.
Perhaps the plugin could forward requests to a local daemon with cache
capability (just like nscd for name service requests) if we have concern in
performance.
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351116#comment-14351116
]
Kai Zheng commented on HADOOP-11683:
bq.we already have user-code running in the NN now
{{UserGroupsMappingProvider}} pluggable interface is a good example, which even
allows to query external LDAP server to perform user-groups mapping. We might
borrow similar idea from it for this.
To allow such an interface for the mapping would also allow to implement the
translation rules in modular approach, even not by user code.
I understand the NameNode concern, yes it's possible to involve overhead for NN
if user provided plugin performs the mapping not fast every time. To alleviate
the pain, we could consider to support cache of the mapping results in the
framework.
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350846#comment-14350846 ] Leo Liou commented on HADOOP-11683: --- Interesting idea about the mapping file. To sync the files on many nodes is still issue, although NFS is one possibility. Still, the users would either write their own plug-in (to an interface), or obtain it from some ISV. It still has to be deployed in a controlled fashion - since users would have to configure it. So, maybe I missed the point about the concern for a plug-in in this case. My other thought is that translation rules are still fairly rigid. None of us can foresee all the different issues or possibilities in the future. I would suggest that to delegate this part to a plug-in and not making it a Hadoop issue is quite attractive. just my 2 cents. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350943#comment-14350943 ] Allen Wittenauer commented on HADOOP-11683: --- bq. I do have some reservations against making this is User specified class though. Considering that this would be user code that would be executed within possibly critical sections of the HDFS code. This is the enough rope to hang yourself principle. It should be hard to do, but not impossible. The vast majority of folks will use the built-in stuff, but the edge case people need it. Besides, we already have user-code running in the NN now. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350096#comment-14350096
]
Arun Suresh commented on HADOOP-11683:
--
[~sunny.cheung], this is definitely an interesting JIRA
I am assuming you are talking about :
{noformat}
john@example.com - user123
foo.sm...@example.com - user789
...
possibly some 200k such entries
{noformat}
It is still possible to do so via the existing rules interface.
Although I agree it would be pretty long.
I do have some reservations against making this is User specified class though.
Considering that this would be user code that would be executed within possibly
critical sections of the HDFS code.
I would consider adding a property such as :
{noformat}
property
namehadoop.security.auth_to_local.mapping-file/name
value/path/to/some/file/value
/property
{noformat}
instead of allowing a user defined class
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350085#comment-14350085 ] Sunny Cheung commented on HADOOP-11683: --- It is worth mentioning that MIT Kerberos 1.12 added a plugin interface (called localauth) to control the relationship between Kerberos principals and local system accounts [1]. And a 3rd party software (SSSD) has leveraged this feature to support calls to getpwnam() passing in a Kerberos principal name to get normalized user profile back [2]. This implies that (to some degrees) arbitrary mapping of Kerberos principals to local system accounts is a common problem in authentication. References: [1] Local authorization interface (localauth) http://web.mit.edu/kerberos/krb5-1.12/doc/plugindev/localauth.html [2] Allow Kerberos Principals in getpwnam() calls https://fedorahosted.org/sssd/wiki/DesignDocs/NSSWithKerberosPrincipal Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[
https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350066#comment-14350066
]
Sunny Cheung commented on HADOOP-11683:
---
Our problem is that normal user principal names can be very different from
their Unix login. Some customers simply have arbitrary mapping between their
Kerberos principals and Unix user accounts. For example, one customer has over
200K users on AD with Kerberos principals in format first name.last
name@REALM (e.g. john@example.com). But their Unix names are in format
userID or just ID (e.g. user123456, 123456).
So, when Kerberos security is enabled on Hadoop clusters, how should we
configure to authenticate these users from Hadoop clients?
The current way is to use the hadoop.security.auth_to_local setting, e.g. from
core-site.xml:
property
namehadoop.security.auth_to_local/name
value
RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/mapred/
RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/hdfs/
RULE:[2:$1@$0](hm@.*EXAMPLE.COM)s/.*/hbase/
RULE:[2:$1@$0](rs@.*EXAMPLE.COM)s/.*/hbase/
DEFAULT/value
descriptionThe mapping from kerberos principal names
to local OS user names./description /property
These name translation rules can handle cases like mapping service accounts'
principals (e.g. nn/host@REALM or dn/host@REALM to hdfs). But that is not
scalable for normal users. There are just too many users to handle (as compared
to the finite amount of service accounts).
Therefore, we would like to ask if alternative name resolution plugin interface
can be supported by Hadoop. It could be similar to the way alternative
authentication plugin is supported for HTTP web-consoles [1]:
property
namehadoop.http.authentication.type/name
valueorg.my.subclass.of.AltKerberosAuthenticationHandler/value
/property
And the plugin interface can be as simple as this function (error handling
ignored here):
String auth_to_local (String krb5Principal) {
...
return unixName;
}
If this plugin interface is supported by Hadoop, then everyone can provide a
plugin to support arbitrary mapping. This will be extremely useful when
administrators need to tighten security on Hadoop with existing Kerberos
infrastructure.
References:
[1] Authentication for Hadoop HTTP web-consoles
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/HttpAuthentication.html
Need a plugin API to translate long principal names to local OS user names
arbitrarily
--
Key: HADOOP-11683
URL: https://issues.apache.org/jira/browse/HADOOP-11683
Project: Hadoop Common
Issue Type: Improvement
Components: security
Reporter: Sunny Cheung
We need a plugin API to translate long principal names (e.g.
john@example.com) to local OS user names (e.g. user123456) arbitrarily.
For some organizations the name translation is straightforward (e.g.
john@example.com to john_doe), and the hadoop.security.auth_to_local
configurable mapping is sufficient to resolve this (see HADOOP-6526).
However, in some other cases the name translation is arbitrary and cannot be
generalized by a set of translation rules easily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11683) Need a plugin API to translate long principal names to local OS user names arbitrarily
[ https://issues.apache.org/jira/browse/HADOOP-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350086#comment-14350086 ] Kai Zheng commented on HADOOP-11683: By *arbitrarily* what did you mean, in the JIRA description ? Would you contribute and do it yourself ? If so I can assign this to you. Need a plugin API to translate long principal names to local OS user names arbitrarily -- Key: HADOOP-11683 URL: https://issues.apache.org/jira/browse/HADOOP-11683 Project: Hadoop Common Issue Type: Improvement Components: security Reporter: Sunny Cheung We need a plugin API to translate long principal names (e.g. john@example.com) to local OS user names (e.g. user123456) arbitrarily. For some organizations the name translation is straightforward (e.g. john@example.com to john_doe), and the hadoop.security.auth_to_local configurable mapping is sufficient to resolve this (see HADOOP-6526). However, in some other cases the name translation is arbitrary and cannot be generalized by a set of translation rules easily. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
