[
https://issues.apache.org/jira/browse/HADOOP-15213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358139#comment-16358139
]
Dhirendra Khanka commented on HADOOP-15213:
-------------------------------------------
Please provide valid references for your assertion that netgroups are meant for
service accounts. I don't mean its a bug. Its just that the implementation
should check if substring(1) = '@' if yes then use netgroups.substring(1) else
just pass netgroups.
I can show you the reference below
[https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/GroupsMapping.html]
Which states below
The groups of a user is determined by a group mapping service provider. Hadoop
supports various group mapping mechanisms, configured by the
{{hadoop.security.group.mapping}} property. Some of them, such as
{{JniBasedUnixGroupsMappingWithFallback}}, use operating systems’ group name
resolution and requires no configuration. But Hadoop also supports special
group mapping mechanisms through LDAP and composition of LDAP and operating
system group name resolution, which require additional configurations.
{{hadoop.security.group.mapping}} can be one of the following:
* *org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping*
This implementation is similar to {{ShellBasedUnixGroupsMapping}}, except that
it executes {color:#FF0000}{{getent netgroup}} command to get netgroup
membership.{color}
The reference link {color:#FF0000}does not in any place state that the
netgroups are meant for service accounts ACLs{color}. and Besides I dont
understand in first place why would service accounts be placed on the
netgroups. Aren't service accounts meant to be local unix accounts.
Also find below debug comment to see negroup api calls in action
{code:java}
tdms@casatdhdp01master01:~> hdfs dfs -ls /user/tdms
18/02/09 09:47:30 DEBUG util.Shell: setsid exited with exit code 0
18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL
jar:file:/usr/hdp/2.5.3.0-37/hadoop/hadoop-common-2.7.3.2.5.3.0-37.jar!/core-default.xml
18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@78186a70
18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL
file:/etc/hadoop/2.5.3.0-37/0/core-site.xml
18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream
java.io.BufferedInputStream@15d9bc04
18/02/09 09:47:30 DEBUG security.SecurityUtil: Setting
hadoop.security.token.service.use_ip to true
18/02/09 09:47:30 DEBUG util.KerberosName: Kerberos krb5 configuration not
found, setting default realm to empty
18/02/09 09:47:30 DEBUG security.Groups: Creating new Groups object
18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsMapping: Using
JniBasedUnixGroupsMapping for Group resolution
18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsNetgroupMapping: Using
JniBasedUnixGroupsNetgroupMapping for Netgroup resolution
18/02/09 09:47:30 DEBUG
security.JniBasedUnixGroupsNetgroupMappingWithFallback: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping
18/02/09 09:47:30 DEBUG security.Groups: Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMappingWithFallback;
cacheTimeout=300000; warningDeltaMs=5000
18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login
18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login commit
18/02/09 09:47:30 DEBUG security.UserGroupInformation: using local
user:UnixPrincipal: tdms
18/02/09 09:47:30 DEBUG security.UserGroupInformation: Using user:
"UnixPrincipal: tdms" with name tdms
18/02/09 09:47:30 DEBUG security.UserGroupInformation: User entry: "tdms"
18/02/09 09:47:30 DEBUG security.UserGroupInformation: UGI loginUser:tdms
(auth:SIMPLE){code}
> JniBasedUnixGroupsNetgroupMapping.java and
> ShellBasedUnixGroupsNetgroupMapping.java use netgroup.substring(1)
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-15213
> URL: https://issues.apache.org/jira/browse/HADOOP-15213
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
> Environment: SUSE Linux Enterprise Server 11 (x86_64)
> VERSION = 11
> PATCHLEVEL = 3
> Reporter: Dhirendra Khanka
> Priority: Minor
>
>
> Part of the code below shown from below 2 classes
> org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping.java
> {code:java}
> protected synchronized List<String> getUsersForNetgroup(String netgroup) {
> String[] users = null;
> try {
> // JNI code does not expect '@' at the begining of the group name
> users = getUsersForNetgroupJNI(netgroup.substring(1));
> } catch (Exception e) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("Error getting users for netgroup " + netgroup, e);
> } else {
> LOG.info("Error getting users for netgroup " + netgroup +
> ": " + e.getMessage());
> }
> }
> if (users != null && users.length != 0) {
> return Arrays.asList(users);
> }
> return new LinkedList<String>();
> }{code}
> org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping.java
>
> {code:java}
> protected String execShellGetUserForNetgroup(final String netgroup)
> throws IOException {
> String result = "";
> try
> { // shell command does not expect '@' at the begining of the group name
> result = Shell.execCommand(
> Shell.getUsersForNetgroupCommand(netgroup.substring(1))); }
> catch (ExitCodeException e)
> { // if we didn't get the group - just return empty list; LOG.warn("error
> getting users for netgroup " + netgroup, e); }
> return result;
> }
> {code}
> The comments from the code above expect the input to contain '@' , however
> when executing the shell directly the output has the below form which does
> not contain any ampersand symbol.
> {code:java}
> :~> getent netgroup mynetgroup1
> mynetgroup1 ( , a3xsds, ) ( , beekvkl, ) ( , redcuan, ) ( ,
> uedfmst, ){code}
>
> I have created a test code and removed the substring function and then ran it
> on the cluster using hadoop jar. The code returned netgroups correctly after
> the modification. I have limited knowledge on netgroup. The issue was
> discovered when
> hadoop.security.group.mapping =
> *org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback* was added
> to core-site.xml and it failed to apply netgroup access.
>
> Also find below debug comment to see negroup api calls in action
> tdms@casatdhdp01master01:~> hdfs dfs -ls /user/tdms
> 18/02/09 09:47:30 DEBUG util.Shell: setsid exited with exit code 0
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL
> jar:file:/usr/hdp/2.5.3.0-37/hadoop/hadoop-common-2.7.3.2.5.3.0-37.jar!/core-default.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@78186a70
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL
> file:/etc/hadoop/2.5.3.0-37/0/core-site.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream
> java.io.BufferedInputStream@15d9bc04
> 18/02/09 09:47:30 DEBUG security.SecurityUtil: Setting
> hadoop.security.token.service.use_ip to true
> 18/02/09 09:47:30 DEBUG util.KerberosName: Kerberos krb5 configuration not
> found, setting default realm to empty
> 18/02/09 09:47:30 DEBUG security.Groups: Creating new Groups object
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Trying to load the
> custom-built native-hadoop library...
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Loaded the native-hadoop
> library
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsMapping: Using
> JniBasedUnixGroupsMapping for Group resolution
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsNetgroupMapping: Using
> JniBasedUnixGroupsNetgroupMapping for Netgroup resolution
> 18/02/09 09:47:30 DEBUG
> security.JniBasedUnixGroupsNetgroupMappingWithFallback: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping
> 18/02/09 09:47:30 DEBUG security.Groups: Group mapping
> impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMappingWithFallback;
> cacheTimeout=300000; warningDeltaMs=5000
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login commit
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: using local
> user:UnixPrincipal: tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: Using user:
> "UnixPrincipal: tdms" with name tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: User entry: "tdms"
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: UGI loginUser:tdms
> (auth:SIMPLE)
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
