[ 
https://issues.apache.org/jira/browse/HADOOP-15213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358563#comment-16358563
 ] 

Kihwal Lee commented on HADOOP-15213:
-------------------------------------

{quote}Please provide valid references for your assertion that netgroups are 
meant for service accounts
{quote}
I've never mentioned "service accounts" and this has nothing to do with service 
accounts. {{AccessControlList}} is used for controlling rpc protocol accesses, 
scheduler queue accesses, etc.

We (Yahoo!) designed it in 2010. [~devaraj] worked on it at that time. We have 
improved various user documentations in recent years, but it was very lacking 
back then because most users were also hadoop core developers. Many still 
remain to be improved. At minimum, we should clarify this in the apache doc. I 
apologize for lacking or misleading documentation. 

I am not claiming netgroups should only be used for ACLs. I am simply saying 
that these modules were designed to support the specific use cases.  If your 
use case isn't well supported with the current offerings, you can start 
discussion in the user mailing list and that could lead to a new feature.

> JniBasedUnixGroupsNetgroupMapping.java and 
> ShellBasedUnixGroupsNetgroupMapping.java use netgroup.substring(1) 
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-15213
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15213
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>         Environment: SUSE Linux Enterprise Server 11 (x86_64)
> VERSION = 11
> PATCHLEVEL = 3
>            Reporter: Dhirendra Khanka
>            Priority: Minor
>
>  
> Part of the code below shown from below 2 classes
>  org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping.java
> {code:java}
>  protected synchronized List<String> getUsersForNetgroup(String netgroup) {
>     String[] users = null;
>     try {
>       // JNI code does not expect '@' at the begining of the group name
>       users = getUsersForNetgroupJNI(netgroup.substring(1));
>     } catch (Exception e) {
>       if (LOG.isDebugEnabled()) {
>         LOG.debug("Error getting users for netgroup " + netgroup, e);
>       } else {
>         LOG.info("Error getting users for netgroup " + netgroup + 
>             ": " + e.getMessage());
>       }
>     }
>     if (users != null && users.length != 0) {
>       return Arrays.asList(users);
>     }
>     return new LinkedList<String>();
>   }{code}
> org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping.java
>  
> {code:java}
> protected String execShellGetUserForNetgroup(final String netgroup)
>  throws IOException {
>  String result = "";
>  try
> { // shell command does not expect '@' at the begining of the group name 
> result = Shell.execCommand( 
> Shell.getUsersForNetgroupCommand(netgroup.substring(1))); }
> catch (ExitCodeException e)
> { // if we didn't get the group - just return empty list; LOG.warn("error 
> getting users for netgroup " + netgroup, e); }
> return result;
>  }
> {code}
>  The comments from the code above expect the input to contain '@' , however 
> when executing the shell directly the output has the below form which does 
> not contain any ampersand symbol.
> {code:java}
> :~> getent netgroup mynetgroup1
> mynetgroup1           ( , a3xsds, ) ( , beekvkl, ) ( , redcuan, ) ( , 
> uedfmst, ){code}
>  
> I have created a test code and removed the substring function and then ran it 
> on the cluster using hadoop jar. The code returned netgroups correctly after 
> the modification. I have limited knowledge on netgroup. The issue was 
> discovered when
> hadoop.security.group.mapping = 
> *org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback* was added 
> to core-site.xml and it failed to apply netgroup access.
>  
> Also find below debug comment to see negroup api calls in action
> tdms@casatdhdp01master01:~> hdfs dfs -ls /user/tdms
> 18/02/09 09:47:30 DEBUG util.Shell: setsid exited with exit code 0
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL 
> jar:file:/usr/hdp/2.5.3.0-37/hadoop/hadoop-common-2.7.3.2.5.3.0-37.jar!/core-default.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream 
> sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@78186a70
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL 
> file:/etc/hadoop/2.5.3.0-37/0/core-site.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream 
> java.io.BufferedInputStream@15d9bc04
> 18/02/09 09:47:30 DEBUG security.SecurityUtil: Setting 
> hadoop.security.token.service.use_ip to true
> 18/02/09 09:47:30 DEBUG util.KerberosName: Kerberos krb5 configuration not 
> found, setting default realm to empty
> 18/02/09 09:47:30 DEBUG security.Groups:  Creating new Groups object
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Trying to load the 
> custom-built native-hadoop library...
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Loaded the native-hadoop 
> library
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsMapping: Using 
> JniBasedUnixGroupsMapping for Group resolution
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsNetgroupMapping: Using 
> JniBasedUnixGroupsNetgroupMapping for Netgroup resolution
> 18/02/09 09:47:30 DEBUG 
> security.JniBasedUnixGroupsNetgroupMappingWithFallback: Group mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping
> 18/02/09 09:47:30 DEBUG security.Groups: Group mapping 
> impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMappingWithFallback;
>  cacheTimeout=300000; warningDeltaMs=5000
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login commit
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: using local 
> user:UnixPrincipal: tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: Using user: 
> "UnixPrincipal: tdms" with name tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: User entry: "tdms"
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: UGI loginUser:tdms 
> (auth:SIMPLE)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to