Aron, TestConnection is just used for lookup purpose only. (To list the resource while maintaining policies). There should be steps to create keytab for rangerlookup, just make sure that you have policy for that user so it can list the hdfs directories/files.
Even if the test connection fails it doesn’t stop you from maintaining policies and using ranger. Regards, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Wednesday, September 20, 2017 at 1:10 PM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup When I am in the Ranger Admin Service Manager -> Edit Service for HDFS, there is a Test Connection button. When I press it, it tries to login with Username and Password. We use keytabs. Tracing the ranger_admin.log, in BaseClient.java, the lookupPrincipal and lookupKeytab are not set. If I force these to be set in the code, it then uses a keytab. I'm not certain how the keytabs are to be specified other than the install.properties file. Clearly I must not have specified them properly. https://github.com/apache/ranger/blob/688807cf74fc434e246a2f7d6c0e71e941178421/agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java#L79-L80 <https://github.com/apache/ranger/blob/688807cf74fc434e246a2f7d6c0e71e941178421/agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java#L79-L80> On Wed, Sep 20, 2017 at 3:00 PM, Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> wrote: Aron, When you say Login in with user and password is that Ranger Admin UI login? Or is the hdfs plugin login into ranger to fetch the policy? Looks like the NPE is not related to Ranger, but please check namenode.log what is there. Please enable debug on namenode and check it out. Thanks, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Wednesday, September 20, 2017 at 12:21 PM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup Thanks. Having lots of issues trying to get this to work. Issue 1 - Admin Server I'm not exactly sure what I am doing right or wrong so far, but it is still trying to login with a user and password rather than a keytab. In BaseClient.login(), I hard-coded the keytab and principal, and then I see a proper HDFS file listing occurring. This however is failing (see issue 2). Looks like it is expecting some xalogin.xml that does not exist to set these properties. I still need to dig into why this does not exist. I'm not clear what authentication mode means exactly. I don't think UNIX/LDAP/AD fit anything we do based on questioning the Hadoop team here. I'm not exactly clear what this setting is used for or which setting we should specify. Issue 2 - Namenode I installed the plugin and was able to restart the name node, but no policy data was in the cache directory, it appears unable to sync. When I do a "hadoop ls" on the command line, I get a NPE: Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.getINodeAttrs(FSPermissionChecker.java:243) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:182) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:499) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:1605) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:1623) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.resolvePath(FSDirectory.java:544) at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getListingInt(FSDirStatAndListingOp.java:55) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:3695) On Tue, Sep 19, 2017 at 2:53 PM, Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> wrote: Hi Aaron Please check this out https://cwiki.apache.org/confluence/display/RANGER/Ranger+installation+in+Kerberized++Environment Regards, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Tuesday, September 19, 2017 at 11:58 AM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: HDFS Kerberos documentation/setup 1) What documentation should I be following to install Ranger manually for a Kerberos Hadoop cluster? I am interested in the HDFS Plugin. This is what I found linked from the apache site, but is very old: https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide 2) Following those instructions, I see "Create a repository in Ranger Policy Manager. E.g. "local_hdfs"." Is this the same as creating a Service? I see Services under HDFS on the Ranger admin server. 3) Creating an HDFS service lists a Username and Password. We don't use passwords for our clusters, but have keytabs. What should this mandatory field be? What is it used for? 4) How is this supposed to be setup for secure clusters? Is there any manually setup example I can be pointed to? Thanks, Aaron
