Yes, the looks like son deserialization issue. Please check the libraries in /hadoop/lib/ and ranger ews/webapp/WEB-INF/lib folder jersey-json*jar is there and all same version.
Regards, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Friday, September 22, 2017 at 6:34 AM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup Apparently the Admin server is sending back an HTML form to have the name node logon. I'm not sure how this is supposed to work. I assume the HDFS plugin would take care of this somehow before syncing the policies or throw an exception that it could not log on? Can someone explain what I should be doing to make this work? Thanks, Aaron On Thu, Sep 21, 2017 at 3:50 PM, Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> wrote: I encountered an error "Class org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer not found" when starting the name node. I added the patch for RANGER-1412 to my enable-pdfs-plugin.sh script and got past this error. Now when I start the namenode, I see this callstack: https://github.com/apache/ranger/blob/ranger-0.7/agents-common/src/main/java/org/apache/ranger/admin/client/RangerAdminRESTClient.java#L141 2017-09-21 20:43:33,571 [main] ERROR util.PolicyRefresher: PolicyRefresher(serviceName=openqe79blue): failed to refresh policies. Will continue to use last known version of policies (-1) com.sun.jersey.api.client.ClientHandlerException: A message body reader for Java class org.apache.ranger.plugin.util.ServicePolicies, and Java type class org.apache.ranger.plugin.util.ServicePolicies, and MIME media type text/html; charset=ISO-8859-1 was not found at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:549) at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:506) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:141) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202) at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:149) at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:157) at org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init(RangerHdfsAuthorizer.java:613) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:98) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:86) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1131) at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:760) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:711) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:905) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1610) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1678) On Wed, Sep 20, 2017 at 4:59 PM, Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> wrote: Aron, Also make sure that the ranger admin conf files /etc/ranger/admin/conf/ranger-admin-site.xml has these rangerlookup kerberos principal. Should be there after installation. Regards, Ramesh From: Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> Date: Wednesday, September 20, 2017 at 1:32 PM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup Aron, TestConnection is just used for lookup purpose only. (To list the resource while maintaining policies). There should be steps to create keytab for rangerlookup, just make sure that you have policy for that user so it can list the hdfs directories/files. Even if the test connection fails it doesn’t stop you from maintaining policies and using ranger. Regards, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Wednesday, September 20, 2017 at 1:10 PM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup When I am in the Ranger Admin Service Manager -> Edit Service for HDFS, there is a Test Connection button. When I press it, it tries to login with Username and Password. We use keytabs. Tracing the ranger_admin.log, in BaseClient.java, the lookupPrincipal and lookupKeytab are not set. If I force these to be set in the code, it then uses a keytab. I'm not certain how the keytabs are to be specified other than the install.properties file. Clearly I must not have specified them properly. https://github.com/apache/ranger/blob/688807cf74fc434e246a2f7d6c0e71e941178421/agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java#L79-L80 <https://github.com/apache/ranger/blob/688807cf74fc434e246a2f7d6c0e71e941178421/agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java#L79-L80> On Wed, Sep 20, 2017 at 3:00 PM, Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> wrote: Aron, When you say Login in with user and password is that Ranger Admin UI login? Or is the hdfs plugin login into ranger to fetch the policy? Looks like the NPE is not related to Ranger, but please check namenode.log what is there. Please enable debug on namenode and check it out. Thanks, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Wednesday, September 20, 2017 at 12:21 PM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: Re: HDFS Kerberos documentation/setup Thanks. Having lots of issues trying to get this to work. Issue 1 - Admin Server I'm not exactly sure what I am doing right or wrong so far, but it is still trying to login with a user and password rather than a keytab. In BaseClient.login(), I hard-coded the keytab and principal, and then I see a proper HDFS file listing occurring. This however is failing (see issue 2). Looks like it is expecting some xalogin.xml that does not exist to set these properties. I still need to dig into why this does not exist. I'm not clear what authentication mode means exactly. I don't think UNIX/LDAP/AD fit anything we do based on questioning the Hadoop team here. I'm not exactly clear what this setting is used for or which setting we should specify. Issue 2 - Namenode I installed the plugin and was able to restart the name node, but no policy data was in the cache directory, it appears unable to sync. When I do a "hadoop ls" on the command line, I get a NPE: Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:238) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.getINodeAttrs(FSPermissionChecker.java:243) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:182) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:499) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:1605) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTraverse(FSDirectory.java:1623) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.resolvePath(FSDirectory.java:544) at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getListingInt(FSDirStatAndListingOp.java:55) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:3695) On Tue, Sep 19, 2017 at 2:53 PM, Ramesh Mani <rm...@hortonworks.com<mailto:rm...@hortonworks.com>> wrote: Hi Aaron Please check this out https://cwiki.apache.org/confluence/display/RANGER/Ranger+installation+in+Kerberized++Environment Regards, Ramesh From: Aaron Gresch <agre...@gmail.com<mailto:agre...@gmail.com>> Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Date: Tuesday, September 19, 2017 at 11:58 AM To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" <user@ranger.apache.org<mailto:user@ranger.apache.org>> Subject: HDFS Kerberos documentation/setup 1) What documentation should I be following to install Ranger manually for a Kerberos Hadoop cluster? I am interested in the HDFS Plugin. This is what I found linked from the apache site, but is very old: https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide 2) Following those instructions, I see "Create a repository in Ranger Policy Manager. E.g. "local_hdfs"." Is this the same as creating a Service? I see Services under HDFS on the Ranger admin server. 3) Creating an HDFS service lists a Username and Password. We don't use passwords for our clusters, but have keytabs. What should this mandatory field be? What is it used for? 4) How is this supposed to be setup for secure clusters? Is there any manually setup example I can be pointed to? Thanks, Aaron
