Apparently the Admin server is sending back an HTML form to have the name node logon. I'm not sure how this is supposed to work. I assume the HDFS plugin would take care of this somehow before syncing the policies or throw an exception that it could not log on?
Can someone explain what I should be doing to make this work? Thanks, Aaron On Thu, Sep 21, 2017 at 3:50 PM, Aaron Gresch <agre...@gmail.com> wrote: > > I encountered an error "Class > org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer > not found" when starting the name node. I added the patch for RANGER-1412 > to my enable-pdfs-plugin.sh script and got past this error. > > Now when I start the namenode, I see this callstack: > > https://github.com/apache/ranger/blob/ranger-0.7/agents- > common/src/main/java/org/apache/ranger/admin/client/ > RangerAdminRESTClient.java#L141 > > > 2017-09-21 20:43:33,571 [main] ERROR util.PolicyRefresher: > PolicyRefresher(serviceName=openqe79blue): failed to refresh policies. > Will continue to use last known version of policies (-1) > > com.sun.jersey.api.client.ClientHandlerException: A message body reader > for Java class org.apache.ranger.plugin.util.ServicePolicies, and Java > type class org.apache.ranger.plugin.util.ServicePolicies, and MIME media > type text/html; charset=ISO-8859-1 was not found > > at com.sun.jersey.api.client.ClientResponse.getEntity( > ClientResponse.java:549) > > at com.sun.jersey.api.client.ClientResponse.getEntity( > ClientResponse.java:506) > > at org.apache.ranger.admin.client.RangerAdminRESTClient. > getServicePoliciesIfUpdated(RangerAdminRESTClient.java:141) > > at org.apache.ranger.plugin.util.PolicyRefresher. > loadPolicyfromPolicyAdmin(PolicyRefresher.java:264) > > at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy( > PolicyRefresher.java:202) > > at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher( > PolicyRefresher.java:149) > > at org.apache.ranger.plugin.service.RangerBasePlugin.init( > RangerBasePlugin.java:157) > > at org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init( > RangerHdfsAuthorizer.java:613) > > at org.apache.ranger.authorization.hadoop. > RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:98) > > at org.apache.ranger.authorization.hadoop. > RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:86) > > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem. > startCommonServices(FSNamesystem.java:1131) > > at org.apache.hadoop.hdfs.server.namenode.NameNode. > startCommonServices(NameNode.java:760) > > at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize( > NameNode.java:711) > > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>( > NameNode.java:905) > > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>( > NameNode.java:884) > > at org.apache.hadoop.hdfs.server.namenode.NameNode. > createNameNode(NameNode.java:1610) > > at org.apache.hadoop.hdfs.server.namenode.NameNode.main( > NameNode.java:1678) > > > On Wed, Sep 20, 2017 at 4:59 PM, Ramesh Mani <rm...@hortonworks.com> > wrote: > >> Aron, >> >> Also make sure that the ranger admin conf files >> /etc/ranger/admin/conf/ranger-admin-site.xml >> has these rangerlookup kerberos principal. Should be there after >> installation. >> >> Regards, >> Ramesh >> >> From: Ramesh Mani <rm...@hortonworks.com> >> Date: Wednesday, September 20, 2017 at 1:32 PM >> >> To: "user@ranger.apache.org" <user@ranger.apache.org> >> Subject: Re: HDFS Kerberos documentation/setup >> >> Aron, >> >> TestConnection is just used for lookup purpose only. (To list the >> resource while maintaining policies). There should be steps to create >> keytab for rangerlookup, just make sure that you have policy for that >> user so it can list the hdfs directories/files. >> >> Even if the test connection fails it doesn’t stop you from maintaining >> policies and using ranger. >> >> Regards, >> Ramesh >> >> From: Aaron Gresch <agre...@gmail.com> >> Reply-To: "user@ranger.apache.org" <user@ranger.apache.org> >> Date: Wednesday, September 20, 2017 at 1:10 PM >> To: "user@ranger.apache.org" <user@ranger.apache.org> >> Subject: Re: HDFS Kerberos documentation/setup >> >> >> When I am in the Ranger Admin Service Manager -> Edit Service for HDFS, >> there is a Test Connection button. When I press it, it tries to login with >> Username and Password. We use keytabs. Tracing the ranger_admin.log, in >> BaseClient.java, the lookupPrincipal and lookupKeytab are not set. If I >> force these to be set in the code, it then uses a keytab. >> >> I'm not certain how the keytabs are to be specified other than the >> install.properties file. Clearly I must not have specified them properly. >> >> https://github.com/apache/ranger/blob/688807cf74fc434e246a2f >> 7d6c0e71e941178421/agents-common/src/main/java/org/ >> apache/ranger/plugin/client/BaseClient.java#L79-L80 >> >> >> <https://github.com/apache/ranger/blob/688807cf74fc434e246a2f7d6c0e71e941178421/agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java#L79-L80> >> >> >> On Wed, Sep 20, 2017 at 3:00 PM, Ramesh Mani <rm...@hortonworks.com> >> wrote: >> >>> Aron, >>> >>> When you say Login in with user and password is that Ranger Admin UI >>> login? Or is the hdfs plugin login into ranger to fetch the policy? >>> >>> Looks like the NPE is not related to Ranger, but please check >>> namenode.log what is there. Please enable debug on namenode and check it >>> out. >>> >>> Thanks, >>> Ramesh >>> >>> From: Aaron Gresch <agre...@gmail.com> >>> Reply-To: "user@ranger.apache.org" <user@ranger.apache.org> >>> Date: Wednesday, September 20, 2017 at 12:21 PM >>> To: "user@ranger.apache.org" <user@ranger.apache.org> >>> Subject: Re: HDFS Kerberos documentation/setup >>> >>> >>> Thanks. >>> >>> Having lots of issues trying to get this to work. >>> >>> Issue 1 - Admin Server >>> >>> I'm not exactly sure what I am doing right or wrong so far, but it is >>> still trying to login with a user and password rather than a keytab. In >>> BaseClient.login(), I hard-coded the keytab and principal, and then I see a >>> proper HDFS file listing occurring. This however is failing (see issue >>> 2). Looks like it is expecting some xalogin.xml that does not exist to set >>> these properties. I still need to dig into why this does not exist. >>> >>> I'm not clear what authentication mode means exactly. I don't think >>> UNIX/LDAP/AD fit anything we do based on questioning the Hadoop team here. >>> I'm not exactly clear what this setting is used for or which setting we >>> should specify. >>> >>> Issue 2 - Namenode >>> >>> I installed the plugin and was able to restart the name node, but no >>> policy data was in the cache directory, it appears unable to sync. When I >>> do a "hadoop ls" on the command line, I get a NPE: >>> >>> Caused by: >>> org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): >>> java.lang.NullPointerException >>> >>> at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:238) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.g >>> etINodeAttrs(FSPermissionChecker.java:243) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.c >>> heckPermission(FSPermissionChecker.java:182) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.c >>> heckTraverse(FSPermissionChecker.java:499) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTrav >>> erse(FSDirectory.java:1605) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkTrav >>> erse(FSDirectory.java:1623) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.resolvePa >>> th(FSDirectory.java:544) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp >>> .getListingInt(FSDirStatAndListingOp.java:55) >>> >>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListi >>> ng(FSNamesystem.java:3695) >>> >>> >>> On Tue, Sep 19, 2017 at 2:53 PM, Ramesh Mani <rm...@hortonworks.com> >>> wrote: >>> >>>> Hi Aaron >>>> >>>> Please check this out https://cwiki.apache.org/c >>>> onfluence/display/RANGER/Ranger+installation+in+Kerberized++Environment >>>> >>>> Regards, >>>> Ramesh >>>> >>>> From: Aaron Gresch <agre...@gmail.com> >>>> Reply-To: "user@ranger.apache.org" <user@ranger.apache.org> >>>> Date: Tuesday, September 19, 2017 at 11:58 AM >>>> To: "user@ranger.apache.org" <user@ranger.apache.org> >>>> Subject: HDFS Kerberos documentation/setup >>>> >>>> 1) What documentation should I be following to install Ranger manually >>>> for a Kerberos Hadoop cluster? I am interested in the HDFS Plugin. >>>> >>>> This is what I found linked from the apache site, but is very old: >>>> >>>> https://cwiki.apache.org/confluence/display/RANGER/Ranger+In >>>> stallation+Guide >>>> >>>> >>>> 2) Following those instructions, I see "Create a repository in Ranger >>>> Policy Manager. E.g. "local_hdfs"." >>>> >>>> Is this the same as creating a Service? I see Services under HDFS on >>>> the Ranger admin server. >>>> >>>> >>>> 3) Creating an HDFS service lists a Username and Password. We don't >>>> use passwords for our clusters, but have keytabs. What should this >>>> mandatory field be? What is it used for? >>>> >>>> >>>> 4) How is this supposed to be setup for secure clusters? Is there any >>>> manually setup example I can be pointed to? >>>> >>>> >>>> Thanks, >>>> Aaron >>>> >>> >>> >> >
