Sorry hit send too early. Actually Access audit tab is still empty. But I can see the HDFS services under audit >> plugins tab
Regards, Aneela Saleem On Jul 30, 2015 4:37 AM, "Aneela Saleem" <[email protected]> wrote: > Yes Audits are being shown now. I just upgraded my hadoop to 2.7.0. > > And secondly another process was running on same port 6080. So I killed > that one and restarted hadoop. And the problem was resolved. > > Now heading towards LDAP user sync and having some issues. I'll start > another thread for this. > > Regards, > Aneela Saleem > On Jul 30, 2015 4:32 AM, "Don Bosco Durai" <[email protected]> wrote: > >> This is good. Are the access audits also showing up now? >> >> Did you do anything special to get this working? >> >> Thanks >> >> Bosco >> >> >> From: Aneela Saleem <[email protected]> >> Reply-To: "[email protected]" < >> [email protected]> >> Date: Wednesday, July 29, 2015 at 1:51 PM >> To: "[email protected]" <[email protected]> >> Subject: Re: Error Setting up Ranger-Admin >> >> Hdfs Plugin has been shown under Audits. Thanks All >> >> On Thu, Jul 30, 2015 at 1:28 AM, Aneela Saleem <[email protected]> >> wrote: >> >>> I'm still unable to see any plugins under Audits, even after upgrading >>> my HDFS to 2.7.0 >>> >>> When i create the Service/Repo from Ranger Admin UI, i get the following >>> Exception: >>> >>> Unable to retrieve any files using given parameters, You can still save >>> the repository and start creating policies, but you would not be able to >>> use autocomplete for resource names. Check *xa_portal.log* for more >>> info. >>> >>> *xa_portal.log *has the following latest logs: >>> >>> 2015-07-30 00:46:51,977 [http-bio-6080-exec-12] INFO >>> org.apache.ranger.biz.SessionMgr (SessionMgr.java:140) - Login Success: >>> loginId=admin, sessionId=71, sessionId=7BF27DADE24C1A2F3A7D4D386A6273BC, >>> requestId=127.0.0.1 >>> 2015-07-30 00:47:10,747 [http-bio-6080-exec-8] INFO >>> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting >>> Policy, policyName: hadoopdev-1-20150729133217 >>> 2015-07-30 00:47:10,905 [http-bio-6080-exec-8] INFO >>> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy >>> Deleted Successfully. PolicyName : hadoopdev-1-20150729133217 >>> 2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN >>> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) - >>> getFilesInDirectory('ranger-plugins/hdfs'): adding >>> /usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0.jar >>> 2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN >>> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) - >>> getFilesInDirectory('ranger-plugins/hdfs'): adding >>> /usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0-tests.jar >>> 2015-07-30 00:48:03,734 [timed-executor-pool-0] INFO >>> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init >>> Login: security not enabled, using username >>> 2015-07-30 00:48:38,154 [timed-executor-pool-0] INFO >>> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init >>> Login: security not enabled, using username >>> 2015-07-30 01:22:41,436 [http-bio-6080-exec-4] INFO >>> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting >>> Policy, policyName: hadoopdev-1-20150729202236 >>> 2015-07-30 01:22:41,526 [http-bio-6080-exec-4] INFO >>> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy >>> Deleted Successfully. PolicyName : hadoopdev-1-20150729202236 >>> 2015-07-30 01:23:10,391 [timed-executor-pool-0] INFO >>> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init >>> Login: security not enabled, using username >>> >>> And where is *xasecure-audit.xml* file located? >>> >>> On Wed, Jul 29, 2015 at 10:36 PM, Don Bosco Durai <[email protected]> >>> wrote: >>> >>>> Yes, either you have to upgrade Hadoop or use Ranger 0.4 >>>> >>>> Thanks >>>> >>>> Bosco >>>> >>>> >>>> From: Aneela Saleem <[email protected]> >>>> Reply-To: "[email protected]" < >>>> [email protected]> >>>> Date: Wednesday, July 29, 2015 at 10:26 AM >>>> >>>> To: "[email protected]" < >>>> [email protected]> >>>> Subject: Re: Error Setting up Ranger-Admin >>>> >>>> I just came to know that Ranger 0.5 is compatible with HDFS 2.7.x and >>>> mine HDFS version is 2.6.0 >>>> >>>> Is it the case? do i need to upgrade Hadoop? >>>> >>>> On Wed, Jul 29, 2015 at 8:09 PM, Aneela Saleem <[email protected]> >>>> wrote: >>>> >>>>> Hi Hanish, >>>>> >>>>> Plugin is still not visible on Audit screen UI >>>>> >>>>> On Wed, Jul 29, 2015 at 7:22 PM, Hanish Bansal < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Aneela, >>>>>> >>>>>> >>>>>> By following below conversation thread, I am getting that you ranger >>>>>> admin UI is working fine but hdfs plugin is not shown in audit logs. If I >>>>>> am right then please try below steps: >>>>>> >>>>>> >>>>>> 1. Copy all the ranger libraries files from "lib" folder to >>>>>> "share/hadoop/hdfs/lib/" >>>>>> cp <hadoop-home>/lib/* <hadoop-home>/share/hadoop/hdfs/lib/ >>>>>> >>>>>> >>>>>> 2. Check if you have created symbolic link for hadoop configuration >>>>>> folder "etc/hadoop" as "conf". If not then create using below command: >>>>>> >>>>>> $ ln -s <hadoop-home>/etc/hadoop <hadoop-home>/conf >>>>>> >>>>>> >>>>>> 3. Restart namenode service. >>>>>> >>>>>> >>>>>> 4. Now login into some other user on machine and try listing files in >>>>>> Hadoop. >>>>>> >>>>>> >>>>>> hadoop fs -ls / >>>>>> >>>>>> >>>>>> Note: In last step, run command using some other user who didn't >>>>>> start the hadoop services. >>>>>> >>>>>> >>>>>> Now check if plugin is visible on audit screen of UI. >>>>>> >>>>>> >>>>>> >>>>>> ------- >>>>>> >>>>>> *Thanks & Regards, Hanish Bansal* >>>>>> Software Engineer, iLabs >>>>>> Impetus Infotech Pvt. Ltd. >>>>>> ------------------------------ >>>>>> *From:* Aneela Saleem <[email protected]> >>>>>> *Sent:* Wednesday, July 29, 2015 7:19 PM >>>>>> *To:* [email protected] >>>>>> >>>>>> *Subject:* Re: Error Setting up Ranger-Admin >>>>>> >>>>>> When i run *./setup.sh * in ranger-admin, i get the following error: >>>>>> >>>>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". >>>>>> >>>>>> On Wed, Jul 29, 2015 at 6:46 PM, Aneela Saleem < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Yes NameNode is running. And i don't see any Ranger-related logs in >>>>>>> NameNode logs >>>>>>> >>>>>>> On Wed, Jul 29, 2015 at 5:41 PM, Loïc Chanel < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> At least, is your NameNode running ? >>>>>>>> And do you can see any Ranger-related logs in the NameNode logs ? >>>>>>>> >>>>>>>> Loïc CHANEL >>>>>>>> Engineering student at TELECOM Nancy >>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>> >>>>>>>> 2015-07-29 14:37 GMT+02:00 Aneela Saleem <[email protected]>: >>>>>>>> >>>>>>>>> I can't figure out what is the issue? as there are no errors in >>>>>>>>> log file >>>>>>>>> >>>>>>>>> On Wed, Jul 29, 2015 at 5:05 PM, Loïc Chanel < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> The logs for ranger plugins are in the NameNode logs, that's why >>>>>>>>>> Bosco told you to take a look there. >>>>>>>>>> >>>>>>>>>> Loïc CHANEL >>>>>>>>>> Engineering student at TELECOM Nancy >>>>>>>>>> Trainee at Worldline - Villeurbanne >>>>>>>>>> >>>>>>>>>> 2015-07-29 12:43 GMT+02:00 Aneela Saleem <[email protected]> >>>>>>>>>> : >>>>>>>>>> >>>>>>>>>>> Hi Bosco, >>>>>>>>>>> >>>>>>>>>>> I tried your solution but it still does not work for me >>>>>>>>>>> >>>>>>>>>>> On Wed, Jul 29, 2015 at 3:22 PM, Aneela Saleem < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> How can i see logs for ranger-hdfs plugin ? >>>>>>>>>>>> >>>>>>>>>>>> Can you please have a look on install.properties file? is it >>>>>>>>>>>> correct? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jul 29, 2015 at 10:57 AM, Don Bosco Durai < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Yes, this issues seems to be HDFS namenode was not stopped. >>>>>>>>>>>>> Can you check the running process by running “ps –eafww | grep >>>>>>>>>>>>> proc_namenode” and if it is running you have to stop it and start >>>>>>>>>>>>> it again. >>>>>>>>>>>>> >>>>>>>>>>>>> Regarding the chown error, it seems the scripts assumes the >>>>>>>>>>>>> user to be “hdfs”. Can you manually update >>>>>>>>>>>>> enable-hdfs-plugin.sh and rerun it? >>>>>>>>>>>>> >>>>>>>>>>>>> Line number 89 of enable-hdfs-plugin.sh >>>>>>>>>>>>> >>>>>>>>>>>>> #CFG_OWNER_INF="${HCOMPONENT_NAME}:${HCOMPONENT_NAME}” >>>>>>>>>>>>> >>>>>>>>>>>>> CFG_OWNER_INF=hduser >>>>>>>>>>>>> >>>>>>>>>>>>> And run ./enable-hdfs-plugin.sh again? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> Bosco >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> From: Ramesh Mani <[email protected]> >>>>>>>>>>>>> Reply-To: "[email protected]" < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:45 PM >>>>>>>>>>>>> >>>>>>>>>>>>> To: "[email protected]" < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin >>>>>>>>>>>>> >>>>>>>>>>>>> By looking at this error it seems that port 50070 >>>>>>>>>>>>> <http://0.0.0.0:50070> is in use. Check that namenode is >>>>>>>>>>>>> already running. If so stop and start. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Ramesh >>>>>>>>>>>>> >>>>>>>>>>>>> From: Aneela Saleem <[email protected]> >>>>>>>>>>>>> Reply-To: "[email protected]" < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:37 PM >>>>>>>>>>>>> To: "[email protected]" < >>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin >>>>>>>>>>>>> >>>>>>>>>>>>> I'm continuously stuck at this point >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Aneela Saleem >>>>>>>>>>>>> On Jul 29, 2015 4:37 AM, "Aneela Saleem" < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> How can we check whether the problem is in Ranger's >>>>>>>>>>>>>> configuration or Hadoop? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Aneela Saleem >>>>>>>>>>>>>> On Jul 29, 2015 2:03 AM, "Aneela Saleem" < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Following are the Logs: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2015-07-29 02:01:22,126 FATAL >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to >>>>>>>>>>>>>>> start namenode. >>>>>>>>>>>>>>> java.net.BindException: Port in use: 0.0.0.0:50070 >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:891) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:827) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:703) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:590) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504) >>>>>>>>>>>>>>> Caused by: java.net.BindException: Address already in use >>>>>>>>>>>>>>> at sun.nio.ch.Net.bind0(Native Method) >>>>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:463) >>>>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:455) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:886) >>>>>>>>>>>>>>> ... 8 more >>>>>>>>>>>>>>> 2015-07-29 02:01:22,128 INFO >>>>>>>>>>>>>>> org.apache.hadoop.util.ExitUtil: Exiting with status 1 >>>>>>>>>>>>>>> 2015-07-29 02:01:22,129 INFO >>>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: >>>>>>>>>>>>>>> /************************************************************ >>>>>>>>>>>>>>> SHUTDOWN_MSG: Shutting down NameNode at aneela-Lenovo-G50-70/ >>>>>>>>>>>>>>> 192.168.1.4 >>>>>>>>>>>>>>> ************************************************************/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:53 AM, Aneela Saleem < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> hduser >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:51 AM, Don Bosco Durai < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This seems to be one of the reason HDFS plugin is not >>>>>>>>>>>>>>>>> enabled. As what user have you installed Hadoop? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Bosco >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> From: Aneela Saleem <[email protected]> >>>>>>>>>>>>>>>>> Reply-To: "[email protected]" < >>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 1:31 PM >>>>>>>>>>>>>>>>> To: "[email protected]" < >>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Also when i run *./enable-hdfs-plugin.sh * >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> i get the following at the end: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> + بدھ جولاي 29 01:28:20 PKT 2015 : Saving current JCE >>>>>>>>>>>>>>>>> file: /etc/ranger/hadoopdev/cred.jceks to >>>>>>>>>>>>>>>>> /etc/ranger/hadoopdev/.cred.jceks.20150729012820 ... >>>>>>>>>>>>>>>>> *chown: invalid user: ‘hdfs:hdfs’* >>>>>>>>>>>>>>>>> Ranger Plugin for hadoop has been enabled. Please restart >>>>>>>>>>>>>>>>> hadoop to ensure that changes are effective. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> NOTE: This message may contain information that is confidential, >>>>>> proprietary, privileged or otherwise protected by law. The message is >>>>>> intended solely for the named addressee. If received in error, please >>>>>> destroy and notify the sender. Any use of this email is prohibited when >>>>>> received in error. Impetus does not represent, warrant and/or guarantee, >>>>>> that the integrity of this communication has been maintained nor that the >>>>>> communication is free of errors, virus, interception or interference. >>>>>> >>>>> >>>>> >>>> >>> >>
