I'm still unable to see any plugins under Audits, even after upgrading my
HDFS to 2.7.0
When i create the Service/Repo from Ranger Admin UI, i get the following
Exception:
Unable to retrieve any files using given parameters, You can still save the
repository and start creating policies, but you would not be able to use
autocomplete for resource names. Check *xa_portal.log* for more info.
*xa_portal.log *has the following latest logs:
2015-07-30 00:46:51,977 [http-bio-6080-exec-12] INFO
org.apache.ranger.biz.SessionMgr (SessionMgr.java:140) - Login Success:
loginId=admin, sessionId=71, sessionId=7BF27DADE24C1A2F3A7D4D386A6273BC,
requestId=127.0.0.1
2015-07-30 00:47:10,747 [http-bio-6080-exec-8] INFO
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting
Policy, policyName: hadoopdev-1-20150729133217
2015-07-30 00:47:10,905 [http-bio-6080-exec-8] INFO
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy
Deleted Successfully. PolicyName : hadoopdev-1-20150729133217
2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN
org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) -
getFilesInDirectory('ranger-plugins/hdfs'): adding
/usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0.jar
2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN
org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) -
getFilesInDirectory('ranger-plugins/hdfs'): adding
/usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0-tests.jar
2015-07-30 00:48:03,734 [timed-executor-pool-0] INFO
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
Login: security not enabled, using username
2015-07-30 00:48:38,154 [timed-executor-pool-0] INFO
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
Login: security not enabled, using username
2015-07-30 01:22:41,436 [http-bio-6080-exec-4] INFO
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting
Policy, policyName: hadoopdev-1-20150729202236
2015-07-30 01:22:41,526 [http-bio-6080-exec-4] INFO
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy
Deleted Successfully. PolicyName : hadoopdev-1-20150729202236
2015-07-30 01:23:10,391 [timed-executor-pool-0] INFO
org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
Login: security not enabled, using username
And where is *xasecure-audit.xml* file located?
On Wed, Jul 29, 2015 at 10:36 PM, Don Bosco Durai <[email protected]> wrote:
> Yes, either you have to upgrade Hadoop or use Ranger 0.4
>
> Thanks
>
> Bosco
>
>
> From: Aneela Saleem <[email protected]>
> Reply-To: "[email protected]" <
> [email protected]>
> Date: Wednesday, July 29, 2015 at 10:26 AM
>
> To: "[email protected]" <[email protected]>
> Subject: Re: Error Setting up Ranger-Admin
>
> I just came to know that Ranger 0.5 is compatible with HDFS 2.7.x and mine
> HDFS version is 2.6.0
>
> Is it the case? do i need to upgrade Hadoop?
>
> On Wed, Jul 29, 2015 at 8:09 PM, Aneela Saleem <[email protected]>
> wrote:
>
>> Hi Hanish,
>>
>> Plugin is still not visible on Audit screen UI
>>
>> On Wed, Jul 29, 2015 at 7:22 PM, Hanish Bansal <
>> [email protected]> wrote:
>>
>>> Hi Aneela,
>>>
>>>
>>> By following below conversation thread, I am getting that you ranger
>>> admin UI is working fine but hdfs plugin is not shown in audit logs. If I
>>> am right then please try below steps:
>>>
>>>
>>> 1. Copy all the ranger libraries files from "lib" folder to
>>> "share/hadoop/hdfs/lib/"
>>> cp <hadoop-home>/lib/* <hadoop-home>/share/hadoop/hdfs/lib/
>>>
>>>
>>> 2. Check if you have created symbolic link for hadoop configuration
>>> folder "etc/hadoop" as "conf". If not then create using below command:
>>>
>>> $ ln -s <hadoop-home>/etc/hadoop <hadoop-home>/conf
>>>
>>>
>>> 3. Restart namenode service.
>>>
>>>
>>> 4. Now login into some other user on machine and try listing files in
>>> Hadoop.
>>>
>>>
>>> hadoop fs -ls /
>>>
>>>
>>> Note: In last step, run command using some other user who didn't start
>>> the hadoop services.
>>>
>>>
>>> Now check if plugin is visible on audit screen of UI.
>>>
>>>
>>>
>>> -------
>>>
>>> *Thanks & Regards, Hanish Bansal*
>>> Software Engineer, iLabs
>>> Impetus Infotech Pvt. Ltd.
>>> ------------------------------
>>> *From:* Aneela Saleem <[email protected]>
>>> *Sent:* Wednesday, July 29, 2015 7:19 PM
>>> *To:* [email protected]
>>>
>>> *Subject:* Re: Error Setting up Ranger-Admin
>>>
>>> When i run *./setup.sh * in ranger-admin, i get the following error:
>>>
>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>>>
>>> On Wed, Jul 29, 2015 at 6:46 PM, Aneela Saleem <[email protected]>
>>> wrote:
>>>
>>>> Yes NameNode is running. And i don't see any Ranger-related logs in
>>>> NameNode logs
>>>>
>>>> On Wed, Jul 29, 2015 at 5:41 PM, Loïc Chanel <
>>>> [email protected]> wrote:
>>>>
>>>>> At least, is your NameNode running ?
>>>>> And do you can see any Ranger-related logs in the NameNode logs ?
>>>>>
>>>>> Loïc CHANEL
>>>>> Engineering student at TELECOM Nancy
>>>>> Trainee at Worldline - Villeurbanne
>>>>>
>>>>> 2015-07-29 14:37 GMT+02:00 Aneela Saleem <[email protected]>:
>>>>>
>>>>>> I can't figure out what is the issue? as there are no errors in log
>>>>>> file
>>>>>>
>>>>>> On Wed, Jul 29, 2015 at 5:05 PM, Loïc Chanel <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> The logs for ranger plugins are in the NameNode logs, that's why
>>>>>>> Bosco told you to take a look there.
>>>>>>>
>>>>>>> Loïc CHANEL
>>>>>>> Engineering student at TELECOM Nancy
>>>>>>> Trainee at Worldline - Villeurbanne
>>>>>>>
>>>>>>> 2015-07-29 12:43 GMT+02:00 Aneela Saleem <[email protected]>:
>>>>>>>
>>>>>>>> Hi Bosco,
>>>>>>>>
>>>>>>>> I tried your solution but it still does not work for me
>>>>>>>>
>>>>>>>> On Wed, Jul 29, 2015 at 3:22 PM, Aneela Saleem <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> How can i see logs for ranger-hdfs plugin ?
>>>>>>>>>
>>>>>>>>> Can you please have a look on install.properties file? is it
>>>>>>>>> correct?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 29, 2015 at 10:57 AM, Don Bosco Durai <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, this issues seems to be HDFS namenode was not stopped. Can
>>>>>>>>>> you check the running process by running “ps –eafww | grep
>>>>>>>>>> proc_namenode”
>>>>>>>>>> and if it is running you have to stop it and start it again.
>>>>>>>>>>
>>>>>>>>>> Regarding the chown error, it seems the scripts assumes the user
>>>>>>>>>> to be “hdfs”. Can you manually update enable-hdfs-plugin.sh and
>>>>>>>>>> rerun it?
>>>>>>>>>>
>>>>>>>>>> Line number 89 of enable-hdfs-plugin.sh
>>>>>>>>>>
>>>>>>>>>> #CFG_OWNER_INF="${HCOMPONENT_NAME}:${HCOMPONENT_NAME}”
>>>>>>>>>>
>>>>>>>>>> CFG_OWNER_INF=hduser
>>>>>>>>>>
>>>>>>>>>> And run ./enable-hdfs-plugin.sh again?
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> Bosco
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> From: Ramesh Mani <[email protected]>
>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>> [email protected]>
>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:45 PM
>>>>>>>>>>
>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>> [email protected]>
>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>
>>>>>>>>>> By looking at this error it seems that port 50070
>>>>>>>>>> <http://0.0.0.0:50070> is in use. Check that namenode is already
>>>>>>>>>> running. If so stop and start.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ramesh
>>>>>>>>>>
>>>>>>>>>> From: Aneela Saleem <[email protected]>
>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>> [email protected]>
>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:37 PM
>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>> [email protected]>
>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>
>>>>>>>>>> I'm continuously stuck at this point
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Aneela Saleem
>>>>>>>>>> On Jul 29, 2015 4:37 AM, "Aneela Saleem" <[email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> How can we check whether the problem is in Ranger's
>>>>>>>>>>> configuration or Hadoop?
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Aneela Saleem
>>>>>>>>>>> On Jul 29, 2015 2:03 AM, "Aneela Saleem" <[email protected]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Following are the Logs:
>>>>>>>>>>>>
>>>>>>>>>>>> 2015-07-29 02:01:22,126 FATAL
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start
>>>>>>>>>>>> namenode.
>>>>>>>>>>>> java.net.BindException: Port in use: 0.0.0.0:50070
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:891)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:827)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:703)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:590)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504)
>>>>>>>>>>>> Caused by: java.net.BindException: Address already in use
>>>>>>>>>>>> at sun.nio.ch.Net.bind0(Native Method)
>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:463)
>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:455)
>>>>>>>>>>>> at
>>>>>>>>>>>> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
>>>>>>>>>>>> at
>>>>>>>>>>>> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:886)
>>>>>>>>>>>> ... 8 more
>>>>>>>>>>>> 2015-07-29 02:01:22,128 INFO org.apache.hadoop.util.ExitUtil:
>>>>>>>>>>>> Exiting with status 1
>>>>>>>>>>>> 2015-07-29 02:01:22,129 INFO
>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
>>>>>>>>>>>> /************************************************************
>>>>>>>>>>>> SHUTDOWN_MSG: Shutting down NameNode at aneela-Lenovo-G50-70/
>>>>>>>>>>>> 192.168.1.4
>>>>>>>>>>>> ************************************************************/
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:53 AM, Aneela Saleem <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> hduser
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:51 AM, Don Bosco Durai <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This seems to be one of the reason HDFS plugin is not
>>>>>>>>>>>>>> enabled. As what user have you installed Hadoop?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bosco
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> From: Aneela Saleem <[email protected]>
>>>>>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>>>>>> [email protected]>
>>>>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 1:31 PM
>>>>>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>>>>>> [email protected]>
>>>>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also when i run *./enable-hdfs-plugin.sh *
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> i get the following at the end:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> + بدھ جولاي 29 01:28:20 PKT 2015 : Saving current JCE file:
>>>>>>>>>>>>>> /etc/ranger/hadoopdev/cred.jceks to
>>>>>>>>>>>>>> /etc/ranger/hadoopdev/.cred.jceks.20150729012820 ...
>>>>>>>>>>>>>> *chown: invalid user: ‘hdfs:hdfs’*
>>>>>>>>>>>>>> Ranger Plugin for hadoop has been enabled. Please restart
>>>>>>>>>>>>>> hadoop to ensure that changes are effective.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> ------------------------------
>>>
>>>
>>>
>>>
>>>
>>>
>>> NOTE: This message may contain information that is confidential,
>>> proprietary, privileged or otherwise protected by law. The message is
>>> intended solely for the named addressee. If received in error, please
>>> destroy and notify the sender. Any use of this email is prohibited when
>>> received in error. Impetus does not represent, warrant and/or guarantee,
>>> that the integrity of this communication has been maintained nor that the
>>> communication is free of errors, virus, interception or interference.
>>>
>>
>>
>
