Hdfs Plugin has been shown under Audits. Thanks All
On Thu, Jul 30, 2015 at 1:28 AM, Aneela Saleem <[email protected]>
wrote:
> I'm still unable to see any plugins under Audits, even after upgrading my
> HDFS to 2.7.0
>
> When i create the Service/Repo from Ranger Admin UI, i get the following
> Exception:
>
> Unable to retrieve any files using given parameters, You can still save
> the repository and start creating policies, but you would not be able to
> use autocomplete for resource names. Check *xa_portal.log* for more info.
>
> *xa_portal.log *has the following latest logs:
>
> 2015-07-30 00:46:51,977 [http-bio-6080-exec-12] INFO
> org.apache.ranger.biz.SessionMgr (SessionMgr.java:140) - Login Success:
> loginId=admin, sessionId=71, sessionId=7BF27DADE24C1A2F3A7D4D386A6273BC,
> requestId=127.0.0.1
> 2015-07-30 00:47:10,747 [http-bio-6080-exec-8] INFO
> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting
> Policy, policyName: hadoopdev-1-20150729133217
> 2015-07-30 00:47:10,905 [http-bio-6080-exec-8] INFO
> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy
> Deleted Successfully. PolicyName : hadoopdev-1-20150729133217
> 2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN
> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) -
> getFilesInDirectory('ranger-plugins/hdfs'): adding
> /usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0.jar
> 2015-07-30 00:48:03,684 [http-bio-6080-exec-8] WARN
> org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:279) -
> getFilesInDirectory('ranger-plugins/hdfs'): adding
> /usr/local/ranger-0.5.0-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hdfs/ranger-hdfs-plugin-0.5.0-tests.jar
> 2015-07-30 00:48:03,734 [timed-executor-pool-0] INFO
> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
> Login: security not enabled, using username
> 2015-07-30 00:48:38,154 [timed-executor-pool-0] INFO
> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
> Login: security not enabled, using username
> 2015-07-30 01:22:41,436 [http-bio-6080-exec-4] INFO
> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1224) - Deleting
> Policy, policyName: hadoopdev-1-20150729202236
> 2015-07-30 01:22:41,526 [http-bio-6080-exec-4] INFO
> org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:1513) - Policy
> Deleted Successfully. PolicyName : hadoopdev-1-20150729202236
> 2015-07-30 01:23:10,391 [timed-executor-pool-0] INFO
> org.apache.ranger.plugin.client.BaseClient (BaseClient.java:104) - Init
> Login: security not enabled, using username
>
> And where is *xasecure-audit.xml* file located?
>
> On Wed, Jul 29, 2015 at 10:36 PM, Don Bosco Durai <[email protected]>
> wrote:
>
>> Yes, either you have to upgrade Hadoop or use Ranger 0.4
>>
>> Thanks
>>
>> Bosco
>>
>>
>> From: Aneela Saleem <[email protected]>
>> Reply-To: "[email protected]" <
>> [email protected]>
>> Date: Wednesday, July 29, 2015 at 10:26 AM
>>
>> To: "[email protected]" <[email protected]>
>> Subject: Re: Error Setting up Ranger-Admin
>>
>> I just came to know that Ranger 0.5 is compatible with HDFS 2.7.x and
>> mine HDFS version is 2.6.0
>>
>> Is it the case? do i need to upgrade Hadoop?
>>
>> On Wed, Jul 29, 2015 at 8:09 PM, Aneela Saleem <[email protected]>
>> wrote:
>>
>>> Hi Hanish,
>>>
>>> Plugin is still not visible on Audit screen UI
>>>
>>> On Wed, Jul 29, 2015 at 7:22 PM, Hanish Bansal <
>>> [email protected]> wrote:
>>>
>>>> Hi Aneela,
>>>>
>>>>
>>>> By following below conversation thread, I am getting that you ranger
>>>> admin UI is working fine but hdfs plugin is not shown in audit logs. If I
>>>> am right then please try below steps:
>>>>
>>>>
>>>> 1. Copy all the ranger libraries files from "lib" folder to
>>>> "share/hadoop/hdfs/lib/"
>>>> cp <hadoop-home>/lib/* <hadoop-home>/share/hadoop/hdfs/lib/
>>>>
>>>>
>>>> 2. Check if you have created symbolic link for hadoop configuration
>>>> folder "etc/hadoop" as "conf". If not then create using below command:
>>>>
>>>> $ ln -s <hadoop-home>/etc/hadoop <hadoop-home>/conf
>>>>
>>>>
>>>> 3. Restart namenode service.
>>>>
>>>>
>>>> 4. Now login into some other user on machine and try listing files in
>>>> Hadoop.
>>>>
>>>>
>>>> hadoop fs -ls /
>>>>
>>>>
>>>> Note: In last step, run command using some other user who didn't start
>>>> the hadoop services.
>>>>
>>>>
>>>> Now check if plugin is visible on audit screen of UI.
>>>>
>>>>
>>>>
>>>> -------
>>>>
>>>> *Thanks & Regards, Hanish Bansal*
>>>> Software Engineer, iLabs
>>>> Impetus Infotech Pvt. Ltd.
>>>> ------------------------------
>>>> *From:* Aneela Saleem <[email protected]>
>>>> *Sent:* Wednesday, July 29, 2015 7:19 PM
>>>> *To:* [email protected]
>>>>
>>>> *Subject:* Re: Error Setting up Ranger-Admin
>>>>
>>>> When i run *./setup.sh * in ranger-admin, i get the following error:
>>>>
>>>> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>>>>
>>>> On Wed, Jul 29, 2015 at 6:46 PM, Aneela Saleem <[email protected]>
>>>> wrote:
>>>>
>>>>> Yes NameNode is running. And i don't see any Ranger-related logs in
>>>>> NameNode logs
>>>>>
>>>>> On Wed, Jul 29, 2015 at 5:41 PM, Loïc Chanel <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> At least, is your NameNode running ?
>>>>>> And do you can see any Ranger-related logs in the NameNode logs ?
>>>>>>
>>>>>> Loïc CHANEL
>>>>>> Engineering student at TELECOM Nancy
>>>>>> Trainee at Worldline - Villeurbanne
>>>>>>
>>>>>> 2015-07-29 14:37 GMT+02:00 Aneela Saleem <[email protected]>:
>>>>>>
>>>>>>> I can't figure out what is the issue? as there are no errors in log
>>>>>>> file
>>>>>>>
>>>>>>> On Wed, Jul 29, 2015 at 5:05 PM, Loïc Chanel <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> The logs for ranger plugins are in the NameNode logs, that's why
>>>>>>>> Bosco told you to take a look there.
>>>>>>>>
>>>>>>>> Loïc CHANEL
>>>>>>>> Engineering student at TELECOM Nancy
>>>>>>>> Trainee at Worldline - Villeurbanne
>>>>>>>>
>>>>>>>> 2015-07-29 12:43 GMT+02:00 Aneela Saleem <[email protected]>:
>>>>>>>>
>>>>>>>>> Hi Bosco,
>>>>>>>>>
>>>>>>>>> I tried your solution but it still does not work for me
>>>>>>>>>
>>>>>>>>> On Wed, Jul 29, 2015 at 3:22 PM, Aneela Saleem <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> How can i see logs for ranger-hdfs plugin ?
>>>>>>>>>>
>>>>>>>>>> Can you please have a look on install.properties file? is it
>>>>>>>>>> correct?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 29, 2015 at 10:57 AM, Don Bosco Durai <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, this issues seems to be HDFS namenode was not stopped. Can
>>>>>>>>>>> you check the running process by running “ps –eafww | grep
>>>>>>>>>>> proc_namenode”
>>>>>>>>>>> and if it is running you have to stop it and start it again.
>>>>>>>>>>>
>>>>>>>>>>> Regarding the chown error, it seems the scripts assumes the user
>>>>>>>>>>> to be “hdfs”. Can you manually update enable-hdfs-plugin.sh and
>>>>>>>>>>> rerun it?
>>>>>>>>>>>
>>>>>>>>>>> Line number 89 of enable-hdfs-plugin.sh
>>>>>>>>>>>
>>>>>>>>>>> #CFG_OWNER_INF="${HCOMPONENT_NAME}:${HCOMPONENT_NAME}”
>>>>>>>>>>>
>>>>>>>>>>> CFG_OWNER_INF=hduser
>>>>>>>>>>>
>>>>>>>>>>> And run ./enable-hdfs-plugin.sh again?
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>> Bosco
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> From: Ramesh Mani <[email protected]>
>>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>>> [email protected]>
>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:45 PM
>>>>>>>>>>>
>>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>>> [email protected]>
>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>>
>>>>>>>>>>> By looking at this error it seems that port 50070
>>>>>>>>>>> <http://0.0.0.0:50070> is in use. Check that namenode is
>>>>>>>>>>> already running. If so stop and start.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ramesh
>>>>>>>>>>>
>>>>>>>>>>> From: Aneela Saleem <[email protected]>
>>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>>> [email protected]>
>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 4:37 PM
>>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>>> [email protected]>
>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>>
>>>>>>>>>>> I'm continuously stuck at this point
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Aneela Saleem
>>>>>>>>>>> On Jul 29, 2015 4:37 AM, "Aneela Saleem" <[email protected]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> How can we check whether the problem is in Ranger's
>>>>>>>>>>>> configuration or Hadoop?
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Aneela Saleem
>>>>>>>>>>>> On Jul 29, 2015 2:03 AM, "Aneela Saleem" <
>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Following are the Logs:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2015-07-29 02:01:22,126 FATAL
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start
>>>>>>>>>>>>> namenode.
>>>>>>>>>>>>> java.net.BindException: Port in use: 0.0.0.0:50070
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:891)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:827)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:703)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:590)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504)
>>>>>>>>>>>>> Caused by: java.net.BindException: Address already in use
>>>>>>>>>>>>> at sun.nio.ch.Net.bind0(Native Method)
>>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:463)
>>>>>>>>>>>>> at sun.nio.ch.Net.bind(Net.java:455)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:886)
>>>>>>>>>>>>> ... 8 more
>>>>>>>>>>>>> 2015-07-29 02:01:22,128 INFO org.apache.hadoop.util.ExitUtil:
>>>>>>>>>>>>> Exiting with status 1
>>>>>>>>>>>>> 2015-07-29 02:01:22,129 INFO
>>>>>>>>>>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
>>>>>>>>>>>>> /************************************************************
>>>>>>>>>>>>> SHUTDOWN_MSG: Shutting down NameNode at aneela-Lenovo-G50-70/
>>>>>>>>>>>>> 192.168.1.4
>>>>>>>>>>>>> ************************************************************/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:53 AM, Aneela Saleem <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> hduser
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jul 29, 2015 at 1:51 AM, Don Bosco Durai <
>>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This seems to be one of the reason HDFS plugin is not
>>>>>>>>>>>>>>> enabled. As what user have you installed Hadoop?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bosco
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> From: Aneela Saleem <[email protected]>
>>>>>>>>>>>>>>> Reply-To: "[email protected]" <
>>>>>>>>>>>>>>> [email protected]>
>>>>>>>>>>>>>>> Date: Tuesday, July 28, 2015 at 1:31 PM
>>>>>>>>>>>>>>> To: "[email protected]" <
>>>>>>>>>>>>>>> [email protected]>
>>>>>>>>>>>>>>> Subject: Re: Error Setting up Ranger-Admin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also when i run *./enable-hdfs-plugin.sh *
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> i get the following at the end:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> + بدھ جولاي 29 01:28:20 PKT 2015 : Saving current JCE file:
>>>>>>>>>>>>>>> /etc/ranger/hadoopdev/cred.jceks to
>>>>>>>>>>>>>>> /etc/ranger/hadoopdev/.cred.jceks.20150729012820 ...
>>>>>>>>>>>>>>> *chown: invalid user: ‘hdfs:hdfs’*
>>>>>>>>>>>>>>> Ranger Plugin for hadoop has been enabled. Please restart
>>>>>>>>>>>>>>> hadoop to ensure that changes are effective.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> NOTE: This message may contain information that is confidential,
>>>> proprietary, privileged or otherwise protected by law. The message is
>>>> intended solely for the named addressee. If received in error, please
>>>> destroy and notify the sender. Any use of this email is prohibited when
>>>> received in error. Impetus does not represent, warrant and/or guarantee,
>>>> that the integrity of this communication has been maintained nor that the
>>>> communication is free of errors, virus, interception or interference.
>>>>
>>>
>>>
>>
>
