Bosco, For now I don’t see any change, what I see in Hafiz manual installation is that classpath hadoop/share/hdfs/lib is the one where the hadoop is looking for.
Action item here would be do a manual install using apache bits on a raw hadoop cluster and test it. Internally, I tested it with Sandbox with manual installation and it works fine. Regards, Ramesh From: Don Bosco Durai <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 11:27 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue Great. Thanks. Can we also update the JIRA with the findings and workarounds. Ramesh, do we need to do any changes from our side? Thanks Bosco From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 10:47 AM To: <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue yes it worked, name node is started On Mon, Nov 30, 2015 at 11:41 PM, Ramesh Mani <[email protected]<mailto:[email protected]>> wrote: Hafiz! Please try placing it in hadoop/share/hdfs/lib and try again. Thanks, Ramesh From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 10:21 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue Ramesh! I tried by copying ranger-hdfs-plugin-impl to hadoop/lib but still same issue On Mon, Nov 30, 2015 at 10:58 PM, Hafiz Mujadid <[email protected]<mailto:[email protected]>> wrote: Yes I am using manual install On Mon, Nov 30, 2015 at 10:56 PM, Don Bosco Durai <[email protected]<mailto:[email protected]>> wrote: Ramesh, Hafiz might be using manual install. Have we tried that with the recent code? We selectively pick jars to be packaged in the components. Thanks Bosco From: Ramesh Mani <[email protected]<mailto:[email protected]>> Reply-To: <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 9:53 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue Hafiz, Under lib directory there is a folder ranger-hdfs-plugin-impl, please copy this folder to hadoop/lib and try? Regards, Ramesh From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 8:44 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue Following are the contents Ramesh! 1. isable-hdfs-plugin.sh 2. install 3. lib 4. upgrade-hdfs-plugin.sh 5. enable-hdfs-plugin.sh 6. install.properties 7. ranger_credential_helper.py 8. upgrade-plugin.py On Mon, Nov 30, 2015 at 9:39 PM, Hafiz Mujadid <[email protected]<mailto:[email protected]>> wrote: in jira i have attached install.properties, which files else you need? On Mon, Nov 30, 2015 at 9:37 PM, Ramesh Mani <[email protected]<mailto:[email protected]>> wrote: Hafiz, Can I get the details on the files where you untarred the plugging tar, it should be like ranger-hdfs-plugin? This issue seems to be that installation is not completely correct. Regards, Ramesh From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Monday, November 30, 2015 at 5:21 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue Ramesh, when i removed the following jars from the hadoop/share/hdfs/lib and try enabling ranger-hdfs-plugin-shim-0.5.0.jar ranger-plugin-classloader-0.5.0.jar I faced following exception java.lang.ClassNotFoundException: Class org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer On Mon, Nov 30, 2015 at 10:16 AM, Ramesh Mani <[email protected]<mailto:[email protected]>> wrote: Hafiz, Can you remove the following jars from the hadoop/share/hdfs/lib and try enabling? ranger-hdfs-plugin-shim-0.5.0.jar ranger-plugin-classloader-0.5.0.jar Regards, Ramesh From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Sunday, November 29, 2015 at 1:52 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: hdfs plugin enable issue hadoop/lib contains following jar files, I also placed these ranger related jars in hadoop/share/hdfs/lib folder ranger-hdfs-plugin-shim-0.5.0.jar ranger-plugin-classloader-0.5.0.jar native folder with following files libhadoop.a libhadooppipes.a libhadoop.so libhadoop.so.1.0.0 libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0 attached is the complete log file thanks On Sun, Nov 29, 2015 at 11:37 AM, Ramesh Mani <[email protected]<mailto:[email protected]>> wrote: Hafiz, Please list the files in hadoop/lib directory , also provide the complete namenode log for me to review? Thanks, Rame From: Hafiz Mujadid <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Saturday, November 28, 2015 at 11:51 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: hdfs plugin enable issue Hi! I have enabled hdfs plugin but after that name node is not starting, when i start namenode, i get following exception 2015-11-29 00:44:20,534 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:673) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:584) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:644) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132) ... 8 more Caused by: java.lang.StackOverflowError at java.lang.Exception.<init>(Exception.java:102) at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89) at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72) at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:383) at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:65) -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID -- Regards: HAFIZ MUJADID
