Qiang Zhang created RANGER-1412:
-----------------------------------
Summary: Start hadoop failed after enabling ranger HDFS plugins
Key: RANGER-1412
URL: https://issues.apache.org/jira/browse/RANGER-1412
Project: Ranger
Issue Type: Bug
Components: plugins
Affects Versions: 1.0.0
Reporter: Qiang Zhang
Assignee: Qiang Zhang
Priority: Critical
Start hadoop failed after enabling ranger HDFS plugins. The error information
is as following:
2017-02-27 02:34:58,885 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:673)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:585)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:645)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)
...............
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:383)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
2017-02-27 02:34:58,922 INFO org.apache.hadoop.util.ExitUtil: Exiting with
status 1
The cause of the issue is that the enable-hdfs-plugin.sh was done as following.
1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-impl
2. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
3. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
lrwxrwxrwx 1 root root 72 Feb 27 02:33 ranger-hdfs-plugin-impl ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl/
lrwxrwxrwx 1 root root 91 Feb 27 02:33
ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
lrwxrwxrwx 1 root root 93 Feb 27 02:33
ranger-plugin-classloader-1.0.0-SNAPSHOT.jar ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
The above link are error. The hadoop can not find dependent packages when
starting. They should link as following.
1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl
2. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
3. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
The program install patch is as following:
1. Install pseudo distributions hadoop. It's path is
/var/local/hadoop/hadoop-2.7.3.
2. ranger hdfs plugin path is /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
