-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57084/
-----------------------------------------------------------
Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O
hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan
Neethiraj, and Velmurugan Periasamy.
Bugs: RANGER-1412
https://issues.apache.org/jira/browse/RANGER-1412
Repository: ranger
Description
-------
Start hadoop failed after enabling ranger HDFS plugins. The error information
is as following:
2017-02-27 02:34:58,885 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:673)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:585)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:645)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)
...............
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:383)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
2017-02-27 02:34:58,922 INFO org.apache.hadoop.util.ExitUtil: Exiting with
status 1
The cause of the issue is that the enable-hdfs-plugin.sh was done as following.
1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-impl
2. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
3. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
lrwxrwxrwx 1 root root 72 Feb 27 02:33 ranger-hdfs-plugin-impl ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl/
lrwxrwxrwx 1 root root 91 Feb 27 02:33
ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
lrwxrwxrwx 1 root root 93 Feb 27 02:33
ranger-plugin-classloader-1.0.0-SNAPSHOT.jar ->
/usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
The above link are error. The hadoop can not find dependent packages when
starting. They should link as following.
1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl
2. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar
3. Link
ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
to
/var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar
The program install patch is as following:
1. Install pseudo distributions hadoop. It's path is
/var/local/hadoop/hadoop-2.7.3.
2. ranger hdfs plugin path is /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin.
Diffs
-----
agents-common/scripts/enable-agent.sh b8b79a9
Diff: https://reviews.apache.org/r/57084/diff/
Testing
-------
Thanks,
Qiang Zhang
