----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57084/#review168462 -----------------------------------------------------------
Ship it! Ship It! - Colm O hEigeartaigh On Feb. 28, 2017, 8:37 a.m., Qiang Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57084/ > ----------------------------------------------------------- > > (Updated Feb. 28, 2017, 8:37 a.m.) > > > Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan > Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-1412 > https://issues.apache.org/jira/browse/RANGER-1412 > > > Repository: ranger > > > Description > ------- > > Start hadoop failed after enabling ranger HDFS plugins. The error information > is as following: > 2017-02-27 02:34:58,885 ERROR > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem > initialization failed. > java.lang.RuntimeException: java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:673) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:585) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:645) > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812) > at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493) > at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559) > ............... > at > org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64) > at > org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43) > at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at java.lang.Class.newInstance(Class.java:383) > at > org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64) > at > org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43) > at sun.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source) > 2017-02-27 02:34:58,922 INFO org.apache.hadoop.util.ExitUtil: Exiting with > status 1 > > The cause of the issue is that the enable-hdfs-plugin.sh was done as > following. > 1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to > /var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-impl > 2. Link > ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar > to > /var/local/hadoop/hadoop-2.7.3/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar > 3. Link > ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar > to > /var/local/hadoop/hadoop-2.7.3/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar > > lrwxrwxrwx 1 root root 72 Feb 27 02:33 ranger-hdfs-plugin-impl -> > /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl/ > lrwxrwxrwx 1 root root 91 Feb 27 02:33 > ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar -> > /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar > lrwxrwxrwx 1 root root 93 Feb 27 02:33 > ranger-plugin-classloader-1.0.0-SNAPSHOT.jar -> > /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar > > The above link are error. The hadoop can not find dependent packages when > starting. They should link as following. > 1. Link ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-impl to > /var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-impl > 2. Link > ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar > to > /var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.0.0-SNAPSHOT.jar > 3. Link > ranger-1.0.0-SNAPSHOT-hdfs-plugin/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar > to > /var/local/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.0.0-SNAPSHOT.jar > > The program install path is as following: > 1. Install pseudo distributions hadoop. It's path is > /var/local/hadoop/hadoop-2.7.3. > 2. ranger hdfs plugin path is /usr/local/ranger-1.0.0-SNAPSHOT-hdfs-plugin. > > Test and verify: > I carefully tested and verified the patch before commit the issue. > > > Diffs > ----- > > agents-common/scripts/enable-agent.sh b8b79a9 > > > Diff: https://reviews.apache.org/r/57084/diff/1/ > > > Testing > ------- > > > Thanks, > > Qiang Zhang > >
