Thanks Stack / Sean Busbey for replying. OS : Ubuntu 16.04.2 , 64 bit. Eclipse : Version: Neon.3 Release (4.6.3) HBase branch : branch-2 Command line test to reproduce : mvn clean package -Dtest=TestIncrementalBackup Reproduce from eclipse , right click on TestIncBackupRestore and run as junit from test class TestIncrementalBackup. No I am not embedding hbase in my application. I have just checked out hbase , switched to branch-2 and run the unit test from command line or from eclipse. Failing with same error in both cases. Yes the trailing period is also present.
Thanks, Amit Kabra. On Tue, Oct 3, 2017 at 8:53 PM, Stack <[email protected]> wrote: > Thank you for the detail. > > Pardon the questions below asking for yet more detail. I am unable to > reproduce locally or on another os (though we see this issue up on our > build box). > > What is your OS when you see the below? > > On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra <[email protected]> > wrote: > > > Hello, > > > > I am using "branch-2" branch of hbase, when I run unit test I get > following > > error for netty "java.lang.UnsatisfiedLinkError: failed to load the > > required native library" > > > > > > This is running a unit test in your eclipse environment? > > You are trying to run an hbase-spark unit test when you see the above? > > > > > > *I already have following set in "maven-surefire-plugin" in pom.xml as > > per http://hbase.apache.org/book.html#thirdparty > > <http://hbase.apache.org/book.html#thirdparty>* > > > > > > > > Are you embedding hbase into your application? > > > > > <systemPropertyVariables> > > <!-- > > <test.build.classes>${test.build.classes}</test.build. > classe > > s> > > --> > > <!--For shaded netty, to find the relocated .so. > > Trick from > > https://stackoverflow.com/questions/33825743/rename- > > files-inside-a-jar-using-some-maven-plugin > > <https://stackoverflow.com/questions/33825743/rename- > files-inside-a-jar-using-some-maven-plugin> > > > > The netty jar has a .so in it. Shading requires rename of > > the .so and then passing a system > > property so netty finds the renamed .so and associates it > > w/ the relocated netty files. > > > > The relocated netty is in hbase-thirdparty dependency. > Just > > set this propery globally rather > > than per module. > > --> > > <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix> > > org.apache.hadoop.hbase.shaded.</org.apache.hadoop. > hbase.shaded.io.netty. > > packagePrefix> > > </systemPropertyVariables> > > > > > > > > *And I see in the code as per HBASE-18271, all io.netty is already > replaced > > with org.apache.hadoop.hbase.shaded.io.netty* > > > > > The trailing period is also present? > > > > > > > If I run a test from eclipse , I see the error immediately and my test > > doesn't run, but when I run from command line , the test runs but I get > the > > error at the end when the mvn command finishes. > > > > > > Is it any eclipse test? > > Thank you. Let me try and fix this this morning. > > S > > > > > > > *Here is the complete error output.* > > > > > > > > [INFO] > > [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ > hbase-spark > > --- > > [INFO] > > [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark --- > > Discovery starting. > > Discovery completed in 1 second, 558 milliseconds. > > Run starting. Expected test count is: 79 > > HBaseDStreamFunctionsSuite: > > Formatting using clusterid: testClusterID > > *** RUN ABORTED *** > > java.io.IOException: Shutting down > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( > > MiniHBaseCluster.java:232) > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( > > MiniHBaseCluster.java:94) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( > > HBaseTestingUtility.java:1124) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:1078) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:949) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:943) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:872) > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( > > HBaseDStreamFunctionsSuite.scala:41) > > at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll. > > scala:187) > > at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( > > HBaseDStreamFunctionsSuite.scala:30) > > ... > > Cause: java.lang.RuntimeException: Failed construction of Master: class > > org.apache.hadoop.hbase.master.HMasterorg.apache. > > hadoop.hbase.shaded.io.netty.channel.epoll. > NativeStaticallyReferencedJniM > > ethods.epollin()I > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( > > JVMClusterUtil.java:145) > > at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster( > > LocalHBaseCluster.java:217) > > at org.apache.hadoop.hbase.LocalHBaseCluster.<init>( > > LocalHBaseCluster.java:152) > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( > > MiniHBaseCluster.java:214) > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( > > MiniHBaseCluster.java:94) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( > > HBaseTestingUtility.java:1124) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:1078) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:949) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:943) > > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( > > HBaseTestingUtility.java:872) > > ... > > Cause: java.lang.UnsatisfiedLinkError: failed to load the required > > native > > library > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > Epoll.ensureAvailability(Epoll.java:78) > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( > > NettyEventLoopGroupConfig.java:61) > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< > > init>(HRegionServer.java:552) > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > > NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > > DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > > at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( > > JVMClusterUtil.java:140) > > ... > > Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase. > > shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM > > ethods.epollin()I > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > NativeStaticallyReferencedJniMethods.epollin(Native Method) > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > Native.<clinit>(Native.java:66) > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > Epoll.<clinit>(Epoll.java:33) > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) > > at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( > > NettyEventLoopGroupConfig.java:61) > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< > > init>(HRegionServer.java:552) > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > > NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > > DelegatingConstructorAccessorImpl.java:45) > > ... > > [INFO] ------------------------------------------------------------ > > ------------ > > [INFO] Reactor Summary: > > [INFO] > > [INFO] Apache HBase ....................................... SUCCESS [ > > 1.575 s] > > [INFO] Apache HBase - Checkstyle .......................... SUCCESS [ > > 0.317 s] > > [INFO] Apache HBase - Annotations ......................... SUCCESS [ > > 0.537 s] > > [INFO] Apache HBase - Build Configuration ................. SUCCESS [ > > 0.053 s] > > [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [ > > 15.410 s] > > [INFO] Apache HBase - Common .............................. SUCCESS [ > > 4.603 s] > > [INFO] Apache HBase - Metrics API ......................... SUCCESS [ > > 1.213 s] > > [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [ > > 0.985 s] > > [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [ > > 0.863 s] > > [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [ > > 1.750 s] > > [INFO] Apache HBase - Protocol ............................ SUCCESS [ > > 4.880 s] > > [INFO] Apache HBase - Client .............................. SUCCESS [ > > 5.233 s] > > [INFO] Apache HBase - Replication ......................... SUCCESS [ > > 1.040 s] > > [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [ > > 1.121 s] > > [INFO] Apache HBase - Procedure ........................... SUCCESS [ > > 1.084 s] > > [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [ > > 0.092 s] > > [INFO] Apache HBase - Server .............................. SUCCESS [ > > 19.849 s] > > [INFO] Apache HBase - MapReduce ........................... SUCCESS [ > > 4.221 s] > > [INFO] Apache HBase - Testing Util ........................ SUCCESS [ > > 3.273 s] > > [INFO] Apache HBase - Thrift .............................. SUCCESS [ > > 5.519 s] > > [INFO] Apache HBase - RSGroup ............................. SUCCESS [ > > 3.408 s] > > [INFO] Apache HBase - Shell ............................... SUCCESS [ > > 3.859 s] > > [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [ > > 4.038 s] > > [INFO] Apache HBase - Backup .............................. SUCCESS > [01:13 > > min] > > [INFO] Apache HBase - Integration Tests ................... SUCCESS [ > > 4.229 s] > > [INFO] Apache HBase - Examples ............................ SUCCESS [ > > 3.471 s] > > [INFO] Apache HBase - Rest ................................ SUCCESS [ > > 4.448 s] > > [INFO] Apache HBase - External Block Cache ................ SUCCESS [ > > 2.040 s] > > [INFO] Apache HBase - Spark ............................... FAILURE [ > > 32.833 s] > > [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED > > [INFO] Apache HBase - Assembly ............................ SKIPPED > > [INFO] Apache HBase - Shaded .............................. SKIPPED > > [INFO] Apache HBase - Shaded - Client ..................... SKIPPED > > [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED > > [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED > > [INFO] Apache HBase - Archetypes .......................... SKIPPED > > [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED > > [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED > > [INFO] Apache HBase - Archetype builder ................... SKIPPED > > [INFO] ------------------------------------------------------------ > > ------------ > > [INFO] BUILD FAILURE > > [INFO] ------------------------------------------------------------ > > ------------ > > [INFO] Total time: 03:26 min > > [INFO] Finished at: 2017-09-27T19:34:35+05:30 > > [INFO] Final Memory: 345M/6055M > > [INFO] ------------------------------------------------------------ > > ------------ > > [ERROR] Failed to execute goal org.scalatest:scalatest-maven- > > plugin:1.0:test > > (test) on project hbase-spark: There are test failures -> [Help 1] > > [ERROR] > > [ERROR] To see the full stack trace of the errors, re-run Maven with the > -e > > switch. > > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > > [ERROR] > > [ERROR] For more information about the errors and possible solutions, > > please read the following articles: > > [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ > > MojoFailureException > > [ERROR] > > [ERROR] After correcting the problems, you can resume the build with the > > command > > [ERROR] mvn <goals> -rf :hbase-spark > > >
