Hello, I am using "branch-2" branch of hbase, when I run unit test I get following error for netty "java.lang.UnsatisfiedLinkError: failed to load the required native library"
*I already have following set in "maven-surefire-plugin" in pom.xml as per http://hbase.apache.org/book.html#thirdparty <http://hbase.apache.org/book.html#thirdparty>* <systemPropertyVariables> <!-- <test.build.classes>${test.build.classes}</test.build.classes> --> <!--For shaded netty, to find the relocated .so. Trick from https://stackoverflow.com/questions/33825743/rename- files-inside-a-jar-using-some-maven-plugin The netty jar has a .so in it. Shading requires rename of the .so and then passing a system property so netty finds the renamed .so and associates it w/ the relocated netty files. The relocated netty is in hbase-thirdparty dependency. Just set this propery globally rather than per module. --> <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix> org.apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase.shaded.io.netty. packagePrefix> </systemPropertyVariables> *And I see in the code as per HBASE-18271, all io.netty is already replaced with org.apache.hadoop.hbase.shaded.io.netty* If I run a test from eclipse , I see the error immediately and my test doesn't run, but when I run from command line , the test runs but I get the error at the end when the mvn command finishes. *Here is the complete error output.* [INFO] [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ hbase-spark --- [INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark --- Discovery starting. Discovery completed in 1 second, 558 milliseconds. Run starting. Expected test count is: 79 HBaseDStreamFunctionsSuite: Formatting using clusterid: testClusterID *** RUN ABORTED *** java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init( MiniHBaseCluster.java:232) at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( MiniHBaseCluster.java:94) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( HBaseTestingUtility.java:1124) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:1078) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:949) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:943) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:872) at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( HBaseDStreamFunctionsSuite.scala:41) at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll. scala:187) at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( HBaseDStreamFunctionsSuite.scala:30) ... Cause: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterorg.apache. hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM ethods.epollin()I at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( JVMClusterUtil.java:145) at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster( LocalHBaseCluster.java:217) at org.apache.hadoop.hbase.LocalHBaseCluster.<init>( LocalHBaseCluster.java:152) at org.apache.hadoop.hbase.MiniHBaseCluster.init( MiniHBaseCluster.java:214) at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( MiniHBaseCluster.java:94) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( HBaseTestingUtility.java:1124) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:1078) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:949) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:943) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( HBaseTestingUtility.java:872) ... Cause: java.lang.UnsatisfiedLinkError: failed to load the required native library at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. Epoll.ensureAvailability(Epoll.java:78) at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( NettyEventLoopGroupConfig.java:61) at org.apache.hadoop.hbase.regionserver.HRegionServer.< init>(HRegionServer.java:552) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( JVMClusterUtil.java:140) ... Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase. shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM ethods.epollin()I at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. NativeStaticallyReferencedJniMethods.epollin(Native Method) at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. Native.<clinit>(Native.java:66) at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. Epoll.<clinit>(Epoll.java:33) at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( NettyEventLoopGroupConfig.java:61) at org.apache.hadoop.hbase.regionserver.HRegionServer.< init>(HRegionServer.java:552) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java:45) ... [INFO] ------------------------------------------------------------ ------------ [INFO] Reactor Summary: [INFO] [INFO] Apache HBase ....................................... SUCCESS [ 1.575 s] [INFO] Apache HBase - Checkstyle .......................... SUCCESS [ 0.317 s] [INFO] Apache HBase - Annotations ......................... SUCCESS [ 0.537 s] [INFO] Apache HBase - Build Configuration ................. SUCCESS [ 0.053 s] [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [ 15.410 s] [INFO] Apache HBase - Common .............................. SUCCESS [ 4.603 s] [INFO] Apache HBase - Metrics API ......................... SUCCESS [ 1.213 s] [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [ 0.985 s] [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [ 0.863 s] [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [ 1.750 s] [INFO] Apache HBase - Protocol ............................ SUCCESS [ 4.880 s] [INFO] Apache HBase - Client .............................. SUCCESS [ 5.233 s] [INFO] Apache HBase - Replication ......................... SUCCESS [ 1.040 s] [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [ 1.121 s] [INFO] Apache HBase - Procedure ........................... SUCCESS [ 1.084 s] [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [ 0.092 s] [INFO] Apache HBase - Server .............................. SUCCESS [ 19.849 s] [INFO] Apache HBase - MapReduce ........................... SUCCESS [ 4.221 s] [INFO] Apache HBase - Testing Util ........................ SUCCESS [ 3.273 s] [INFO] Apache HBase - Thrift .............................. SUCCESS [ 5.519 s] [INFO] Apache HBase - RSGroup ............................. SUCCESS [ 3.408 s] [INFO] Apache HBase - Shell ............................... SUCCESS [ 3.859 s] [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [ 4.038 s] [INFO] Apache HBase - Backup .............................. SUCCESS [01:13 min] [INFO] Apache HBase - Integration Tests ................... SUCCESS [ 4.229 s] [INFO] Apache HBase - Examples ............................ SUCCESS [ 3.471 s] [INFO] Apache HBase - Rest ................................ SUCCESS [ 4.448 s] [INFO] Apache HBase - External Block Cache ................ SUCCESS [ 2.040 s] [INFO] Apache HBase - Spark ............................... FAILURE [ 32.833 s] [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED [INFO] Apache HBase - Assembly ............................ SKIPPED [INFO] Apache HBase - Shaded .............................. SKIPPED [INFO] Apache HBase - Shaded - Client ..................... SKIPPED [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED [INFO] Apache HBase - Archetypes .......................... SKIPPED [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED [INFO] Apache HBase - Archetype builder ................... SKIPPED [INFO] ------------------------------------------------------------ ------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------ ------------ [INFO] Total time: 03:26 min [INFO] Finished at: 2017-09-27T19:34:35+05:30 [INFO] Final Memory: 345M/6055M [INFO] ------------------------------------------------------------ ------------ [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project hbase-spark: There are test failures -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hbase-spark
