nvm I just retried up on linux and repro'd the below. Let me take a look....
Thanks, S On Tue, Oct 3, 2017 at 8:23 AM, Stack <[email protected]> wrote: > Thank you for the detail. > > Pardon the questions below asking for yet more detail. I am unable to > reproduce locally or on another os (though we see this issue up on our > build box). > > What is your OS when you see the below? > > On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra <[email protected]> > wrote: > >> Hello, >> >> I am using "branch-2" branch of hbase, when I run unit test I get >> following >> error for netty "java.lang.UnsatisfiedLinkError: failed to load the >> required native library" >> >> >> This is running a unit test in your eclipse environment? > > You are trying to run an hbase-spark unit test when you see the above? > > > > >> *I already have following set in "maven-surefire-plugin" in pom.xml as >> per http://hbase.apache.org/book.html#thirdparty >> <http://hbase.apache.org/book.html#thirdparty>* >> >> >> > > Are you embedding hbase into your application? > > > >> <systemPropertyVariables> >> <!-- >> <test.build.classes>${test.bui >> ld.classes}</test.build.classes> >> --> >> <!--For shaded netty, to find the relocated .so. >> Trick from >> https://stackoverflow.com/questions/33825743/rename- >> files-inside-a-jar-using-some-maven-plugin >> <https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin> >> >> The netty jar has a .so in it. Shading requires rename of >> the .so and then passing a system >> property so netty finds the renamed .so and associates it >> w/ the relocated netty files. >> >> The relocated netty is in hbase-thirdparty dependency. >> Just >> set this propery globally rather >> than per module. >> --> >> <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix> >> org.apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase.shaded.io.netty. >> packagePrefix> >> </systemPropertyVariables> >> >> >> >> *And I see in the code as per HBASE-18271, all io.netty is already >> replaced >> with org.apache.hadoop.hbase.shaded.io.netty* >> >> > The trailing period is also present? > > > >> >> If I run a test from eclipse , I see the error immediately and my test >> doesn't run, but when I run from command line , the test runs but I get >> the >> error at the end when the mvn command finishes. >> >> >> Is it any eclipse test? > > Thank you. Let me try and fix this this morning. > > S > > > > > >> *Here is the complete error output.* >> >> >> >> [INFO] >> [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ hbase-spark >> --- >> [INFO] >> [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark --- >> Discovery starting. >> Discovery completed in 1 second, 558 milliseconds. >> Run starting. Expected test count is: 79 >> HBaseDStreamFunctionsSuite: >> Formatting using clusterid: testClusterID >> *** RUN ABORTED *** >> java.io.IOException: Shutting down >> at org.apache.hadoop.hbase.MiniHBaseCluster.init( >> MiniHBaseCluster.java:232) >> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( >> MiniHBaseCluster.java:94) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( >> HBaseTestingUtility.java:1124) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:1078) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:949) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:943) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:872) >> at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( >> HBaseDStreamFunctionsSuite.scala:41) >> at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll. >> scala:187) >> at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll( >> HBaseDStreamFunctionsSuite.scala:30) >> ... >> Cause: java.lang.RuntimeException: Failed construction of Master: class >> org.apache.hadoop.hbase.master.HMasterorg.apache. >> hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM >> ethods.epollin()I >> at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( >> JVMClusterUtil.java:145) >> at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster( >> LocalHBaseCluster.java:217) >> at org.apache.hadoop.hbase.LocalHBaseCluster.<init>( >> LocalHBaseCluster.java:152) >> at org.apache.hadoop.hbase.MiniHBaseCluster.init( >> MiniHBaseCluster.java:214) >> at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( >> MiniHBaseCluster.java:94) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster( >> HBaseTestingUtility.java:1124) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:1078) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:949) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:943) >> at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster( >> HBaseTestingUtility.java:872) >> ... >> Cause: java.lang.UnsatisfiedLinkError: failed to load the required >> native >> library >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> Epoll.ensureAvailability(Epoll.java:78) >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) >> at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( >> NettyEventLoopGroupConfig.java:61) >> at org.apache.hadoop.hbase.regionserver.HRegionServer.< >> init>(HRegionServer.java:552) >> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance( >> NativeConstructorAccessorImpl.java:62) >> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread( >> JVMClusterUtil.java:140) >> ... >> Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase. >> shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM >> ethods.epollin()I >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> NativeStaticallyReferencedJniMethods.epollin(Native Method) >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> Native.<clinit>(Native.java:66) >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> Epoll.<clinit>(Epoll.java:33) >> at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) >> at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>( >> NettyEventLoopGroupConfig.java:61) >> at org.apache.hadoop.hbase.regionserver.HRegionServer.< >> init>(HRegionServer.java:552) >> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance( >> NativeConstructorAccessorImpl.java:62) >> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> DelegatingConstructorAccessorImpl.java:45) >> ... >> [INFO] ------------------------------------------------------------ >> ------------ >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Apache HBase ....................................... SUCCESS [ >> 1.575 s] >> [INFO] Apache HBase - Checkstyle .......................... SUCCESS [ >> 0.317 s] >> [INFO] Apache HBase - Annotations ......................... SUCCESS [ >> 0.537 s] >> [INFO] Apache HBase - Build Configuration ................. SUCCESS [ >> 0.053 s] >> [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [ >> 15.410 s] >> [INFO] Apache HBase - Common .............................. SUCCESS [ >> 4.603 s] >> [INFO] Apache HBase - Metrics API ......................... SUCCESS [ >> 1.213 s] >> [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [ >> 0.985 s] >> [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [ >> 0.863 s] >> [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [ >> 1.750 s] >> [INFO] Apache HBase - Protocol ............................ SUCCESS [ >> 4.880 s] >> [INFO] Apache HBase - Client .............................. SUCCESS [ >> 5.233 s] >> [INFO] Apache HBase - Replication ......................... SUCCESS [ >> 1.040 s] >> [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [ >> 1.121 s] >> [INFO] Apache HBase - Procedure ........................... SUCCESS [ >> 1.084 s] >> [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [ >> 0.092 s] >> [INFO] Apache HBase - Server .............................. SUCCESS [ >> 19.849 s] >> [INFO] Apache HBase - MapReduce ........................... SUCCESS [ >> 4.221 s] >> [INFO] Apache HBase - Testing Util ........................ SUCCESS [ >> 3.273 s] >> [INFO] Apache HBase - Thrift .............................. SUCCESS [ >> 5.519 s] >> [INFO] Apache HBase - RSGroup ............................. SUCCESS [ >> 3.408 s] >> [INFO] Apache HBase - Shell ............................... SUCCESS [ >> 3.859 s] >> [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [ >> 4.038 s] >> [INFO] Apache HBase - Backup .............................. SUCCESS [01:13 >> min] >> [INFO] Apache HBase - Integration Tests ................... SUCCESS [ >> 4.229 s] >> [INFO] Apache HBase - Examples ............................ SUCCESS [ >> 3.471 s] >> [INFO] Apache HBase - Rest ................................ SUCCESS [ >> 4.448 s] >> [INFO] Apache HBase - External Block Cache ................ SUCCESS [ >> 2.040 s] >> [INFO] Apache HBase - Spark ............................... FAILURE [ >> 32.833 s] >> [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED >> [INFO] Apache HBase - Assembly ............................ SKIPPED >> [INFO] Apache HBase - Shaded .............................. SKIPPED >> [INFO] Apache HBase - Shaded - Client ..................... SKIPPED >> [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED >> [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED >> [INFO] Apache HBase - Archetypes .......................... SKIPPED >> [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED >> [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED >> [INFO] Apache HBase - Archetype builder ................... SKIPPED >> [INFO] ------------------------------------------------------------ >> ------------ >> [INFO] BUILD FAILURE >> [INFO] ------------------------------------------------------------ >> ------------ >> [INFO] Total time: 03:26 min >> [INFO] Finished at: 2017-09-27T19:34:35+05:30 >> [INFO] Final Memory: 345M/6055M >> [INFO] ------------------------------------------------------------ >> ------------ >> [ERROR] Failed to execute goal org.scalatest:scalatest-maven- >> plugin:1.0:test >> (test) on project hbase-spark: There are test failures -> [Help 1] >> [ERROR] >> [ERROR] To see the full stack trace of the errors, re-run Maven with the >> -e >> switch. >> [ERROR] Re-run Maven using the -X switch to enable full debug logging. >> [ERROR] >> [ERROR] For more information about the errors and possible solutions, >> please read the following articles: >> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ >> MojoFailureException >> [ERROR] >> [ERROR] After correcting the problems, you can resume the build with the >> command >> [ERROR] mvn <goals> -rf :hbase-spark >> > >
