It just came in... Looks better on jenkins than my linux machine (4m total): https://issues.apache.org/jira/browse/HBASE-18606?focusedCommentId=16190154&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16190154
Let me try pushing it. St.Ack On Tue, Oct 3, 2017 at 12:00 PM, Stack <[email protected]> wrote: > Yeah (12x the processors, 8x RAM, 12x the disks...). Lets see how patch > does one jenkins... (HBASE-18606). > S > > On Tue, Oct 3, 2017 at 11:48 AM, Sean Busbey <[email protected]> wrote: > >> the linux box roughly as capable as your laptop? >> >> On Tue, Oct 3, 2017 at 12:51 PM, Stack <[email protected]> wrote: >> > Tests passed eventually for me: >> > >> > real 47m6.498s >> > user 5m29.671s >> > sys 0m41.885s >> > >> > ... which is a big diff from macosx run. >> > >> > Need to look into this. >> > >> > St.Ack >> > >> > >> > On Tue, Oct 3, 2017 at 10:03 AM, Stack <[email protected]> wrote: >> > >> >> The below gets us further but now I see that the spark tests take a >> really >> >> long time to run on linux but complete promptly on macosx (2m 55s). >> >> Looking.... >> >> >> >> St.Ack >> >> >> >> On Tue, Oct 3, 2017 at 9:13 AM, Stack <[email protected]> wrote: >> >> >> >>> This seems to work for me. Does it work for you? >> >>> >> >>> >> >>> diff --git a/hbase-spark/pom.xml b/hbase-spark/pom.xml >> >>> index 594aa2a..6d191e3 100644 >> >>> --- a/hbase-spark/pom.xml >> >>> +++ b/hbase-spark/pom.xml >> >>> @@ -568,6 +568,9 @@ >> >>> <junitxml>.</junitxml> >> >>> <filereports>WDF TestSuite.txt</filereports> >> >>> <parallel>false</parallel> >> >>> + <systemProperties> >> >>> + <org.apache.hadoop.hbase.shade >> d.io.netty.packagePrefix>org. >> >>> apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase. >> >>> shaded.io.netty.packagePrefix> >> >>> + </systemProperties> >> >>> </configuration> >> >>> <executions> >> >>> <execution> >> >>> >> >>> St.Ack >> >>> >> >>> On Tue, Oct 3, 2017 at 8:45 AM, Amit Kabra <[email protected]> >> >>> wrote: >> >>> >> >>>> Thanks Stack / Sean Busbey for replying. >> >>>> >> >>>> OS : Ubuntu 16.04.2 , 64 bit. >> >>>> Eclipse : Version: Neon.3 Release (4.6.3) >> >>>> HBase branch : branch-2 >> >>>> Command line test to reproduce : mvn clean package >> >>>> -Dtest=TestIncrementalBackup >> >>>> Reproduce from eclipse , right click on TestIncBackupRestore and run >> as >> >>>> junit from test class TestIncrementalBackup. >> >>>> No I am not embedding hbase in my application. I have just checked >> out >> >>>> hbase , switched to branch-2 and run the unit test from command line >> or >> >>>> from eclipse. Failing with same error in both cases. >> >>>> Yes the trailing period is also present. >> >>>> >> >>>> Thanks, >> >>>> Amit Kabra. >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> On Tue, Oct 3, 2017 at 8:53 PM, Stack <[email protected]> wrote: >> >>>> >> >>>> > Thank you for the detail. >> >>>> > >> >>>> > Pardon the questions below asking for yet more detail. I am unable >> to >> >>>> > reproduce locally or on another os (though we see this issue up on >> our >> >>>> > build box). >> >>>> > >> >>>> > What is your OS when you see the below? >> >>>> > >> >>>> > On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra < >> [email protected]> >> >>>> > wrote: >> >>>> > >> >>>> > > Hello, >> >>>> > > >> >>>> > > I am using "branch-2" branch of hbase, when I run unit test I get >> >>>> > following >> >>>> > > error for netty "java.lang.UnsatisfiedLinkError: failed to load >> the >> >>>> > > required native library" >> >>>> > > >> >>>> > > >> >>>> > > This is running a unit test in your eclipse environment? >> >>>> > >> >>>> > You are trying to run an hbase-spark unit test when you see the >> above? >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > > *I already have following set in "maven-surefire-plugin" in >> pom.xml >> >>>> as >> >>>> > > per http://hbase.apache.org/book.html#thirdparty >> >>>> > > <http://hbase.apache.org/book.html#thirdparty>* >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > >> >>>> > Are you embedding hbase into your application? >> >>>> > >> >>>> > >> >>>> > >> >>>> > > <systemPropertyVariables> >> >>>> > > <!-- >> >>>> > > <test.build.classes>${test.bu >> ild.classes}</test.build. >> >>>> > classe >> >>>> > > s> >> >>>> > > --> >> >>>> > > <!--For shaded netty, to find the relocated .so. >> >>>> > > Trick from >> >>>> > > https://stackoverflow.com/que >> stions/33825743/rename- >> >>>> > > files-inside-a-jar-using-some-maven-plugin >> >>>> > > <https://stackoverflow.com/questions/33825743/rename- >> >>>> > files-inside-a-jar-using-some-maven-plugin> >> >>>> > > >> >>>> > > The netty jar has a .so in it. Shading requires >> >>>> rename of >> >>>> > > the .so and then passing a system >> >>>> > > property so netty finds the renamed .so and >> >>>> associates it >> >>>> > > w/ the relocated netty files. >> >>>> > > >> >>>> > > The relocated netty is in hbase-thirdparty >> >>>> dependency. >> >>>> > Just >> >>>> > > set this propery globally rather >> >>>> > > than per module. >> >>>> > > --> >> >>>> > > <org.apache.hadoop.hbase.shad >> >>>> ed.io.netty.packagePrefix> >> >>>> > > org.apache.hadoop.hbase.shaded.</org.apache.hadoop. >> >>>> > hbase.shaded.io.netty. >> >>>> > > packagePrefix> >> >>>> > > </systemPropertyVariables> >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > *And I see in the code as per HBASE-18271, all io.netty is >> already >> >>>> > replaced >> >>>> > > with org.apache.hadoop.hbase.shaded.io.netty* >> >>>> > > >> >>>> > > >> >>>> > The trailing period is also present? >> >>>> > >> >>>> > >> >>>> > >> >>>> > > >> >>>> > > If I run a test from eclipse , I see the error immediately and my >> >>>> test >> >>>> > > doesn't run, but when I run from command line , the test runs >> but I >> >>>> get >> >>>> > the >> >>>> > > error at the end when the mvn command finishes. >> >>>> > > >> >>>> > > >> >>>> > > Is it any eclipse test? >> >>>> > >> >>>> > Thank you. Let me try and fix this this morning. >> >>>> > >> >>>> > S >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > > *Here is the complete error output.* >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > [INFO] >> >>>> > > [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ >> >>>> > hbase-spark >> >>>> > > --- >> >>>> > > [INFO] >> >>>> > > [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark >> --- >> >>>> > > Discovery starting. >> >>>> > > Discovery completed in 1 second, 558 milliseconds. >> >>>> > > Run starting. Expected test count is: 79 >> >>>> > > HBaseDStreamFunctionsSuite: >> >>>> > > Formatting using clusterid: testClusterID >> >>>> > > *** RUN ABORTED *** >> >>>> > > java.io.IOException: Shutting down >> >>>> > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( >> >>>> > > MiniHBaseCluster.java:232) >> >>>> > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( >> >>>> > > MiniHBaseCluster.java:94) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniHBaseCl >> >>>> uster( >> >>>> > > HBaseTestingUtility.java:1124) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:1078) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:949) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:943) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:872) >> >>>> > > at org.apache.hadoop.hbase.spark. >> HBaseDStreamFunctionsSuite.bef >> >>>> oreAll( >> >>>> > > HBaseDStreamFunctionsSuite.scala:41) >> >>>> > > at org.scalatest.BeforeAndAfterAl >> l$class.beforeAll(BeforeAndAft >> >>>> erAll. >> >>>> > > scala:187) >> >>>> > > at org.apache.hadoop.hbase.spark. >> HBaseDStreamFunctionsSuite.bef >> >>>> oreAll( >> >>>> > > HBaseDStreamFunctionsSuite.scala:30) >> >>>> > > ... >> >>>> > > Cause: java.lang.RuntimeException: Failed construction of >> Master: >> >>>> class >> >>>> > > org.apache.hadoop.hbase.master.HMasterorg.apache. >> >>>> > > hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > NativeStaticallyReferencedJniM >> >>>> > > ethods.epollin()I >> >>>> > > at org.apache.hadoop.hbase.util.J >> VMClusterUtil.createMasterThread( >> >>>> > > JVMClusterUtil.java:145) >> >>>> > > at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster( >> >>>> > > LocalHBaseCluster.java:217) >> >>>> > > at org.apache.hadoop.hbase.LocalHBaseCluster.<init>( >> >>>> > > LocalHBaseCluster.java:152) >> >>>> > > at org.apache.hadoop.hbase.MiniHBaseCluster.init( >> >>>> > > MiniHBaseCluster.java:214) >> >>>> > > at org.apache.hadoop.hbase.MiniHBaseCluster.<init>( >> >>>> > > MiniHBaseCluster.java:94) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniHBaseCl >> >>>> uster( >> >>>> > > HBaseTestingUtility.java:1124) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:1078) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:949) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:943) >> >>>> > > at org.apache.hadoop.hbase.HBaseT >> estingUtility.startMiniCluster( >> >>>> > > HBaseTestingUtility.java:872) >> >>>> > > ... >> >>>> > > Cause: java.lang.UnsatisfiedLinkError: failed to load the >> required >> >>>> > > native >> >>>> > > library >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > Epoll.ensureAvailability(Epoll.java:78) >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) >> >>>> > > at org.apache.hadoop.hbase.util.N >> ettyEventLoopGroupConfig.<init>( >> >>>> > > NettyEventLoopGroupConfig.java:61) >> >>>> > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< >> >>>> > > init>(HRegionServer.java:552) >> >>>> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java: >> 475) >> >>>> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0( >> Native >> >>>> > Method) >> >>>> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( >> >>>> > > NativeConstructorAccessorImpl.java:62) >> >>>> > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> >>>> > > DelegatingConstructorAccessorImpl.java:45) >> >>>> > > at java.lang.reflect.Constructor. >> newInstance(Constructor.java:423) >> >>>> > > at org.apache.hadoop.hbase.util.J >> VMClusterUtil.createMasterThread( >> >>>> > > JVMClusterUtil.java:140) >> >>>> > > ... >> >>>> > > Cause: java.lang.UnsatisfiedLinkError: >> org.apache.hadoop.hbase. >> >>>> > > shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM >> >>>> > > ethods.epollin()I >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > NativeStaticallyReferencedJniMethods.epollin(Native Method) >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > Native.<clinit>(Native.java:66) >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > Epoll.<clinit>(Epoll.java:33) >> >>>> > > at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll. >> >>>> > > EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38) >> >>>> > > at org.apache.hadoop.hbase.util.N >> ettyEventLoopGroupConfig.<init>( >> >>>> > > NettyEventLoopGroupConfig.java:61) >> >>>> > > at org.apache.hadoop.hbase.regionserver.HRegionServer.< >> >>>> > > init>(HRegionServer.java:552) >> >>>> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java: >> 475) >> >>>> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0( >> Native >> >>>> > Method) >> >>>> > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( >> >>>> > > NativeConstructorAccessorImpl.java:62) >> >>>> > > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> >>>> > > DelegatingConstructorAccessorImpl.java:45) >> >>>> > > ... >> >>>> > > [INFO] ------------------------------ >> ------------------------------ >> >>>> > > ------------ >> >>>> > > [INFO] Reactor Summary: >> >>>> > > [INFO] >> >>>> > > [INFO] Apache HBase ....................................... >> SUCCESS >> >>>> [ >> >>>> > > 1.575 s] >> >>>> > > [INFO] Apache HBase - Checkstyle .......................... >> SUCCESS [ >> >>>> > > 0.317 s] >> >>>> > > [INFO] Apache HBase - Annotations ......................... >> SUCCESS [ >> >>>> > > 0.537 s] >> >>>> > > [INFO] Apache HBase - Build Configuration ................. >> SUCCESS [ >> >>>> > > 0.053 s] >> >>>> > > [INFO] Apache HBase - Shaded Protocol ..................... >> SUCCESS [ >> >>>> > > 15.410 s] >> >>>> > > [INFO] Apache HBase - Common .............................. >> SUCCESS [ >> >>>> > > 4.603 s] >> >>>> > > [INFO] Apache HBase - Metrics API ......................... >> SUCCESS [ >> >>>> > > 1.213 s] >> >>>> > > [INFO] Apache HBase - Hadoop Compatibility ................ >> SUCCESS [ >> >>>> > > 0.985 s] >> >>>> > > [INFO] Apache HBase - Metrics Implementation .............. >> SUCCESS [ >> >>>> > > 0.863 s] >> >>>> > > [INFO] Apache HBase - Hadoop Two Compatibility ............ >> SUCCESS [ >> >>>> > > 1.750 s] >> >>>> > > [INFO] Apache HBase - Protocol ............................ >> SUCCESS [ >> >>>> > > 4.880 s] >> >>>> > > [INFO] Apache HBase - Client .............................. >> SUCCESS [ >> >>>> > > 5.233 s] >> >>>> > > [INFO] Apache HBase - Replication ......................... >> SUCCESS [ >> >>>> > > 1.040 s] >> >>>> > > [INFO] Apache HBase - Prefix Tree ......................... >> SUCCESS [ >> >>>> > > 1.121 s] >> >>>> > > [INFO] Apache HBase - Procedure ........................... >> SUCCESS [ >> >>>> > > 1.084 s] >> >>>> > > [INFO] Apache HBase - Resource Bundle ..................... >> SUCCESS [ >> >>>> > > 0.092 s] >> >>>> > > [INFO] Apache HBase - Server .............................. >> SUCCESS [ >> >>>> > > 19.849 s] >> >>>> > > [INFO] Apache HBase - MapReduce ........................... >> SUCCESS [ >> >>>> > > 4.221 s] >> >>>> > > [INFO] Apache HBase - Testing Util ........................ >> SUCCESS [ >> >>>> > > 3.273 s] >> >>>> > > [INFO] Apache HBase - Thrift .............................. >> SUCCESS [ >> >>>> > > 5.519 s] >> >>>> > > [INFO] Apache HBase - RSGroup ............................. >> SUCCESS [ >> >>>> > > 3.408 s] >> >>>> > > [INFO] Apache HBase - Shell ............................... >> SUCCESS >> >>>> [ >> >>>> > > 3.859 s] >> >>>> > > [INFO] Apache HBase - Coprocessor Endpoint ................ >> SUCCESS [ >> >>>> > > 4.038 s] >> >>>> > > [INFO] Apache HBase - Backup .............................. >> SUCCESS >> >>>> > [01:13 >> >>>> > > min] >> >>>> > > [INFO] Apache HBase - Integration Tests ................... >> SUCCESS [ >> >>>> > > 4.229 s] >> >>>> > > [INFO] Apache HBase - Examples ............................ >> SUCCESS [ >> >>>> > > 3.471 s] >> >>>> > > [INFO] Apache HBase - Rest ................................ >> SUCCESS >> >>>> [ >> >>>> > > 4.448 s] >> >>>> > > [INFO] Apache HBase - External Block Cache ................ >> SUCCESS [ >> >>>> > > 2.040 s] >> >>>> > > [INFO] Apache HBase - Spark ............................... >> FAILURE >> >>>> [ >> >>>> > > 32.833 s] >> >>>> > > [INFO] Apache HBase - Spark Integration Tests ............. >> SKIPPED >> >>>> > > [INFO] Apache HBase - Assembly ............................ >> SKIPPED >> >>>> > > [INFO] Apache HBase - Shaded .............................. >> SKIPPED >> >>>> > > [INFO] Apache HBase - Shaded - Client ..................... >> SKIPPED >> >>>> > > [INFO] Apache HBase - Shaded - MapReduce .................. >> SKIPPED >> >>>> > > [INFO] Apache HBase Shaded Packaging Invariants ........... >> SKIPPED >> >>>> > > [INFO] Apache HBase - Archetypes .......................... >> SKIPPED >> >>>> > > [INFO] Apache HBase - Exemplar for hbase-client archetype . >> SKIPPED >> >>>> > > [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype >> >>>> SKIPPED >> >>>> > > [INFO] Apache HBase - Archetype builder ................... >> SKIPPED >> >>>> > > [INFO] ------------------------------ >> ------------------------------ >> >>>> > > ------------ >> >>>> > > [INFO] BUILD FAILURE >> >>>> > > [INFO] ------------------------------ >> ------------------------------ >> >>>> > > ------------ >> >>>> > > [INFO] Total time: 03:26 min >> >>>> > > [INFO] Finished at: 2017-09-27T19:34:35+05:30 >> >>>> > > [INFO] Final Memory: 345M/6055M >> >>>> > > [INFO] ------------------------------ >> ------------------------------ >> >>>> > > ------------ >> >>>> > > [ERROR] Failed to execute goal org.scalatest:scalatest-maven- >> >>>> > > plugin:1.0:test >> >>>> > > (test) on project hbase-spark: There are test failures -> [Help >> 1] >> >>>> > > [ERROR] >> >>>> > > [ERROR] To see the full stack trace of the errors, re-run Maven >> with >> >>>> the >> >>>> > -e >> >>>> > > switch. >> >>>> > > [ERROR] Re-run Maven using the -X switch to enable full debug >> >>>> logging. >> >>>> > > [ERROR] >> >>>> > > [ERROR] For more information about the errors and possible >> solutions, >> >>>> > > please read the following articles: >> >>>> > > [ERROR] [Help 1] http://cwiki.apache.org/conflu >> ence/display/MAVEN/ >> >>>> > > MojoFailureException >> >>>> > > [ERROR] >> >>>> > > [ERROR] After correcting the problems, you can resume the build >> with >> >>>> the >> >>>> > > command >> >>>> > > [ERROR] mvn <goals> -rf :hbase-spark >> >>>> > > >> >>>> > >> >>>> >> >>> >> >>> >> >> >> > >
