[jira] [Created] (HBASE-19913) Split TestStochasticLoadBalancer2
Duo Zhang created HBASE-19913: - Summary: Split TestStochasticLoadBalancer2 Key: HBASE-19913 URL: https://issues.apache.org/jira/browse/HBASE-19913 Project: HBase Issue Type: Sub-task Components: test Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19912) The flag "writeToWAL" of Region#checkAndRowMutate is useless
Chia-Ping Tsai created HBASE-19912: -- Summary: The flag "writeToWAL" of Region#checkAndRowMutate is useless Key: HBASE-19912 URL: https://issues.apache.org/jira/browse/HBASE-19912 Project: HBase Issue Type: Improvement Reporter: Chia-Ping Tsai Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19897) RowMutations should follow the fluent pattern
[
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-19897:
We can apply the mutation to the adder as BufferedMutator does
> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
> Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the
> fluent interface. Also, Changing the return type from {{Void}} to
> {{RowMutations}} won't break the API BC (unless someone has interest in
> {{Void}} object...)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19911) Convert some tests from small to medium because they are timing out: TestNettyRpcServer
stack created HBASE-19911: - Summary: Convert some tests from small to medium because they are timing out: TestNettyRpcServer Key: HBASE-19911 URL: https://issues.apache.org/jira/browse/HBASE-19911 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0-beta-2 Found some more timeouts of small tests. TestNettyRpcServer https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/11307/testReport/junit/org.apache.hadoop.hbase.ipc/TestNettyRpcServer/org_apache_hadoop_hbase_ipc_TestNettyRpcServer/ On local machine takes 14 seconds to run 1 test. In the above failure..., the test needs another second or so to complete It has been running 30 seconds. Starts a minihbasecluster, creates a table, then shuts down. Shouldn't even take 3 seconds but thats another story. TestClientClusterStatus is a small test that starts 5 regionservers and 3 masters. Then does some stopping of servers, etc. .. in same test run... https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/11307/testReport/junit/org.apache.hadoop.hbase/TestClientClusterStatus/org_apache_hadoop_hbase_TestClientClusterStatus/ ... it timed out after 30 seconds. Its almost done w/ shutdown but not quite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19910) TestBucketCache TimesOut
stack created HBASE-19910: - Summary: TestBucketCache TimesOut Key: HBASE-19910 URL: https://issues.apache.org/jira/browse/HBASE-19910 Project: HBase Issue Type: Sub-task Reporter: stack See https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/11303/testReport/org.apache.hadoop.hbase.master.balancer/TestRegionLocationFinder/org_apache_hadoop_hbase_master_balancer_TestRegionLocationFinder/ This is small test. Runs fast locally. 8 tests. Each is a second or two. Odd though up on jenkins is that in the middle of one, there is a 19 second pause. See here: 2018-02-01 00:56:30,013 INFO [Time-limited test] util.ByteBufferArray(70): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2018-02-01 00:56:49,678 INFO [Time-limited test] bucket.BucketCache(279): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 Here is full test run: 2018-02-01 00:56:29,981 INFO [Time-limited test] hbase.ResourceChecker(148): before: io.hfile.bucket.TestBucketCache#testInvalidCacheSplitFactorConfig[1: blockSize=16,384, bucketSizes=[I@20322d26] Thread=77, OpenFileDescriptor=263, MaxFileDescriptor=1048576, SystemLoadAverage=2127, ProcessCount=9, AvailableMemoryMB=7801 2018-02-01 00:56:30,013 INFO [Time-limited test] util.ByteBufferArray(70): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2018-02-01 00:56:49,678 INFO [Time-limited test] bucket.BucketCache(279): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2018-02-01 00:56:49,689 INFO [Time-limited test] bucket.BucketAllocator(334): Cache totalSize=33288192, buckets=63, bucket capacity=528384=(4*132096)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2018-02-01 00:56:49,690 INFO [Time-limited test] bucket.BucketCache(322): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=16 KB, writerThreadNum=3, writerQLen=64, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2018-02-01 00:56:50,020 INFO [Time-limited test] util.ByteBufferArray(70): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2018-02-01 00:56:50,080 ERROR [Time-limited test] util.ByteBufferArray(101): Buffer creation interrupted java.lang.InterruptedException at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) at java.util.concurrent.FutureTask.get(FutureTask.java:191) at org.apache.hadoop.hbase.util.ByteBufferArray.createBuffers(ByteBufferArray.java:96) at org.apache.hadoop.hbase.util.ByteBufferArray.(ByteBufferArray.java:74) at org.apache.hadoop.hbase.io.hfile.bucket.ByteBufferIOEngine.(ByteBufferIOEngine.java:86) at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getIOEngineFromName(BucketCache.java:384) at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.(BucketCache.java:262) at org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCache.checkConfigValues(TestBucketCache.java:387) at org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCache.testInvalidCacheSplitFactorConfig(TestBucketCache.java:377) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) a
[jira] [Created] (HBASE-19909) TestRegionLocationFinder Timeout
stack created HBASE-19909: - Summary: TestRegionLocationFinder Timeout Key: HBASE-19909 URL: https://issues.apache.org/jira/browse/HBASE-19909 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0-beta-2 This test is timing out a bunch in runs since we moved over to the nice new fancy, smancy, timeout thingymajig. Similar to HBASE-19908, I see that on Jenkins, the test is making progress but is running at a slower rate. This is a 'smalltest' that starts a minicluster with 5 servers creating a table with 26 odd regions. On my uncontested machine, it takes 20 seconds to complete the create table. On jenkins it takes 29 seconds (see https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/11303/testReport/org.apache.hadoop.hbase.master.balancer/TestRegionLocationFinder/org_apache_hadoop_hbase_master_balancer_TestRegionLocationFinder/) Small tests are supposed to complete inside 30 seconds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19908) TestCoprocessorShortCircuitRPC Timeout....
stack created HBASE-19908: - Summary: TestCoprocessorShortCircuitRPC Timeout Key: HBASE-19908 URL: https://issues.apache.org/jira/browse/HBASE-19908 Project: HBase Issue Type: Sub-task Reporter: stack Timedout in HBASE-19906 Comparing a local run (16seconds total) to a timed out run up on jenkins, I see it takes my local test 5 seconds to get the STOPPED server log line. On jenkins in this timed out test it takes 30 seconds. Test is still running when it is killed. Let me make it a medium test. https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/lastCompletedBuild/testReport/org.apache.hadoop.hbase.coprocessor/TestCoprocessorShortCircuitRPC/org_apache_hadoop_hbase_coprocessor_TestCoprocessorShortCircuitRPC/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] The first HBase 1.4.1 release candidate (RC0) is available
+1 non-binding download src & bin tar ball - OK signatures & sums- OK build from source (openjdk version "1.8.0_151") - OK rat check - OK unit tests - OK start local instance from bin & CRUD from shell - OK LTT write, read1 million rows, 2 cols/row - OK check logs - OK On Wed, Jan 31, 2018 at 11:54 AM, Andrew Purtell wrote: > > could not compile against hadoop3 (several issues, will file jira later) > > Thanks, that would be helpful. > > We need to discuss it of course but I don't think we should consider > ticking the Hadoop 3 is supported box on 1.x until the next minor release, > 1.5, at which time we could do this work. > > > On Wed, Jan 31, 2018 at 10:20 AM, Mike Drob wrote: > > > +1 non-binding > > > > sigs/sums good > > compiles against hadoop2 (7u80) > > ran hbase-downstreamer tests against staging repo > > verified upgrade from 1.4.1 to 2.0.0-beta1 > > > > could not compile against hadoop3 (several issues, will file jira later) > > > > On Wed, Jan 31, 2018 at 8:52 AM, Ashish Singhi > > wrote: > > > > > +1 (non-binding) > > > > > > Downloaded the binary and checked the following, > > > > > > - Checked LICENSE and NOTICE files, looks ok > > > - Apache RAT check, looks good > > > > > > Verified the following in a non-secure setup, > > > > > > - Exercised basic shell commands > > > - Ran LTT with 1M row, checked read and write operations > > > - Poked around webUI > > > - Started Rest server and executed some commands > > > - Checked logs > > > - Browsed book > > > > > > Regards, > > > Ashish > > > > > > On Thu, Jan 25, 2018 at 8:15 AM, Andrew Purtell > > > wrote: > > > > > > > The first HBase 1.4.1 release candidate (RC0) is available for > download > > > at > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ and > Maven > > > > artifacts are available in the temporary repository > > > > https://repository.apache.org/content/repositories/ > orgapachehbase-1194 > > . > > > > > > > > The git tag corresponding to the candidate is '1.4.1RC0' > (4b25debc83). > > > > > > > > A detailed source and binary compatibility report for this release is > > > > available for your review at > > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ > > > > compat-check-report.html > > > > . > > > > > > > > A list of the 38 issues resolved in this release can be found at > > > > https://s.apache.org/tx1w . > > > > > > > > Please try out the candidate and vote +1/0/-1. > > > > > > > > This vote will be open for at least 72 hours. Unless objection I will > > try > > > > to close it Friday February 2, 2018 if we have sufficient votes. > > > > > > > > Prior to making this announcement I made the following preflight > > checks: > > > > > > > >- RAT check passes (7u80) > > > >- Unit test suite passes 10 of 10 iterations (8u131) > > > >- LTT load 1M rows with 100% verification and 20% updates (8u131) > > > >- PE sequentialWrite, sequentialRead, randomWrite, randomRead, > > > >scanRange100 (8u131) > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrew > > > > > > > > Words like orphans lost among the crosstalk, meaning torn from > truth's > > > > decrepit hands > > > >- A23, Crosstalk > > > > > > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
[jira] [Created] (HBASE-19907) TestMetaWithReplicas still flakey
stack created HBASE-19907: - Summary: TestMetaWithReplicas still flakey Key: HBASE-19907 URL: https://issues.apache.org/jira/browse/HBASE-19907 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 2.0.0-beta-2 Still fails because all meta replicas arrive at same server even though supposedly protection against this added by me in HBASE-19840. --- Test set: org.apache.hadoop.hbase.client.TestMetaWithReplicas --- Tests run: 5, Failures: 0, Errors: 2, Skipped: 1, Time elapsed: 600.251 s <<< FAILURE! - in org.apache.hadoop.hbase.client.TestMetaWithReplicas org.apache.hadoop.hbase.client.TestMetaWithReplicas Time elapsed: 563.656 s <<< ERROR! org.junit.runners.model.TestTimedOutException: test timed out after 600 seconds at org.apache.hadoop.hbase.client.TestMetaWithReplicas.shutdownMetaAndDoValidations(TestMetaWithReplicas.java:255) at org.apache.hadoop.hbase.client.TestMetaWithReplicas.testShutdownHandling(TestMetaWithReplicas.java:181) org.apache.hadoop.hbase.client.TestMetaWithReplicas Time elapsed: 563.656 s <<< ERROR! java.lang.Exception: Appears to be stuck in thread NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49912 The move of hbase:meta actually moves it back to same server no good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19906) TestZooKeeper Timeout
stack created HBASE-19906: - Summary: TestZooKeeper Timeout Key: HBASE-19906 URL: https://issues.apache.org/jira/browse/HBASE-19906 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 2.0.0-beta-2 TestZooKeeper is timing out causing hbase2 failures and breaking HBASE-Flaky-Tests-branch2.0.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] The first HBase 1.4.1 release candidate (RC0) is available
> could not compile against hadoop3 (several issues, will file jira later) Thanks, that would be helpful. We need to discuss it of course but I don't think we should consider ticking the Hadoop 3 is supported box on 1.x until the next minor release, 1.5, at which time we could do this work. On Wed, Jan 31, 2018 at 10:20 AM, Mike Drob wrote: > +1 non-binding > > sigs/sums good > compiles against hadoop2 (7u80) > ran hbase-downstreamer tests against staging repo > verified upgrade from 1.4.1 to 2.0.0-beta1 > > could not compile against hadoop3 (several issues, will file jira later) > > On Wed, Jan 31, 2018 at 8:52 AM, Ashish Singhi > wrote: > > > +1 (non-binding) > > > > Downloaded the binary and checked the following, > > > > - Checked LICENSE and NOTICE files, looks ok > > - Apache RAT check, looks good > > > > Verified the following in a non-secure setup, > > > > - Exercised basic shell commands > > - Ran LTT with 1M row, checked read and write operations > > - Poked around webUI > > - Started Rest server and executed some commands > > - Checked logs > > - Browsed book > > > > Regards, > > Ashish > > > > On Thu, Jan 25, 2018 at 8:15 AM, Andrew Purtell > > wrote: > > > > > The first HBase 1.4.1 release candidate (RC0) is available for download > > at > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ and Maven > > > artifacts are available in the temporary repository > > > https://repository.apache.org/content/repositories/orgapachehbase-1194 > . > > > > > > The git tag corresponding to the candidate is '1.4.1RC0' (4b25debc83). > > > > > > A detailed source and binary compatibility report for this release is > > > available for your review at > > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ > > > compat-check-report.html > > > . > > > > > > A list of the 38 issues resolved in this release can be found at > > > https://s.apache.org/tx1w . > > > > > > Please try out the candidate and vote +1/0/-1. > > > > > > This vote will be open for at least 72 hours. Unless objection I will > try > > > to close it Friday February 2, 2018 if we have sufficient votes. > > > > > > Prior to making this announcement I made the following preflight > checks: > > > > > >- RAT check passes (7u80) > > >- Unit test suite passes 10 of 10 iterations (8u131) > > >- LTT load 1M rows with 100% verification and 20% updates (8u131) > > >- PE sequentialWrite, sequentialRead, randomWrite, randomRead, > > >scanRange100 (8u131) > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: [VOTE] The first HBase 1.4.1 release candidate (RC0) is available
+1 non-binding sigs/sums good compiles against hadoop2 (7u80) ran hbase-downstreamer tests against staging repo verified upgrade from 1.4.1 to 2.0.0-beta1 could not compile against hadoop3 (several issues, will file jira later) On Wed, Jan 31, 2018 at 8:52 AM, Ashish Singhi wrote: > +1 (non-binding) > > Downloaded the binary and checked the following, > > - Checked LICENSE and NOTICE files, looks ok > - Apache RAT check, looks good > > Verified the following in a non-secure setup, > > - Exercised basic shell commands > - Ran LTT with 1M row, checked read and write operations > - Poked around webUI > - Started Rest server and executed some commands > - Checked logs > - Browsed book > > Regards, > Ashish > > On Thu, Jan 25, 2018 at 8:15 AM, Andrew Purtell > wrote: > > > The first HBase 1.4.1 release candidate (RC0) is available for download > at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ and Maven > > artifacts are available in the temporary repository > > https://repository.apache.org/content/repositories/orgapachehbase-1194 . > > > > The git tag corresponding to the candidate is '1.4.1RC0' (4b25debc83). > > > > A detailed source and binary compatibility report for this release is > > available for your review at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ > > compat-check-report.html > > . > > > > A list of the 38 issues resolved in this release can be found at > > https://s.apache.org/tx1w . > > > > Please try out the candidate and vote +1/0/-1. > > > > This vote will be open for at least 72 hours. Unless objection I will try > > to close it Friday February 2, 2018 if we have sufficient votes. > > > > Prior to making this announcement I made the following preflight checks: > > > >- RAT check passes (7u80) > >- Unit test suite passes 10 of 10 iterations (8u131) > >- LTT load 1M rows with 100% verification and 20% updates (8u131) > >- PE sequentialWrite, sequentialRead, randomWrite, randomRead, > >scanRange100 (8u131) > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > >
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
I just set hadoopqa to be 0.7.0 again with an upped proclimit to see if this fixes our OOME failures.. HadoopQA builds numbered 11295 and later will have this change. Thanks S On Wed, Jan 31, 2018 at 6:46 AM, Stack wrote: > Note that I reverted our yetus version last night. It discombobulated our > builds (OOMEs). Meantime, you'll have to do the patch naming trick for > another day or so. Our test runs seem to use an ungodly number of file > descriptors Stay tuned. > S > > On Mon, Jan 29, 2018 at 10:56 PM, Stack wrote: > >> Our brothers and sisters over in yetus-land made a release that deals w/ >> the changed JIRA behavior regards ordering attached-patches. No need of >> deleting all but the intended patch going forward nor gymnastics with >> prefixes when naming. It seems to be working properly. The one-liner >> change that moves us from yetus 0.6.0 to 0.7.0 has been pushed to all >> active branches and our hadoopqa up on jenkins has been configured to use >> it going forward. >> >> FYI, >> S >> >> On Fri, Dec 8, 2017 at 12:13 PM, Stack wrote: >> >>> Thanks Andrew. I disabled the job. Use the nightly going forward. The >>> jdk7 builds seem to run fine. The jdk8 has some timeout going on. Need to >>> dig in. You can see here: >>> >>> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Ni >>> ghtly/job/branch-1.4/ >>> >>> Thanks, >>> M >>> >>> On Fri, Dec 8, 2017 at 11:29 AM, Andrew Purtell >>> wrote: >>> Ok with me, Stack. Thanks for asking. On Thu, Nov 30, 2017 at 5:33 PM, Stack wrote: > On the move over to nightly test runs: > > 1.2 nightly had a successful build last night after the branch-1 > stabilization effort (HBASE-19204) and fixing a few unit test failures. See > build 150 > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > 20Nightly/job/branch-1.2/ > It then failed, 151, because of timed out test. Need to dig in. Clean up a > few more unit tests and branch-1.2 is probably ready for a release-cutting. > > 1.3 has a few flakies. The last build failed because of: > > Test Result (1 failure / ±0) > org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation. > testCFKeyRotation > > Just a little effort should turn 1.3 green. > > I was going to disable the 1.4 job, > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/, in favor of > the 1.4 nightly, > https://builds.apache.org/view/H-L/view/HBase/job/HBase% > 20Nightly/job/branch-1.4/, > if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, and > master too. > > Thanks, > S > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > Example of the new nice reporting: vhttps://builds.apache.org/ > > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/ > > S > > > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: > > > >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8, > >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a > good > >> while now. In their place, refer to an ongoing Sean "Nightly" project, > an > >> effort he has been at for a while. It does more checking with pretty > >> reports that will help figuring general stability over time. See under > >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ > >> See the nightly builds for 1.2 and 1.3. They have some teething issues > >> still but are almost there. See the 1.2 build from last night. In recent > >> days, the 1.2 branch went from trash-can fire to stable. See how all > tests > >> passed in the last build but then we failed generating the src bundle on > >> the end (this is what I mean by 'teething' issue). Will work on fixing > this > >> last step and moving over 1.4, etc., in the next few days. > >> > >> FYI, > >> St.Ack > >> > >> > >> On Tue, Nov 7, 2017 at 7:45 AM, Stack wrote: > >> > >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey wrote: > >>> > > Should I be able to see the machine dir when I look at nightlies > output? > > (Was trying to see what else is running). > > Ah. we don't have the same machine sampling on nightly as we do in > precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc > repeatedly) that includes pulling that information gathering into a > place where we could also use it in nightly. > > > >>> Sweet. > >>> > >>> > >>> > Did we ever figure out how many cores we expect our tests to need? It > looks like the Hadoop nodes have 8 cores. (with 2 executors that means
Re: [VOTE] The first HBase 1.4.1 release candidate (RC0) is available
+1 (non-binding) Downloaded the binary and checked the following, - Checked LICENSE and NOTICE files, looks ok - Apache RAT check, looks good Verified the following in a non-secure setup, - Exercised basic shell commands - Ran LTT with 1M row, checked read and write operations - Poked around webUI - Started Rest server and executed some commands - Checked logs - Browsed book Regards, Ashish On Thu, Jan 25, 2018 at 8:15 AM, Andrew Purtell wrote: > The first HBase 1.4.1 release candidate (RC0) is available for download at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ and Maven > artifacts are available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1194 . > > The git tag corresponding to the candidate is '1.4.1RC0' (4b25debc83). > > A detailed source and binary compatibility report for this release is > available for your review at > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ > compat-check-report.html > . > > A list of the 38 issues resolved in this release can be found at > https://s.apache.org/tx1w . > > Please try out the candidate and vote +1/0/-1. > > This vote will be open for at least 72 hours. Unless objection I will try > to close it Friday February 2, 2018 if we have sufficient votes. > > Prior to making this announcement I made the following preflight checks: > >- RAT check passes (7u80) >- Unit test suite passes 10 of 10 iterations (8u131) >- LTT load 1M rows with 100% verification and 20% updates (8u131) >- PE sequentialWrite, sequentialRead, randomWrite, randomRead, >scanRange100 (8u131) > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: Testing and CI -- Apache Jenkins Builds (WAS -> Re: Testing)
Note that I reverted our yetus version last night. It discombobulated our builds (OOMEs). Meantime, you'll have to do the patch naming trick for another day or so. Our test runs seem to use an ungodly number of file descriptors Stay tuned. S On Mon, Jan 29, 2018 at 10:56 PM, Stack wrote: > Our brothers and sisters over in yetus-land made a release that deals w/ > the changed JIRA behavior regards ordering attached-patches. No need of > deleting all but the intended patch going forward nor gymnastics with > prefixes when naming. It seems to be working properly. The one-liner > change that moves us from yetus 0.6.0 to 0.7.0 has been pushed to all > active branches and our hadoopqa up on jenkins has been configured to use > it going forward. > > FYI, > S > > On Fri, Dec 8, 2017 at 12:13 PM, Stack wrote: > >> Thanks Andrew. I disabled the job. Use the nightly going forward. The >> jdk7 builds seem to run fine. The jdk8 has some timeout going on. Need to >> dig in. You can see here: >> >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Ni >> ghtly/job/branch-1.4/ >> >> Thanks, >> M >> >> On Fri, Dec 8, 2017 at 11:29 AM, Andrew Purtell >> wrote: >> >>> Ok with me, Stack. Thanks for asking. >>> >>> >>> On Thu, Nov 30, 2017 at 5:33 PM, Stack wrote: >>> >>> > On the move over to nightly test runs: >>> > >>> > 1.2 nightly had a successful build last night after the branch-1 >>> > stabilization effort (HBASE-19204) and fixing a few unit test >>> failures. See >>> > build 150 >>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase% >>> > 20Nightly/job/branch-1.2/ >>> > It then failed, 151, because of timed out test. Need to dig in. Clean >>> up a >>> > few more unit tests and branch-1.2 is probably ready for a >>> release-cutting. >>> > >>> > 1.3 has a few flakies. The last build failed because of: >>> > >>> > Test Result (1 failure / ±0) >>> > org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation. >>> > testCFKeyRotation >>> > >>> > Just a little effort should turn 1.3 green. >>> > >>> > I was going to disable the 1.4 job, >>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase-1.4/, in >>> favor of >>> > the 1.4 nightly, >>> > https://builds.apache.org/view/H-L/view/HBase/job/HBase% >>> > 20Nightly/job/branch-1.4/, >>> > if ok w/ you Andrew Purtell... And move over the branch-1, branch-2, >>> and >>> > master too. >>> > >>> > Thanks, >>> > S >>> > >>> > >>> > >>> > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: >>> > >>> > > Example of the new nice reporting: vhttps://builds.apache.org/ >>> > > view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/ >>> > > S >>> > > >>> > > On Wed, Nov 29, 2017 at 8:06 AM, Stack wrote: >>> > > >>> > >> Note that I have disabled the HBase-1.2-JDK7, HBase-1.2-JDK8, >>> > >> HBase-1.3-JDK7, and HBase-1.3-JDK8 jobs. They have been broken for a >>> > good >>> > >> while now. In their place, refer to an ongoing Sean "Nightly" >>> project, >>> > an >>> > >> effort he has been at for a while. It does more checking with pretty >>> > >> reports that will help figuring general stability over time. See >>> under >>> > >> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/ >>> > >> See the nightly builds for 1.2 and 1.3. They have some teething >>> issues >>> > >> still but are almost there. See the 1.2 build from last night. In >>> recent >>> > >> days, the 1.2 branch went from trash-can fire to stable. See how all >>> > tests >>> > >> passed in the last build but then we failed generating the src >>> bundle on >>> > >> the end (this is what I mean by 'teething' issue). Will work on >>> fixing >>> > this >>> > >> last step and moving over 1.4, etc., in the next few days. >>> > >> >>> > >> FYI, >>> > >> St.Ack >>> > >> >>> > >> >>> > >> On Tue, Nov 7, 2017 at 7:45 AM, Stack wrote: >>> > >> >>> > >>> On Tue, Nov 7, 2017 at 6:10 AM, Sean Busbey >>> wrote: >>> > >>> >>> > > Should I be able to see the machine dir when I look at nightlies >>> > output? >>> > > (Was trying to see what else is running). >>> > >>> > Ah. we don't have the same machine sampling on nightly as we do in >>> > precommit. I am 80% on a patch for HBASE-19189 (run test ad-hoc >>> > repeatedly) that includes pulling that information gathering >>> into a >>> > place where we could also use it in nightly. >>> > >>> > >>> > >>> Sweet. >>> > >>> >>> > >>> >>> > >>> >>> > Did we ever figure out how many cores we expect our tests to >>> need? It >>> > looks like the Hadoop nodes have 8 cores. (with 2 executors that >>> means >>> > 4 is our fair share) >>> > >>> > >>> > >>> At the end of the thread inquiry I suggested that we don't use >>> enough >>> > >>> cores, that we could up our fork counts and tests would complete in >>> > less >>> > >>> time. I wanted to experiment some w/ high fork counts -- 16 or so >>> -- >>> > to see >>> > >>> if concurrent running brought on more failure. >>> > >>> >>>
[jira] [Created] (HBASE-19905) ReplicationSyncUp tool will not exit if a peer replication is disabled
Ashish Singhi created HBASE-19905: - Summary: ReplicationSyncUp tool will not exit if a peer replication is disabled Key: HBASE-19905 URL: https://issues.apache.org/jira/browse/HBASE-19905 Project: HBase Issue Type: Bug Components: Replication Affects Versions: 1.3.1 Reporter: Ashish Singhi Assignee: Ashish Singhi -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19904) Find a better way to deal with the cyclic dependency when initializing WAL and Replication
Duo Zhang created HBASE-19904: - Summary: Find a better way to deal with the cyclic dependency when initializing WAL and Replication Key: HBASE-19904 URL: https://issues.apache.org/jira/browse/HBASE-19904 Project: HBase Issue Type: Improvement Reporter: Duo Zhang When implementing synchronous replication, I found that we need to depend more on replication in WAL so it is even more pain... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19903) Split TestShell so it will not time out
Duo Zhang created HBASE-19903: - Summary: Split TestShell so it will not time out Key: HBASE-19903 URL: https://issues.apache.org/jira/browse/HBASE-19903 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang It runs more than 10 minutes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
