[GitHub] [hbase] infraio commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
infraio commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182#issuecomment-588074037 The failed ut not related. Ping @Apache9 for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[ https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039754#comment-17039754 ] Michael Stack commented on HBASE-23867: --- !Screen Shot 2020-02-18 at 11.15.49 PM.png! Here are the flakies again after two builds that show this patch seems to have stopped this test failing. > [Flakey Test] > TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts > - > > Key: HBASE-23867 > URL: https://issues.apache.org/jira/browse/HBASE-23867 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch, Screen Shot > 2020-02-18 at 10.45.22 PM.png, Screen Shot 2020-02-18 at 11.15.49 PM.png > > > Test has this on it: > @Test // Test is flakey. TODO: Fix! > Let me cut it down. It runs for a while. > It fails nearly every run since HBASE-23865 Up flakey history from 5 to > 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[ https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23867: -- Attachment: Screen Shot 2020-02-18 at 11.15.49 PM.png > [Flakey Test] > TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts > - > > Key: HBASE-23867 > URL: https://issues.apache.org/jira/browse/HBASE-23867 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch, Screen Shot > 2020-02-18 at 10.45.22 PM.png, Screen Shot 2020-02-18 at 11.15.49 PM.png > > > Test has this on it: > @Test // Test is flakey. TODO: Fix! > Let me cut it down. It runs for a while. > It fails nearly every run since HBASE-23865 Up flakey history from 5 to > 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[ https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039742#comment-17039742 ] Michael Stack commented on HBASE-23867: --- !Screen Shot 2020-02-18 at 10.45.22 PM.png! What https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html is currently showing for this test. > [Flakey Test] > TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts > - > > Key: HBASE-23867 > URL: https://issues.apache.org/jira/browse/HBASE-23867 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch, Screen Shot > 2020-02-18 at 10.45.22 PM.png > > > Test has this on it: > @Test // Test is flakey. TODO: Fix! > Let me cut it down. It runs for a while. > It fails nearly every run since HBASE-23865 Up flakey history from 5 to > 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[ https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23867: -- Attachment: Screen Shot 2020-02-18 at 10.45.22 PM.png > [Flakey Test] > TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts > - > > Key: HBASE-23867 > URL: https://issues.apache.org/jira/browse/HBASE-23867 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch, Screen Shot > 2020-02-18 at 10.45.22 PM.png > > > Test has this on it: > @Test // Test is flakey. TODO: Fix! > Let me cut it down. It runs for a while. > It fails nearly every run since HBASE-23865 Up flakey history from 5 to > 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[
https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23867.
---
Fix Version/s: 2.3.0
3.0.0
Assignee: Michael Stack
Resolution: Fixed
What I pushed on branch-2 and master:
{code}
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaSameHosts.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaSameHosts.java
index b9c3ec8ef2..470230deb1 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaSameHosts.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaSameHosts.java
@@ -1,4 +1,4 @@
-/**
+/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
@@ -24,27 +24,27 @@ import java.util.TreeMap;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.ServerName;
import org.apache.hadoop.hbase.client.RegionInfo;
-import org.apache.hadoop.hbase.testclassification.LargeTests;
import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-@Category({ MasterTests.class, LargeTests.class })
+@Category({ MasterTests.class, MediumTests.class })
public class TestStochasticLoadBalancerRegionReplicaSameHosts extends
BalancerTestBase {
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestStochasticLoadBalancerRegionReplicaSameHosts.class);
- @Test // Test is flakey. TODO: Fix!
+ @Test
public void testRegionReplicationOnMidClusterSameHosts() {
conf.setLong(StochasticLoadBalancer.MAX_STEPS_KEY, 200L);
conf.setLong("hbase.master.balancer.stochastic.maxRunningTime", 90 *
1000); // 90 sec
conf.setFloat("hbase.master.balancer.stochastic.maxMovePercent", 1.0f);
loadBalancer.setConf(conf);
-int numHosts = 100;
-int numRegions = 100 * 100;
+int numHosts = 30;
+int numRegions = 30 * 30;
int replication = 3; // 3 replicas per region
int numRegionsPerServer = 5;
int numTables = 10;
{code}
> [Flakey Test]
> TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
> -
>
> Key: HBASE-23867
> URL: https://issues.apache.org/jira/browse/HBASE-23867
> Project: HBase
> Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch
>
>
> Test has this on it:
> @Test // Test is flakey. TODO: Fix!
> Let me cut it down. It runs for a while.
> It fails nearly every run since HBASE-23865 Up flakey history from 5 to
> 10 which upped the flakey test so it ran with more fork count.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
[ https://issues.apache.org/jira/browse/HBASE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23867: -- Attachment: 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch > [Flakey Test] > TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts > - > > Key: HBASE-23867 > URL: https://issues.apache.org/jira/browse/HBASE-23867 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Priority: Major > Attachments: > 0001-HBASE-23867-Flakey-Test-TestStochasticLoadBalancerRe.patch > > > Test has this on it: > @Test // Test is flakey. TODO: Fix! > Let me cut it down. It runs for a while. > It fails nearly every run since HBASE-23865 Up flakey history from 5 to > 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23867) [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts
Michael Stack created HBASE-23867: - Summary: [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts Key: HBASE-23867 URL: https://issues.apache.org/jira/browse/HBASE-23867 Project: HBase Issue Type: Task Reporter: Michael Stack Test has this on it: @Test // Test is flakey. TODO: Fix! Let me cut it down. It runs for a while. It fails nearly every run since HBASE-23865 Up flakey history from 5 to 10 which upped the flakey test so it ran with more fork count. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-588055687 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 37s | master passed | | +1 :green_heart: | compile | 3m 0s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 46s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 57s | the patch passed | | +1 :green_heart: | compile | 3m 7s | the patch passed | | +1 :green_heart: | javac | 3m 7s | the patch passed | | +1 :green_heart: | hadolint | 0m 1s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 1s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 2m 42s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 68m 51s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 123m 53s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1183 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 84f5a00683ba 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1183/out/precommit/personality/provided.sh | | git revision | master / 86e76c2bd9 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/5/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/5/testReport/ | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/5/console | | versions | git=2.24.1 maven=2018-06-17T18:33:14Z) shellcheck=0.7.0 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182#issuecomment-588051570 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 22s | master passed | | +1 :green_heart: | compile | 1m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 5m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | | +0 :ok: | spotbugs | 5m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 31s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 10s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | checkstyle | 1m 23s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 18m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | | +1 :green_heart: | findbugs | 6m 59s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 143m 31s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 214m 49s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestNamespaceReplicationWithBulkLoadedData | | | hadoop.hbase.replication.TestMasterReplication | | | hadoop.hbase.security.access.TestAccessController3 | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | | hadoop.hbase.replication.TestReplicationStatusAfterLagging | | | hadoop.hbase.replication.regionserver.TestReplicationSink | | | hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster | | | hadoop.hbase.replication.TestSyncReplicationStandbyKillRS | | | hadoop.hbase.replication.regionserver.TestWALEntryStream | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1182 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ebdb9156ab47 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1182/out/precommit/personality/provided.sh | | git revision | master / 86e76c2bd9 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/4/testReport/ | | Max. process+thread count | 6954 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-23866) More test classifications
[ https://issues.apache.org/jira/browse/HBASE-23866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-23866. --- Fix Version/s: 2.3.0 3.0.0 Assignee: Michael Stack Resolution: Fixed Pushed trivial changes to branch-2 and master. > More test classifications > - > > Key: HBASE-23866 > URL: https://issues.apache.org/jira/browse/HBASE-23866 > Project: HBase > Issue Type: Task > Components: test > Environment: Minor stuff but helps w/ my testing odyssey >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Trivial > Fix For: 3.0.0, 2.3.0 > > Attachments: 0001-HBASE-23866-More-test-classifications.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23866) More test classifications
[
https://issues.apache.org/jira/browse/HBASE-23866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039706#comment-17039706
]
Michael Stack commented on HBASE-23866:
---
{code}
HBASE-23866 More test classifications
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncMetaRegionLocator.java
Add wait on no region in transition in setup before proceeding.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncResultScannerCursor.java
Make test large. Takes longer than 50seconds.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableRSCrashPublish.java
Make large because it fails when networking is weird... when VPN or
something is up. See HBASE-23850. When large, doesn't get run
when you do mvn test.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableScanAll.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestMultiRowRangeFilter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
Make it large. Takes too long.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSeparateClientZKCluster.java
Use data dir instead of /tmp.
M
hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestInfoServersACL.java
Wait till mbean is up before proceeding w/ test.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/SecureTestCluster.java
Test for null CLUSTER in tearDown.
{code}
> More test classifications
> -
>
> Key: HBASE-23866
> URL: https://issues.apache.org/jira/browse/HBASE-23866
> Project: HBase
> Issue Type: Task
> Components: test
> Environment: Minor stuff but helps w/ my testing odyssey
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-HBASE-23866-More-test-classifications.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23866) More test classifications
[ https://issues.apache.org/jira/browse/HBASE-23866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23866: -- Attachment: 0001-HBASE-23866-More-test-classifications.patch > More test classifications > - > > Key: HBASE-23866 > URL: https://issues.apache.org/jira/browse/HBASE-23866 > Project: HBase > Issue Type: Task > Components: test > Environment: Minor stuff but helps w/ my testing odyssey >Reporter: Michael Stack >Priority: Trivial > Attachments: 0001-HBASE-23866-More-test-classifications.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23866) More test classifications
Michael Stack created HBASE-23866: - Summary: More test classifications Key: HBASE-23866 URL: https://issues.apache.org/jira/browse/HBASE-23866 Project: HBase Issue Type: Task Components: test Environment: Minor stuff but helps w/ my testing odyssey Reporter: Michael Stack -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23865) Up flakey history from 5 to 10
[
https://issues.apache.org/jira/browse/HBASE-23865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039698#comment-17039698
]
Hudson commented on HBASE-23865:
Results for branch branch-2
[build #2485 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2485/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2485//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2485//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2485//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Up flakey history from 5 to 10
> --
>
> Key: HBASE-23865
> URL: https://issues.apache.org/jira/browse/HBASE-23865
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-HBASE-23865-Up-flakey-history-from-5-to-10.patch
>
>
> Also change the second and first part fort counts to be 1C rather than
> hardcoded 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22749) Distributed MOB compactions
[
https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039695#comment-17039695
]
Hudson commented on HBASE-22749:
Results for branch HBASE-22749
[build #8 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/8/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/8//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/8//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/8//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Distributed MOB compactions
>
>
> Key: HBASE-22749
> URL: https://issues.apache.org/jira/browse/HBASE-22749
> Project: HBase
> Issue Type: New Feature
> Components: mob
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-22749-branch-2.2-v4.patch,
> HBASE-22749-master-v1.patch, HBASE-22749-master-v2.patch,
> HBASE-22749-master-v3.patch, HBASE-22749-master-v4.patch,
> HBase-MOB-2.0-v3.0.pdf
>
>
> There are several drawbacks in the original MOB 1.0 (Moderate Object
> Storage) implementation, which can limit the adoption of the MOB feature:
> # MOB compactions are executed in a Master as a chore, which limits
> scalability because all I/O goes through a single HBase Master server.
> # Yarn/Mapreduce framework is required to run MOB compactions in a scalable
> way, but this won’t work in a stand-alone HBase cluster.
> # Two separate compactors for MOB and for regular store files and their
> interactions can result in a data loss (see HBASE-22075)
> The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible
> implementation, which is free of the above drawbacks and can be used as a
> drop in replacement in existing MOB deployments. So, these are design goals
> of a MOB 2.0:
> # Make MOB compactions scalable without relying on Yarn/Mapreduce framework
> # Provide unified compactor for both MOB and regular store files
> # Make it more robust especially w.r.t. to data losses.
> # Simplify and reduce the overall MOB code.
> # Provide 100% compatible implementation with MOB 1.0.
> # No migration of data should be required between MOB 1.0 and MOB 2.0 - just
> software upgrade.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-588026968 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/5/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-588026591 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 56s | master passed | | +1 :green_heart: | compile | 3m 17s | master passed | | +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 50s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 30s | the patch passed | | +1 :green_heart: | compile | 3m 19s | the patch passed | | +1 :green_heart: | javac | 3m 19s | the patch passed | | +1 :green_heart: | hadolint | 0m 1s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 5m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 19s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 106m 2s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 166m 55s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationSmallTestsSync | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1183 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux d4b8f5e6a200 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1183/out/precommit/personality/provided.sh | | git revision | master / 86e76c2bd9 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/4/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/4/testReport/ | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/4/console | | versions | git=2.24.1 maven=2018-06-17T18:33:14Z) shellcheck=0.7.0 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23858) Set version as 2.1.10-SNAPSHOT in branch-2.1
[
https://issues.apache.org/jira/browse/HBASE-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039677#comment-17039677
]
Hudson commented on HBASE-23858:
Results for branch branch-2.1
[build #1808 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Set version as 2.1.10-SNAPSHOT in branch-2.1
>
>
> Key: HBASE-23858
> URL: https://issues.apache.org/jira/browse/HBASE-23858
> Project: HBase
> Issue Type: Sub-task
> Components: build, pom
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.1.10
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23859) Modify "Block locality" of RegionServer Web UI to human readable percentage
[
https://issues.apache.org/jira/browse/HBASE-23859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039676#comment-17039676
]
Hudson commented on HBASE-23859:
Results for branch branch-2.1
[build #1808 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1808//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify "Block locality" of RegionServer Web UI to human readable percentage
> ---
>
> Key: HBASE-23859
> URL: https://issues.apache.org/jira/browse/HBASE-23859
> Project: HBase
> Issue Type: Improvement
>Affects Versions: master
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: image-2020-02-17-20-36-50-084.png,
> image-2020-02-17-20-38-52-990.png
>
>
> The unit of "Block locality" in Web UI just like picture 1
> !image-2020-02-17-20-36-50-084.png!
> I think we should change it to percentage unit like picture 2
> !image-2020-02-17-20-38-52-990.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1114: HBASE-23146 Support CheckAndMutate with multiple conditions
Apache-HBase commented on issue #1114: HBASE-23146 Support CheckAndMutate with multiple conditions URL: https://github.com/apache/hbase/pull/1114#issuecomment-588005398 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 8 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 50s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 29s | master passed | | +1 :green_heart: | compile | 4m 22s | master passed | | +1 :green_heart: | checkstyle | 3m 18s | master passed | | +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 28s | master passed | | +0 :ok: | spotbugs | 1m 43s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 13m 20s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 46s | the patch passed | | +1 :green_heart: | compile | 3m 52s | the patch passed | | +1 :green_heart: | cc | 3m 52s | the patch passed | | +1 :green_heart: | javac | 3m 52s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 0m 9s | The patch passed checkstyle in hbase-protocol | | +1 :green_heart: | checkstyle | 0m 34s | hbase-client: The patch generated 0 new + 85 unchanged - 26 fixed = 85 total (was 111) | | -1 :x: | checkstyle | 1m 22s | hbase-server: The patch generated 1 new + 449 unchanged - 10 fixed = 450 total (was 459) | | +1 :green_heart: | checkstyle | 0m 47s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 17s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 55s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 26s | the patch passed | | +1 :green_heart: | javadoc | 2m 20s | the patch passed | | +1 :green_heart: | findbugs | 13m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 43s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 27s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 1m 0s | hbase-client in the patch passed. | | -1 :x: | unit | 96m 49s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 3m 16s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 4m 16s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 2m 18s | The patch does not generate ASF License warnings. | | | | 217m 46s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestNamespaceReplication | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1114 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 016c0766d222 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1114/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/3/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/3/artifact/out/patch-unit-hbase-server.txt | | Test
[GitHub] [hbase] Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182#issuecomment-588002930 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 1s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | shadedjars | 5m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 47s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 40s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 38s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | | +1 :green_heart: | findbugs | 6m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 97m 6s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 167m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1182 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 799dccd0d6be 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1182/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/3/testReport/ | | Max. process+thread count | 6456 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1167: HBASE-18095 Zookeeper-less client connection umbrella
Apache-HBase commented on issue #1167: HBASE-18095 Zookeeper-less client connection umbrella URL: https://github.com/apache/hbase/pull/1167#issuecomment-587999623 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 3s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 95 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 47s | master passed | | +1 :green_heart: | compile | 3m 21s | master passed | | +1 :green_heart: | checkstyle | 2m 47s | master passed | | +0 :ok: | refguide | 5m 24s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 23s | master passed | | +0 :ok: | spotbugs | 1m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 26m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 34s | the patch passed | | +1 :green_heart: | compile | 3m 27s | the patch passed | | +1 :green_heart: | cc | 3m 27s | the patch passed | | +1 :green_heart: | javac | 3m 27s | the patch passed | | -1 :x: | checkstyle | 2m 41s | root: The patch generated 1 new + 753 unchanged - 96 fixed = 754 total (was 849) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 5m 24s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 31s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 9m 7s | the patch passed | | +1 :green_heart: | javadoc | 5m 31s | the patch passed | | +1 :green_heart: | findbugs | 28m 15s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 151m 6s | root in the patch failed. | | +1 :green_heart: | asflicense | 3m 42s | The patch does not generate ASF License warnings. | | | | 303m 34s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1167 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool refguide | | uname | Linux 2fa7278d949c 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1167/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/artifact/out/branch-site/book.html | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/artifact/out/diff-checkstyle-root.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/testReport/ | | Max. process+thread count | 6751 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-server hbase-mapreduce hbase-thrift . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1167/6/console |
[GitHub] [hbase] Apache-HBase commented on issue #1180: HBASE-23834. HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch.
Apache-HBase commented on issue #1180: HBASE-23834. HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch. URL: https://github.com/apache/hbase/pull/1180#issuecomment-587999218 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 39s | master passed | | +1 :green_heart: | compile | 3m 3s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 56s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 59s | the patch passed | | +1 :green_heart: | compile | 3m 2s | the patch passed | | +1 :green_heart: | javac | 3m 2s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 37s | patch has no errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 12m 39s | The patch causes 52 errors with Hadoop v3.1.2. | | +1 :green_heart: | javadoc | 2m 55s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 5m 4s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 51m 26s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.http.TestHttpServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1180/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1180 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux e4cd8897e4ff 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1180/out/precommit/personality/provided.sh | | git revision | master / 86e76c2bd9 | | Default Java | 1.8.0_181 | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1180/2/artifact/out/patch-javac-3.1.2.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1180/2/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1180/2/testReport/ | | Max. process+thread count | 665 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1180/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23859) Modify "Block locality" of RegionServer Web UI to human readable percentage
[
https://issues.apache.org/jira/browse/HBASE-23859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039616#comment-17039616
]
Hudson commented on HBASE-23859:
Results for branch branch-2.2
[build #790 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/790/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/790//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/790//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/790//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify "Block locality" of RegionServer Web UI to human readable percentage
> ---
>
> Key: HBASE-23859
> URL: https://issues.apache.org/jira/browse/HBASE-23859
> Project: HBase
> Issue Type: Improvement
>Affects Versions: master
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: image-2020-02-17-20-36-50-084.png,
> image-2020-02-17-20-38-52-990.png
>
>
> The unit of "Block locality" in Web UI just like picture 1
> !image-2020-02-17-20-36-50-084.png!
> I think we should change it to percentage unit like picture 2
> !image-2020-02-17-20-38-52-990.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587988837 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/4/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] jojochuang commented on issue #1180: HBASE-23834. HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch.
jojochuang commented on issue #1180: HBASE-23834. HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch. URL: https://github.com/apache/hbase/pull/1180#issuecomment-587986679 Not sure if it's caused by my change. Triggered a rebuild of the precommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587984665 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -1 :x: | yetus | 0m 4s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1183 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/3/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587984396 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/3/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587982596 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 13s | Docker mode activated. | | -1 :x: | yetus | 0m 4s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1183 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/2/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587981510 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587977337 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 39s | Docker mode activated. | | -1 :x: | yetus | 0m 3s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1183 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/1/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183#issuecomment-587976414 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1183/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-23865) Up flakey history from 5 to 10
[
https://issues.apache.org/jira/browse/HBASE-23865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039589#comment-17039589
]
Michael Stack edited comment on HBASE-23865 at 2/19/20 12:34 AM:
-
What I pushed on master and branch-2:
{code}
>From 86e76c2bd9597e1b02f73039ab83af7c31e45617 Mon Sep 17 00:00:00 2001
From: stack
Date: Tue, 18 Feb 2020 16:30:25 -0800
Subject: [PATCH] HBASE-23865 Up flakey history from 5 to 10
---
dev-support/flaky-tests/flaky-reporting.Jenkinsfile | 2 +-
dev-support/flaky-tests/run-flaky-tests.Jenkinsfile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
index d5e750a47e..37d557c82d 100644
--- a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
+++ b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
@@ -40,7 +40,7 @@ pipeline {
set -x
fi
declare -a flaky_args
- flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 5)
+ flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 10)
flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase-Flaky-Tests/job/${BRANCH_NAME}" --is-yetus False
--max-builds 30)
docker build -t hbase-dev-support dev-support
docker run -v "${WORKSPACE}":/hbase --workdir=/hbase
hbase-dev-support python dev-support/flaky-tests/report-flakies.py --mvn -v
"${flaky_args[@]}"
diff --git a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
index 5b0d581dd4..939bc407c7 100644
--- a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
+++ b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
@@ -61,7 +61,7 @@ pipeline {
else
echo "Skipped gathering machine environment because we couldn't
read the script to do so."
fi
-mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=3
-Dsurefire.secondPartForkCount=3
+mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=1C
-Dsurefire.secondPartForkCount=1C
else
echo "set of flaky tests is currently empty."
fi
--
2.19.1
{code}
was (Author: stack):
What I pushed on master and branch-2:
>From 86e76c2bd9597e1b02f73039ab83af7c31e45617 Mon Sep 17 00:00:00 2001
From: stack
Date: Tue, 18 Feb 2020 16:30:25 -0800
Subject: [PATCH] HBASE-23865 Up flakey history from 5 to 10
---
dev-support/flaky-tests/flaky-reporting.Jenkinsfile | 2 +-
dev-support/flaky-tests/run-flaky-tests.Jenkinsfile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
index d5e750a47e..37d557c82d 100644
--- a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
+++ b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
@@ -40,7 +40,7 @@ pipeline {
set -x
fi
declare -a flaky_args
- flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 5)
+ flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 10)
flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase-Flaky-Tests/job/${BRANCH_NAME}" --is-yetus False
--max-builds 30)
docker build -t hbase-dev-support dev-support
docker run -v "${WORKSPACE}":/hbase --workdir=/hbase
hbase-dev-support python dev-support/flaky-tests/report-flakies.py --mvn -v
"${flaky_args[@]}"
diff --git a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
index 5b0d581dd4..939bc407c7 100644
--- a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
+++ b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
@@ -61,7 +61,7 @@ pipeline {
else
echo "Skipped gathering machine environment because we couldn't
read the script to do so."
fi
-mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=3
-Dsurefire.secondPartForkCount=3
+mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=1C
-Dsurefire.secondPartForkCount=1C
else
echo "set of flaky tests is currently empty."
fi
--
2.19.1
> Up flakey history from 5 to 10
>
[jira] [Resolved] (HBASE-23865) Up flakey history from 5 to 10
[
https://issues.apache.org/jira/browse/HBASE-23865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23865.
---
Fix Version/s: 2.3.0
3.0.0
Assignee: Michael Stack
Resolution: Fixed
What I pushed on master and branch-2:
>From 86e76c2bd9597e1b02f73039ab83af7c31e45617 Mon Sep 17 00:00:00 2001
From: stack
Date: Tue, 18 Feb 2020 16:30:25 -0800
Subject: [PATCH] HBASE-23865 Up flakey history from 5 to 10
---
dev-support/flaky-tests/flaky-reporting.Jenkinsfile | 2 +-
dev-support/flaky-tests/run-flaky-tests.Jenkinsfile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
index d5e750a47e..37d557c82d 100644
--- a/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
+++ b/dev-support/flaky-tests/flaky-reporting.Jenkinsfile
@@ -40,7 +40,7 @@ pipeline {
set -x
fi
declare -a flaky_args
- flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 5)
+ flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase%20Nightly/job/${BRANCH_NAME}" --is-yetus True
--max-builds 10)
flaky_args=("${flaky_args[@]}" --urls
"${JENKINS_URL}/job/HBase-Flaky-Tests/job/${BRANCH_NAME}" --is-yetus False
--max-builds 30)
docker build -t hbase-dev-support dev-support
docker run -v "${WORKSPACE}":/hbase --workdir=/hbase
hbase-dev-support python dev-support/flaky-tests/report-flakies.py --mvn -v
"${flaky_args[@]}"
diff --git a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
index 5b0d581dd4..939bc407c7 100644
--- a/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
+++ b/dev-support/flaky-tests/run-flaky-tests.Jenkinsfile
@@ -61,7 +61,7 @@ pipeline {
else
echo "Skipped gathering machine environment because we couldn't
read the script to do so."
fi
-mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=3
-Dsurefire.secondPartForkCount=3
+mvn package "${mvn_args[@]}" -Dtest="$(cat includes.txt)"
-Dmaven.test.redirectTestOutputToFile=true -Dsurefire.firstPartForkCount=1C
-Dsurefire.secondPartForkCount=1C
else
echo "set of flaky tests is currently empty."
fi
--
2.19.1
> Up flakey history from 5 to 10
> --
>
> Key: HBASE-23865
> URL: https://issues.apache.org/jira/browse/HBASE-23865
> Project: HBase
> Issue Type: Improvement
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-HBASE-23865-Up-flakey-history-from-5-to-10.patch
>
>
> Also change the second and first part fort counts to be 1C rather than
> hardcoded 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23865) Up flakey history from 5 to 10
[ https://issues.apache.org/jira/browse/HBASE-23865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-23865: -- Attachment: 0001-HBASE-23865-Up-flakey-history-from-5-to-10.patch > Up flakey history from 5 to 10 > -- > > Key: HBASE-23865 > URL: https://issues.apache.org/jira/browse/HBASE-23865 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Priority: Major > Attachments: 0001-HBASE-23865-Up-flakey-history-from-5-to-10.patch > > > Also change the second and first part fort counts to be 1C rather than > hardcoded 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23865) Up flakey history from 5 to 10
[ https://issues.apache.org/jira/browse/HBASE-23865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039587#comment-17039587 ] Michael Stack commented on HBASE-23865: --- I'm talking about these flakies that are kept for each branch... https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html > Up flakey history from 5 to 10 > -- > > Key: HBASE-23865 > URL: https://issues.apache.org/jira/browse/HBASE-23865 > Project: HBase > Issue Type: Improvement >Reporter: Michael Stack >Priority: Major > > Also change the second and first part fort counts to be 1C rather than > hardcoded 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23865) Up flakey history from 5 to 10
Michael Stack created HBASE-23865: - Summary: Up flakey history from 5 to 10 Key: HBASE-23865 URL: https://issues.apache.org/jira/browse/HBASE-23865 Project: HBase Issue Type: Improvement Reporter: Michael Stack Also change the second and first part fort counts to be 1C rather than hardcoded 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #1183: HBASE-23767 WIP do not merge
ndimiduk opened a new pull request #1183: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1183 Rebuild our Dockerfile with support for multiple JDK versions. Try to use it with Github/precommit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk closed pull request #1171: HBASE-23767 WIP do not merge
ndimiduk closed pull request #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587972369 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -1 :x: | yetus | 0m 5s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/19/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/19/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587972193 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/19/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22710) Wrong result in one case of scan that use raw and versions and filter together
[
https://issues.apache.org/jira/browse/HBASE-22710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039585#comment-17039585
]
Geoffrey Jacoby commented on HBASE-22710:
-
[~zhangduo], [~filtertip] Any update on HBASE-22710?
We've recently encountered this bug in the 2.x-based branches of Phoenix where
our secondary-index building code recently changed to use a Phoenix-specific
Filter (SkipScanFilter) and a raw scan. See PHOENIX-5736. The change to use the
SkipScanFilter gave us a nice perf boost, and we'd really prefer to avoid
reverting if we can avoid it.
To Duo's point about whether Filters should support raw scans: I think the
answer has to be yes, because there are valid cases where someone might want to
use a _particular_ Filter or Filters and a raw scan. However, it also makes
sense to me that not _all_ Filters may be valid in a raw Scan context.
As separate work, perhaps we could add a supportsRawScan() method to Filter
(with a default impl of true in FilterBase for backwards compatibility), and if
the query matcher catches a raw Scan using a Filter with supportsRawScan
returning false, the client gets back a helpful non-retriable exception. That
way at least there are no surprises, and no silently-returning-wrong-answers,
which is the current behavior.
> Wrong result in one case of scan that use raw and versions and filter
> together
> ---
>
> Key: HBASE-22710
> URL: https://issues.apache.org/jira/browse/HBASE-22710
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 2.0.0
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
>
> create 'testScanRaw',\{NAME => 'f', VERSIONS => 1}
>
> put 'testScanRaw','r1','f:q','1'
> put 'testScanRaw','r1','f:q','2'
> put 'testScanRaw','r1','f:q','3'
>
> hbase(main):005:0> scan 'testScanRaw',\{RAW => true, STARTROW => 'r1',
> STOPROW=>'r1',VERSIONS=>2}
> ROW COLUMN+CELL
> r1 column=f:q, timestamp=1563430154757, value=3
> r1 column=f:q, timestamp=1563430153120, value=2
>
> hbase(main):006:0> scan 'testScanRaw',\{RAW => true, STARTROW => 'r1',
> STOPROW=>'r1',VERSIONS=>2,FILTER => "(QualifierFilter (=, 'binary:q'))"}
> ROW COLUMN+CELL
> r1 column=f:q, timestamp=1563430154757, value=3
>
> BTW,the result is right in hbase1.2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587971456 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -1 :x: | yetus | 0m 3s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/18/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/18/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587971305 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/18/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587968309 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/17/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587968484 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -1 :x: | yetus | 0m 4s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/17/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/17/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587966268 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 59s | Docker mode activated. | | -1 :x: | yetus | 0m 5s | Unprocessed flag(s): --findbugs-strict-precheck --findbugs-home | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/16/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/16/console | | versions | git=2.24.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587965355 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/16/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-23767) Add JDK11 to the Jenkinsfile(s) for matrix builds
[ https://issues.apache.org/jira/browse/HBASE-23767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-23767: Assignee: Nick Dimiduk > Add JDK11 to the Jenkinsfile(s) for matrix builds > - > > Key: HBASE-23767 > URL: https://issues.apache.org/jira/browse/HBASE-23767 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > We already test against multiple JDK versions in a handful of places. Let's > get JDK11 added to the mix. Applies to pre-commit and nightly... anywhere > else? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count
[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039560#comment-17039560 ] Mark Robert Miller edited comment on HBASE-23779 at 2/18/20 11:45 PM: -- In the meantime, some some ideas on JVM args that be able to help the situation a little. -XX:-UseContainerSupport - when running in docker, query docker for hardware info, not the host. I have not played with it, but makes sense to me. -XX:ActiveProcessorCount - fake the active processer count to 1 - when running multiple jvms, you don't want each one to think it should grab 32 gc threads on your 32 hyperthreaded core machine. I'd much prefer to run 32 JVMs and have 32 gc threads. -XX:+AlwaysPreTouch -XX:+UseTransparentHugePages - given so much JVM startup and shutdowns, some things might help more when there is more JVM reuse - so much time and cost is in creating large heap jvms and spinning up lots of unused resources and threads that it's hard to win much with good tuning - but huge pages, always a nice win. -XX:+UseParallelGC -XX:TieredStopAtLevel=1 - even when G1 is available, this is gold standard for test speed. Maybe with like -XX:ParallelGCThreads=1 if not using ActiveProcessorCout. Probably not a big advantage when each med/large JVM is spinning up thousands of useless threads that eat ram, heap and os resources, but something, and probably pretty nice for small test single jvm runs now. It's also generally best to pin Xms and Xmx vs eat all the resizing cost. I suggest the opposite above given the RAM reqs - to keep the JVMs that don't need it from sucking up so much RAM unnecessarily - but with good reuse, you want to pin them. Most of the tests don't need these huge heaps and limits - I think that having them for the bad apples and outliers just allows the 95% of tests to easily be wasteful and misbehave. * Note: UseContainerSupport is enabled by default - the above disables - so guess that should already be in affect. was (Author: markrmiller): In the meantime, some some ideas on JVM args that be able to help the situation a little. -XX:-UseContainerSupport - when running in docker, query docker for hardware info, not the host. I have not played with it, but makes sense to me. -XX:ActiveProcessorCount - fake the active processer count to 1 - when running multiple jvms, you don't want each one to think it should grab 32 gc threads on your 32 hyperthreaded core machine. I'd much prefer to run 32 JVMs and have 32 gc threads. -XX:+AlwaysPreTouch -XX:+UseTransparentHugePages - given so much JVM startup and shutdowns, some things might help more when there is more JVM reuse - so much time and cost is in creating large heap jvms and spinning up lots of unused resources and threads that it's hard to win much with good tuning - but huge pages, always a nice win. -XX:+UseParallelGC -XX:TieredStopAtLevel=1 - even when G1 is available, this is gold standard for test speed. Maybe with like -XX:ParallelGCThreads=1 if not using ActiveProcessorCout. Probably not a big advantage when each med/large JVM is spinning up thousands of useless threads that eat ram, heap and os resources, but something, and probably pretty nice for small test single jvm runs now. It's also generally best to pin Xms and Xmx vs eat all the resizing cost. I suggest the opposite above given the RAM reqs - to keep the JVMs that don't need it from sucking up so much RAM unnecessarily - but with good reuse, you want to pin them. Most of the tests don't need these huge heaps and limits - I think that having them for the bad apples and outliers just allows the 95% of tests to easily be wasteful and misbehave. > Up the default fork count to make builds complete faster; make count relative > to CPU count > -- > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587960962 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 0m 4s | root in master failed. | | -1 :x: | compile | 0m 4s | root in master failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 3s | root in master failed with JDK v2019-10-15. | | +1 :green_heart: | shadedjars | 0m 1s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v2019-10-15. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in the patch failed. | | -1 :x: | compile | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javac | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 3s | root in the patch failed with JDK v2019-10-15. | | -1 :x: | javac | 0m 3s | root in the patch failed with JDK v2019-10-15. | | +1 :green_heart: | hadolint | 0m 2s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 0m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 0m 3s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v2019-10-15. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 3s | root in the patch failed. | | +0 :ok: | asflicense | 0m 4s | ASF License check generated no output? | | | | 6m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 809b6ac48dab 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1171/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 2019-10-15 | | Multi-JDK versions | /usr/lib/jvm/java-1.8-openjdk:1.8.0_242 /usr/lib/jvm/java-11-openjdk:2019-10-15 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/branch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/branch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/branch-compile-root-jdk2019-10-15.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/branch-javadoc-root-jdk1.8.0_242.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/branch-javadoc-root-jdk2019-10-15.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/artifact/out/patch-compile-root-jdk2019-10-15.txt | | javac |
[jira] [Comment Edited] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count
[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039560#comment-17039560 ] Mark Robert Miller edited comment on HBASE-23779 at 2/18/20 11:40 PM: -- In the meantime, some some ideas on JVM args that be able to help the situation a little. -XX:-UseContainerSupport - when running in docker, query docker for hardware info, not the host. I have not played with it, but makes sense to me. -XX:ActiveProcessorCount - fake the active processer count to 1 - when running multiple jvms, you don't want each one to think it should grab 32 gc threads on your 32 hyperthreaded core machine. I'd much prefer to run 32 JVMs and have 32 gc threads. -XX:+AlwaysPreTouch -XX:+UseTransparentHugePages - given so much JVM startup and shutdowns, some things might help more when there is more JVM reuse - so much time and cost is in creating large heap jvms and spinning up lots of unused resources and threads that it's hard to win much with good tuning - but huge pages, always a nice win. -XX:+UseParallelGC -XX:TieredStopAtLevel=1 - even when G1 is available, this is gold standard for test speed. Maybe with like -XX:ParallelGCThreads=1 if not using ActiveProcessorCout. Probably not a big advantage when each med/large JVM is spinning up thousands of useless threads that eat ram, heap and os resources, but something, and probably pretty nice for small test single jvm runs now. It's also generally best to pin Xms and Xmx vs eat all the resizing cost. I suggest the opposite above given the RAM reqs - to keep the JVMs that don't need it from sucking up so much RAM unnecessarily - but with good reuse, you want to pin them. Most of the tests don't need these huge heaps and limits - I think that having them for the bad apples and outliers just allows the 95% of tests to easily be wasteful and misbehave. was (Author: markrmiller): In the meantime, some some ideas on JVM args that be able to help the situation a little. -XX:-UseContainerSupport - when running in docker, query docker for hardware info, not the host. I have no played with it, but makes sense to me. -XX:ActiveProcessorCount - fake the active process count to 1 - when running multiple jvms, you don't want each one to think it should grab 32 gc threads on your 32 hyperthreaded core machine. I'd much prefer to run 32 JVMs and have 32 gc threads. -XX:+AlwaysPreTouch -XX:+UseTransparentHugePages - given so much JVM startup and shutdowns, some things might help more when there is more JVM reuse - so much time and cost is in creating large heap jvms and spinning up lots of unused resources and threads that it's hard to win much with good tuning - but huge pages, always a nice win. -XX:+UseParallelGC -XX:TieredStopAtLevel=1 - even when G1 is available, this is gold standard for test speed. Maybe with like -XX:ParallelGCThreads=1 if no using ActiveProcessorCout. Probably not a big advantage when each med/large JVM is spinning up thousands of useless threads that eat ram, heap and os resources, but something, and probably pretty nice for small test single jvm runs now. It's also generally best to pin Xms and Xmx vs eat all the resizing cost. I suggest the opposite above given the RAM reqs - to keep the JVMs that don't need it from sucking up so much RAM unnecessarily - but with good reuse, you want to pin them. Most of the tests don't need these huge heaps and limits - I think that having them for the bad apples and outliers just allows the 95% of tests to easily be wasteful and misbehave. > Up the default fork count to make builds complete faster; make count relative > to CPU count > -- > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a look > see. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count
[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039560#comment-17039560 ] Mark Robert Miller commented on HBASE-23779: In the meantime, some some ideas on JVM args that be able to help the situation a little. -XX:-UseContainerSupport - when running in docker, query docker for hardware info, not the host. I have no played with it, but makes sense to me. -XX:ActiveProcessorCount - fake the active process count to 1 - when running multiple jvms, you don't want each one to think it should grab 32 gc threads on your 32 hyperthreaded core machine. I'd much prefer to run 32 JVMs and have 32 gc threads. -XX:+AlwaysPreTouch -XX:+UseTransparentHugePages - given so much JVM startup and shutdowns, some things might help more when there is more JVM reuse - so much time and cost is in creating large heap jvms and spinning up lots of unused resources and threads that it's hard to win much with good tuning - but huge pages, always a nice win. -XX:+UseParallelGC -XX:TieredStopAtLevel=1 - even when G1 is available, this is gold standard for test speed. Maybe with like -XX:ParallelGCThreads=1 if no using ActiveProcessorCout. Probably not a big advantage when each med/large JVM is spinning up thousands of useless threads that eat ram, heap and os resources, but something, and probably pretty nice for small test single jvm runs now. It's also generally best to pin Xms and Xmx vs eat all the resizing cost. I suggest the opposite above given the RAM reqs - to keep the JVMs that don't need it from sucking up so much RAM unnecessarily - but with good reuse, you want to pin them. Most of the tests don't need these huge heaps and limits - I think that having them for the bad apples and outliers just allows the 95% of tests to easily be wasteful and misbehave. > Up the default fork count to make builds complete faster; make count relative > to CPU count > -- > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a look > see. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587959250 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/15/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22827) Expose multi-region merge in shell and Admin API
[ https://issues.apache.org/jira/browse/HBASE-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039555#comment-17039555 ] Sakthi commented on HBASE-22827: Hey [~stack], I think this would be a nice feature to be added in our 2.1 and 2.2 as well. 2.2 already has everything set in place so we should be fine there. 2.1 needs to have HBASE-22040 for us to be able to land this as well. What do you think? > Expose multi-region merge in shell and Admin API > > > Key: HBASE-22827 > URL: https://issues.apache.org/jira/browse/HBASE-22827 > Project: HBase > Issue Type: Improvement > Components: Admin, shell >Reporter: Michael Stack >Assignee: Sakthi >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: hbase-22827.branch-2.001.patch > > > HBASE-22777 adds being able to merge more than two regions at once. It is > only available internally currently for use by hbck2 doing fixup of overlaps > in hbase:meta. This issue is about exposing it via the Admin Interface and in > turn, via the shell. Probably best if old two region merge method is > deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22040) Add mergeRegionsAsync with a List of region names method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039553#comment-17039553 ] Sakthi commented on HBASE-22040: Hey [~zhangduo] any reason why this one couldn't make it to branch-2.1? > Add mergeRegionsAsync with a List of region names method in AsyncAdmin > -- > > Key: HBASE-22040 > URL: https://issues.apache.org/jira/browse/HBASE-22040 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0 > > Attachments: HBASE-22040-branch-2.patch, HBASE-22040-v1.patch, > HBASE-22040-v2.patch, HBASE-22040-v3.patch, HBASE-22040.patch > > > Although we only support merging two regions until now, but the rpc protocol > does support passing more than two regions to master. > So I think we should provide the methods, but need to add comments to say > that for now we only support merging two regions so do not try to pass more > than two regions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] brfrn169 commented on issue #1114: HBASE-23146 Support CheckAndMutate with multiple conditions
brfrn169 commented on issue #1114: HBASE-23146 Support CheckAndMutate with multiple conditions URL: https://github.com/apache/hbase/pull/1114#issuecomment-587946687 I just force-pushed the new patch with changing the new API. I added a new method which accepts a row and filter as parameters to construct a Builder. @Apache9 Could you please review it when you get a chance? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22749) Distributed MOB compactions
[
https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039530#comment-17039530
]
Hudson commented on HBASE-22749:
Results for branch HBASE-22749
[build #7 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/7/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/7//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/7//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22749/7//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Distributed MOB compactions
>
>
> Key: HBASE-22749
> URL: https://issues.apache.org/jira/browse/HBASE-22749
> Project: HBase
> Issue Type: New Feature
> Components: mob
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Major
> Attachments: HBASE-22749-branch-2.2-v4.patch,
> HBASE-22749-master-v1.patch, HBASE-22749-master-v2.patch,
> HBASE-22749-master-v3.patch, HBASE-22749-master-v4.patch,
> HBase-MOB-2.0-v3.0.pdf
>
>
> There are several drawbacks in the original MOB 1.0 (Moderate Object
> Storage) implementation, which can limit the adoption of the MOB feature:
> # MOB compactions are executed in a Master as a chore, which limits
> scalability because all I/O goes through a single HBase Master server.
> # Yarn/Mapreduce framework is required to run MOB compactions in a scalable
> way, but this won’t work in a stand-alone HBase cluster.
> # Two separate compactors for MOB and for regular store files and their
> interactions can result in a data loss (see HBASE-22075)
> The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible
> implementation, which is free of the above drawbacks and can be used as a
> drop in replacement in existing MOB deployments. So, these are design goals
> of a MOB 2.0:
> # Make MOB compactions scalable without relying on Yarn/Mapreduce framework
> # Provide unified compactor for both MOB and regular store files
> # Make it more robust especially w.r.t. to data losses.
> # Simplify and reduce the overall MOB code.
> # Provide 100% compatible implementation with MOB 1.0.
> # No migration of data should be required between MOB 1.0 and MOB 2.0 - just
> software upgrade.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23146) Support CheckAndMutate with multiple conditions
[ https://issues.apache.org/jira/browse/HBASE-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-23146: - Summary: Support CheckAndMutate with multiple conditions (was: Support CheckAndMutate with multi conditions) > Support CheckAndMutate with multiple conditions > --- > > Key: HBASE-23146 > URL: https://issues.apache.org/jira/browse/HBASE-23146 > Project: HBase > Issue Type: New Feature >Reporter: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Currently, checkAndMutate supports only single condition. Supporting > checkAndMutate with multi conditions is useful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] busbey commented on issue #1179: HBASE-23861. Reconcile Hadoop version.
busbey commented on issue #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#issuecomment-587943008 Looks great to me This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587941716 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in master failed. | | -1 :x: | compile | 0m 3s | root in master failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 3s | root in master failed with JDK v2019-10-15. | | +1 :green_heart: | shadedjars | 0m 1s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v2019-10-15. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in the patch failed. | | -1 :x: | compile | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javac | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 3s | root in the patch failed with JDK v2019-10-15. | | -1 :x: | javac | 0m 3s | root in the patch failed with JDK v2019-10-15. | | +1 :green_heart: | hadolint | 0m 2s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 0m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 0m 3s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v2019-10-15. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 3s | root in the patch failed. | | +0 :ok: | asflicense | 0m 3s | ASF License check generated no output? | | | | 3m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux d0ecd2106f20 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1171/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 2019-10-15 | | Multi-JDK versions | /usr/lib/jvm/java-1.8-openjdk:1.8.0_242 /usr/lib/jvm/java-11-openjdk:2019-10-15 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/branch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/branch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/branch-compile-root-jdk2019-10-15.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/branch-javadoc-root-jdk1.8.0_242.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/branch-javadoc-root-jdk2019-10-15.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/artifact/out/patch-compile-root-jdk2019-10-15.txt | | javac |
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587940763 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/14/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23146) Support CheckAndMutate with multi conditions
[ https://issues.apache.org/jira/browse/HBASE-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-23146: - Fix Version/s: 2.3.0 3.0.0 > Support CheckAndMutate with multi conditions > > > Key: HBASE-23146 > URL: https://issues.apache.org/jira/browse/HBASE-23146 > Project: HBase > Issue Type: New Feature >Reporter: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Currently, checkAndMutate supports only single condition. Supporting > checkAndMutate with multi conditions is useful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1179: HBASE-23861. Reconcile Hadoop version.
Apache-HBase commented on issue #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#issuecomment-587935028 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 43s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 49s | master passed | | +1 :green_heart: | compile | 3m 21s | master passed | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 30s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 31s | the patch passed | | +1 :green_heart: | compile | 3m 21s | the patch passed | | +1 :green_heart: | javac | 3m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 29s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 152m 13s | root in the patch passed. | | +1 :green_heart: | asflicense | 1m 1s | The patch does not generate ASF License warnings. | | | | 215m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1179 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 90ec5b7ba16a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1179/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/2/testReport/ | | Max. process+thread count | 7255 (vs. ulimit of 1) | | modules | C: hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587926618 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +0 :ok: | hadolint | 0m 0s | hadolint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in master failed. | | -1 :x: | compile | 0m 4s | root in master failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 4s | root in master failed with JDK v2019-10-15. | | +1 :green_heart: | shadedjars | 0m 1s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v2019-10-15. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in the patch failed. | | -1 :x: | compile | 0m 4s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javac | 0m 4s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 4s | root in the patch failed with JDK v2019-10-15. | | -1 :x: | javac | 0m 4s | root in the patch failed with JDK v2019-10-15. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 0m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 0m 4s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 4s | root in the patch failed with JDK v2019-10-15. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 3s | root in the patch failed. | | +0 :ok: | asflicense | 0m 3s | ASF License check generated no output? | | | | 2m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 9aedce99d280 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1171/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 2019-10-15 | | Multi-JDK versions | /usr/lib/jvm/java-1.8-openjdk:1.8.0_242 /usr/lib/jvm/java-11-openjdk:2019-10-15 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/branch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/branch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/branch-compile-root-jdk2019-10-15.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/branch-javadoc-root-jdk1.8.0_242.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/branch-javadoc-root-jdk2019-10-15.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/artifact/out/patch-compile-root-jdk2019-10-15.txt | | javac |
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587925841 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/13/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587925246 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shellcheck | 0m 0s | Shellcheck was not available. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +0 :ok: | hadolint | 0m 0s | hadolint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in master failed. | | -1 :x: | compile | 0m 4s | root in master failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 4s | root in master failed with JDK v2019-10-15. | | +1 :green_heart: | shadedjars | 0m 1s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 3s | root in master failed with JDK v2019-10-15. | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 0m 3s | root in the patch failed. | | -1 :x: | compile | 0m 4s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javac | 0m 4s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | compile | 0m 4s | root in the patch failed with JDK v2019-10-15. | | -1 :x: | javac | 0m 4s | root in the patch failed with JDK v2019-10-15. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 0m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 0m 4s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 3s | root in the patch failed with JDK v1.8.0_242. | | -1 :x: | javadoc | 0m 4s | root in the patch failed with JDK v2019-10-15. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 3s | root in the patch failed. | | +0 :ok: | asflicense | 0m 4s | ASF License check generated no output? | | | | 3m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1171 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux f8cac34295c7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1171/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 2019-10-15 | | Multi-JDK versions | /usr/lib/jvm/java-1.8-openjdk:1.8.0_242 /usr/lib/jvm/java-11-openjdk:2019-10-15 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/branch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/branch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/branch-compile-root-jdk2019-10-15.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/branch-javadoc-root-jdk1.8.0_242.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/branch-javadoc-root-jdk2019-10-15.txt | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/patch-compile-root-jdk1.8.0_242.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/artifact/out/patch-compile-root-jdk2019-10-15.txt | | javac |
[GitHub] [hbase] Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge
Apache-HBase commented on issue #1171: HBASE-23767 WIP do not merge URL: https://github.com/apache/hbase/pull/1171#issuecomment-587923838 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1171/12/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039406#comment-17039406
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #80 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/80/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/80//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/80//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/80//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.7.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23795) Enable all tests to be run in parallel on reused JVMs.
[ https://issues.apache.org/jira/browse/HBASE-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039393#comment-17039393 ] Mark Robert Miller commented on HBASE-23795: I've started making some progress here. With some tuning, these large tests are not so large after all. I have to do some work to make sure they clean up after themselves so that JVMs dont get dirtier and dirtier over time, but all of these tests can run in parallel and much faster than they are now. > Enable all tests to be run in parallel on reused JVMs. > -- > > Key: HBASE-23795 > URL: https://issues.apache.org/jira/browse/HBASE-23795 > Project: HBase > Issue Type: Wish >Reporter: Mark Robert Miller >Priority: Major > > I'd like to be able to run HBase tests in under 30-40 minutes on good > parallel hardware. > It will require some small changes / fixes for that wish to come true. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] busbey commented on issue #921: HBASE-22749: Distributed MOB compactions
busbey commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-587650528 qabot looks great. I got some noise on my nightly branch. checking on it today. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on issue #1179: HBASE-23861. Reconcile Hadoop version.
busbey commented on issue #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#issuecomment-587649367 I reran the qabot. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22827) Expose multi-region merge in shell and Admin API
[
https://issues.apache.org/jira/browse/HBASE-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039332#comment-17039332
]
Hudson commented on HBASE-22827:
Results for branch branch-2
[build #2482 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2482/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2482//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2482//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2482//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Expose multi-region merge in shell and Admin API
>
>
> Key: HBASE-22827
> URL: https://issues.apache.org/jira/browse/HBASE-22827
> Project: HBase
> Issue Type: Improvement
> Components: Admin, shell
>Reporter: Michael Stack
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: hbase-22827.branch-2.001.patch
>
>
> HBASE-22777 adds being able to merge more than two regions at once. It is
> only available internally currently for use by hbck2 doing fixup of overlaps
> in hbase:meta. This issue is about exposing it via the Admin Interface and in
> turn, via the shell. Probably best if old two region merge method is
> deprecated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-23854) Documentation update of external_apis.adoc#example-scala-code
[
https://issues.apache.org/jira/browse/HBASE-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Rodionov reassigned HBASE-23854:
-
Assignee: (was: Vladimir Rodionov)
> Documentation update of external_apis.adoc#example-scala-code
> -
>
> Key: HBASE-23854
> URL: https://issues.apache.org/jira/browse/HBASE-23854
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Michael Heil
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-23854.patch
>
>
> Update the Example Scala Code in the Reference Guide as it contains
> deprecated content such as
> * new HBaseConfiguration()
> * new HTable(conf, "mytable")
> * add(Bytes.toBytes("ids"),Bytes.toBytes("id1"),Bytes.toBytes("one"))
> Replace it with:
> * HBaseConfiguration.create()
> * TableName.valueOf({color:#6a8759}"mytable"{color})
> *
> addColumn(Bytes.toBytes({color:#6a8759}"ids"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"id1"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"one"{color}))
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r379974849
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,154 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
Review comment:
I am not in favor of removing Lmax Disruptor completely, in fact when it
comes to ring buffer in Java, Disruptor provides the best performance due to
mechanical sympathy. As of now, our usecase might not be complex enough to have
significant perf improvement using Disruptor but this is supposed to get more
data e.g. tooLong records too in ring buffer, more usecases - keeping too long
and too slow together etc. Thinking of all that, we should definitely have
Disruptor here.
At consumer side, EvictingQueue makes sense because as I mentioned, in
future, for different usecases, we can have different DataStructures e.g.
separate EvictingQueue for tooSlow, tooLarge etc and additional logic if
required for filter purpose etc. All these consumer logic can change based on
requirement specifically from Get API POV, how we want to retrieve them. But if
we remove the Disruptor, we will have to start dealing with consumer specific
logic in producer i.e. send tooSlow event to one specific queue, send tooLarge
or some other event to other queue etc which doesn't seem right.
Hence, I am not onboard with removal of Disruptor completely which is the
primary entry point for all logs of our interest.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182#issuecomment-587549572 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 58s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | | +0 :ok: | spotbugs | 4m 32s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 30s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 28s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedjars | 5m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 18m 8s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | | +1 :green_heart: | findbugs | 4m 37s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 105m 48s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 43s | The patch does not generate ASF License warnings. | | | | 169m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1182 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 66885089d599 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1182/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/2/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/2/testReport/ | | Max. process+thread count | 6628 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1181: HBASE-23862 Fix flaky TestSnapshotFromMaster in 1.x versions
Apache-HBase commented on issue #1181: HBASE-23862 Fix flaky TestSnapshotFromMaster in 1.x versions URL: https://github.com/apache/hbase/pull/1181#issuecomment-587445071 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 23s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | compile | 0m 43s | branch-1 passed with JDK v1.7.0_252 | | +1 :green_heart: | checkstyle | 1m 37s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.7.0_252 | | +0 :ok: | spotbugs | 2m 52s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 49s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 56s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 32s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | findbugs | 2m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 126m 39s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 166m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1181 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 357da37bfcc0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1181/out/precommit/personality/provided.sh | | git revision | branch-1 / b9c676c | | Default Java | 1.7.0_252 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/2/testReport/ | | Max. process+thread count | 4795 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23857) Add 2.1.9 to download page
[
https://issues.apache.org/jira/browse/HBASE-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039037#comment-17039037
]
Hudson commented on HBASE-23857:
Results for branch master
[build #1634 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1634/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add 2.1.9 to download page
> --
>
> Key: HBASE-23857
> URL: https://issues.apache.org/jira/browse/HBASE-23857
> Project: HBase
> Issue Type: Sub-task
> Components: website
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23859) Modify "Block locality" of RegionServer Web UI to human readable percentage
[
https://issues.apache.org/jira/browse/HBASE-23859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039038#comment-17039038
]
Hudson commented on HBASE-23859:
Results for branch master
[build #1634 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1634/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Modify "Block locality" of RegionServer Web UI to human readable percentage
> ---
>
> Key: HBASE-23859
> URL: https://issues.apache.org/jira/browse/HBASE-23859
> Project: HBase
> Issue Type: Improvement
>Affects Versions: master
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: image-2020-02-17-20-36-50-084.png,
> image-2020-02-17-20-38-52-990.png
>
>
> The unit of "Block locality" in Web UI just like picture 1
> !image-2020-02-17-20-36-50-084.png!
> I think we should change it to percentage unit like picture 2
> !image-2020-02-17-20-38-52-990.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23838) Adding debug logging to a few ExportSnapshot tests
[
https://issues.apache.org/jira/browse/HBASE-23838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039036#comment-17039036
]
Hudson commented on HBASE-23838:
Results for branch master
[build #1634 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1634/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1634//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Adding debug logging to a few ExportSnapshot tests
> --
>
> Key: HBASE-23838
> URL: https://issues.apache.org/jira/browse/HBASE-23838
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Trivial
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch,
> 0001-HBASE-23838-Adding-debug-logging-to-a-few-ExportSnap.patch
>
>
> Bit of extra logging in tests to help me diagnose where they are going wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service
wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service URL: https://github.com/apache/hbase/pull/1174#discussion_r380624073 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java ## @@ -2458,6 +2443,48 @@ public FileArchiveNotificationResponse reportFileArchival(RpcController controll // HBCK Services + @Override + public MergeTableRegionsResponse mergeRegions(RpcController c, MergeTableRegionsRequest request) Review comment: Found a bit confusing to have two merge methods. Should we just simplify this Master API by keeping the new merge method only? I mean, if it is possible to do merges without master fully initialised, then why should we bother enforcing it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service
wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service URL: https://github.com/apache/hbase/pull/1174#discussion_r380625259 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Hbck.java ## @@ -143,4 +143,17 @@ * Fix Meta. */ void fixMeta() throws IOException; + + /** Review comment: Maybe worth clarify what the `List regions` is expected to be. Regions name? Encoded name? Id? Any of those? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service
wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service URL: https://github.com/apache/hbase/pull/1174#discussion_r380623924 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java ## @@ -2458,6 +2443,48 @@ public FileArchiveNotificationResponse reportFileArchival(RpcController controll // HBCK Services + @Override + public MergeTableRegionsResponse mergeRegions(RpcController c, MergeTableRegionsRequest request) Review comment: Is the overridden `mergeRegions` method still needed elsewhere? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service
wchevreuil commented on a change in pull request #1174: HBASE-23827 [HBCK2] Add mergeRegions method to hbck service URL: https://github.com/apache/hbase/pull/1174#discussion_r380623924 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java ## @@ -2458,6 +2443,48 @@ public FileArchiveNotificationResponse reportFileArchival(RpcController controll // HBCK Services + @Override + public MergeTableRegionsResponse mergeRegions(RpcController c, MergeTableRegionsRequest request) Review comment: Is the overridden `mergeRegions` method still needed elsewhere? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache-HBase commented on issue #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182#issuecomment-587402110 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 59s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 5m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | | +0 :ok: | spotbugs | 4m 34s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 32s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 47s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | -1 :x: | checkstyle | 1m 15s | hbase-server: The patch generated 2 new + 116 unchanged - 0 fixed = 118 total (was 116) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 36s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | | +1 :green_heart: | findbugs | 5m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 101m 24s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 165m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1182 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ba7df4c35510 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1182/out/precommit/personality/provided.sh | | git revision | master / c4fb1cd234 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/1/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/1/testReport/ | | Max. process+thread count | 6800 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1182/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1181: HBASE-23862 Fix flaky TestSnapshotFromMaster in 1.x versions
Apache-HBase commented on issue #1181: HBASE-23862 Fix flaky TestSnapshotFromMaster in 1.x versions URL: https://github.com/apache/hbase/pull/1181#issuecomment-587379417 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 25s | branch-1 passed | | +1 :green_heart: | compile | 0m 42s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | compile | 0m 43s | branch-1 passed with JDK v1.7.0_252 | | +1 :green_heart: | checkstyle | 1m 36s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 58s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.7.0_252 | | +0 :ok: | spotbugs | 2m 50s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 46s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 54s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 53s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 0m 39s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | findbugs | 2m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 128m 31s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 174m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1181 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux cc0d232e09e2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1181/out/precommit/personality/provided.sh | | git revision | branch-1 / b9c676c | | Default Java | 1.7.0_252 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/1/testReport/ | | Max. process+thread count | 4587 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1181/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache9 commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380564544
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
But if in the future we schedule these procedures in other places, we need
to add the check there...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23853) [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits
[
https://issues.apache.org/jira/browse/HBASE-23853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038939#comment-17038939
]
Hudson commented on HBASE-23853:
Results for branch branch-2
[build #2481 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2481/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2481//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2481//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2481//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits
> --
>
> Key: HBASE-23853
> URL: https://issues.apache.org/jira/browse/HBASE-23853
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23853-Flakey-Test-TestBlockEvictionFromClient-.patch, Screen Shot
> 2020-02-15 at 2.36.47 PM.png
>
>
> Fails 12-25% of the time in the GCE build.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-23864: -- Assignee: Guanghao Zhang > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-23864.PNG > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] infraio commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
infraio commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380534011
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
Yes. I checked this. Now only two place can schedule split/merge task. One
is the rpc request of master. Another one is from RS's report.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache9 commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380532313
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
But split and merge can be scheduled by background task? And also, if you
remove the checks here, then you need to add the check everywhere...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
infraio commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380530802
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
I thought the race is not a big deal? And our admin client never have this
guarantee: interrupt the schduled Split/Merge after split/merge is disabled.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] binlijin commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
binlijin commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380524138
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
Agree with Duo, others are good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
Apache9 commented on a change in pull request #1182: HBASE-23864 No need to
submit SplitTableRegionProcedure/MergeTableReg…
URL: https://github.com/apache/hbase/pull/1182#discussion_r380521047
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -453,13 +453,6 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
throw new MergeRegionException("Skip merging regions " +
RegionInfo.getShortNameToLog(regionsToMerge) + ", because we are
snapshotting " + tn);
}
-if
(!env.getMasterServices().isSplitOrMergeEnabled(MasterSwitchType.MERGE)) {
Review comment:
You can add extra checks before scheduling the procedure to make the request
fail fast, but I do not think we should remove the checks in the procedure?
There could be race, that when scheduling, split and merge are enabled but when
you actually execute the procedure, the flags are turned to disabled...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] infraio opened a new pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
infraio opened a new pull request #1182: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1182 …ionsProcedure when split/merge is disabled This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23864: --- Summary: No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled (was: No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch is disabled) > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Priority: Major > Attachments: HBASE-23864.PNG > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
