[jira] [Updated] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23864: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2, branch-2.2 and branch-2.1. Pushed the addendum patch to master. > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10 > > Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG, > HBASE-23864.branch-2.1.002.patch, HBASE-23864.branch-2.1.003.patch, > HBASE-23864.branch-2.1.patch > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045193#comment-17045193 ] Guanghao Zhang commented on HBASE-23864: The failed ut not related. Let me commit the patch to branch-2.1, too. > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10 > > Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG, > HBASE-23864.branch-2.1.002.patch, HBASE-23864.branch-2.1.003.patch, > HBASE-23864.branch-2.1.patch > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code)
Apache-HBase commented on issue #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code) URL: https://github.com/apache/hbase/pull/1210#issuecomment-591260477 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 1m 3s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 24s | master passed | | +1 :green_heart: | compile | 4m 50s | master passed | | +1 :green_heart: | checkstyle | 3m 15s | master passed | | +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 10m 7s | master passed | | +0 :ok: | spotbugs | 0m 29s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 45s | branch/hbase-shaded no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 17s | branch/hbase-shaded/hbase-shaded-client no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 11s | branch/hbase-shaded/hbase-shaded-with-hadoop-check-invariants no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 20s | branch/hbase-shaded/hbase-shaded-check-invariants no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 29s | branch/hbase-shaded/hbase-shaded-testing-util no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 2m 34s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 31s | root in the patch failed. | | -1 :x: | compile | 0m 12s | root in the patch failed. | | -1 :x: | cc | 0m 12s | root in the patch failed. | | -1 :x: | javac | 0m 12s | root in the patch failed. | | -1 :x: | checkstyle | 0m 11s | The patch fails to run checkstyle in root | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 11s | The patch has no ill-formed XML file. | | -1 :x: | shadedjars | 0m 29s | patch has 11 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 0m 12s | The patch causes 10 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 0m 22s | The patch causes 10 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 0m 33s | The patch causes 10 errors with Hadoop v3.1.2. | | -1 :x: | hbaseprotoc | 0m 12s | hbase-shaded in the patch failed. | | -1 :x: | hbaseprotoc | 0m 14s | root in the patch failed. | | -1 :x: | hbaseprotoc | 0m 8s | hbase-client in the patch failed. | | -1 :x: | hbaseprotoc | 0m 8s | hbase-common in the patch failed. | | -1 :x: | hbaseprotoc | 0m 9s | hbase-external-blockcache in the patch failed. | | -1 :x: | hbaseprotoc | 0m 9s | hbase-it in the patch failed. | | -1 :x: | hbaseprotoc | 0m 9s | hbase-mapreduce in the patch failed. | | -1 :x: | hbaseprotoc | 0m 8s | hbase-protocol-shaded in the patch failed. | | -1 :x: | hbaseprotoc | 0m 9s | hbase-server in the patch failed. | | -1 :x: | hbaseprotoc | 0m 8s | hbase-shaded-client in the patch failed. | | -1 :x: | hbaseprotoc | 0m 9s | hbase-shaded-testing-util in the patch failed. | | -1 :x: | hbaseprotoc | 0m 10s | hbase-zookeeper in the patch failed. | | -1 :x: | javadoc | 0m 10s | hbase-shaded in the patch failed. | | -1 :x: | javadoc | 0m 10s | root in the patch failed. | | -1 :x: | javadoc | 0m 8s | hbase-client in the patch failed. | | -1 :x: | javadoc | 0m 8s | hbase-common in the patch failed. | | -1 :x: | javadoc | 0m 8s | hbase-external-blockcache in the patch failed. | | -1 :x: | javadoc | 0m 8s | hbase-it in the patch failed. | | -1 :x: | javadoc | 0m 10s | hbase-mapreduce in the patch failed. | | -1 :x: | javadoc | 0m 7s | hbase-protocol-shaded in the patch failed. | | -1 :x: | javadoc | 0m 9s
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045182#comment-17045182
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #285 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/285/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/285//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/285//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/285//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job URL: https://github.com/apache/hbase/pull/1195#issuecomment-591257167 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 6s | master passed | | +1 :green_heart: | compile | 4m 5s | master passed with JDK v2020-01-14 | | +1 :green_heart: | compile | 3m 21s | master passed with JDK v1.8.0_232 | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. | | +1 :green_heart: | javadoc | 2m 53s | master passed with JDK v1.8.0_232 | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 29s | the patch passed | | +1 :green_heart: | compile | 4m 16s | the patch passed with JDK v2020-01-14 | | +1 :green_heart: | javac | 4m 16s | the patch passed | | +1 :green_heart: | compile | 3m 43s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javac | 3m 43s | the patch passed | | +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 18m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 19s | root in the patch failed with JDK v2020-01-14. | | +1 :green_heart: | javadoc | 3m 13s | the patch passed with JDK v1.8.0_232 | ||| _ Other Tests _ | | +1 :green_heart: | unit | 155m 56s | root in the patch passed with JDK v1.8.0_232. | | +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. | | | | 237m 24s | | | Reason | Tests | |---:|:--| | JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1195 | | Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 5329e638c2d9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh | | git revision | master / ecbed33092 | | Default Java | 1.8.0_232 | | Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/branch-javadoc-root-jdk2020-01-14.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/patch-javadoc-root-jdk2020-01-14.txt | | JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/testReport/ | | Max. process+thread count | 6586 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
[GitHub] [hbase] Apache-HBase commented on issue #1209: HBASE-23146 Support CheckAndMutate with multiple conditions
Apache-HBase commented on issue #1209: HBASE-23146 Support CheckAndMutate with multiple conditions URL: https://github.com/apache/hbase/pull/1209#issuecomment-591255065 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 14s | branch-2 passed | | +1 :green_heart: | compile | 3m 35s | branch-2 passed | | +1 :green_heart: | checkstyle | 3m 18s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 26s | branch-2 passed | | +0 :ok: | spotbugs | 1m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 10m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 52s | the patch passed | | +1 :green_heart: | compile | 3m 39s | the patch passed | | +1 :green_heart: | cc | 3m 39s | the patch passed | | +1 :green_heart: | javac | 3m 39s | the patch passed | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-protocol | | -1 :x: | checkstyle | 0m 36s | hbase-client: The patch generated 2 new + 132 unchanged - 26 fixed = 134 total (was 158) | | +1 :green_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 0 new + 468 unchanged - 8 fixed = 468 total (was 476) | | +1 :green_heart: | checkstyle | 0m 35s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 20s | hbase-rest: The patch generated 0 new + 105 unchanged - 1 fixed = 105 total (was 106) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 31s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 25s | the patch passed | | +1 :green_heart: | javadoc | 2m 24s | the patch passed | | +1 :green_heart: | findbugs | 11m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 41s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 29s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 2m 1s | hbase-client in the patch passed. | | -1 :x: | unit | 61m 28s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 2m 23s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 6s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 2m 57s | The patch does not generate ASF License warnings. | | | | 159m 48s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.http.TestInfoServersACL | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1209/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1209 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux f94dc0f6bde7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1209/out/precommit/personality/provided.sh | | git revision | branch-2 / 2c3621690d | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1209/1/artifact/out/diff-checkstyle-hbase-client.txt | | unit |
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[
https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045138#comment-17045138
]
HBase QA commented on HBASE-23864:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
15s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-2.1 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
41s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 46s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.http.TestInfoServersACL |
| | hadoop.hbase.ipc.TestNettyIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1147/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23864 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994600/HBASE-23864.branch-2.1.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 00ff31876fd0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / b81a1c1cb5 |
| Default Java | 1.8.0_181 |
| unit |
[GitHub] [hbase] jojochuang opened a new pull request #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code)
jojochuang opened a new pull request #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code) URL: https://github.com/apache/hbase/pull/1210 This is a straightforward API migration, plus, merge HBASE-23758 (Propagate trace context to server correctly) into this patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] jojochuang commented on issue #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code)
jojochuang commented on issue #1210: HBASE-23757. [OpenTracing] Migrate from HTrace to OpenTracing (Java code) URL: https://github.com/apache/hbase/pull/1210#issuecomment-591236141 Like PR #1202 this one's not going to compile for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23758) [OpenTracing] Propagate trace context to server correctly
[ https://issues.apache.org/jira/browse/HBASE-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HBASE-23758. - Resolution: Duplicate Merging this into HBASE-23757. > [OpenTracing] Propagate trace context to server correctly > - > > Key: HBASE-23758 > URL: https://issues.apache.org/jira/browse/HBASE-23758 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > HBASE-18601 breaks trace propagation. Instead of fixing it for HTrace, we > should fix it for OpenTracing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23755) [OpenTracing] Declare HTrace is unusable in the user doc
[
https://issues.apache.org/jira/browse/HBASE-23755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045134#comment-17045134
]
Hudson commented on HBASE-23755:
Results for branch branch-2.1
[build #1816 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1816/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1816//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1816//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1816//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [OpenTracing] Declare HTrace is unusable in the user doc
>
>
> Key: HBASE-23755
> URL: https://issues.apache.org/jira/browse/HBASE-23755
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: Screen Shot 2020-02-21 at 5.18.12 PM.png
>
>
> The trace doesn't work at all in HBase 2.0 and above after HBASE-18601 (the
> trace doesn't get picked up at the server side). We should make a note in the
> user doc stating it is
> # unusable
> # deprecated in HBase 2.x because HTrace is in Attic.
> # removed from HBase 3.0 and above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] jojochuang commented on a change in pull request #1202: HBASE-23756. [OpenTracing] Add OpenTracing dependency and helper methods.
jojochuang commented on a change in pull request #1202: HBASE-23756.
[OpenTracing] Add OpenTracing dependency and helper methods.
URL: https://github.com/apache/hbase/pull/1202#discussion_r384263191
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
##
@@ -34,17 +51,24 @@
private static HTraceConfiguration conf;
private static Tracer tracer;
+ private static io.opentracing.Tracer otTracer;
+
+ private static final Logger LOG =
LoggerFactory.getLogger(TraceUtil.class.getName());
+
+
private TraceUtil() {
}
- public static void initTracer(Configuration c) {
-if (c != null) {
- conf = new HBaseHTraceConfiguration(c);
-}
+ public static void initTracer(Configuration c, String serviceName) {
+if (!GlobalTracer.isRegistered()) {
+ io.jaegertracing.Configuration conf =
io.jaegertracing.Configuration.fromEnv(serviceName);
+ io.opentracing.Tracer tracer = conf.getTracerBuilder().build();
-if (tracer == null && conf != null) {
- tracer = new Tracer.Builder("Tracer").conf(conf).build();
+ GlobalTracer.register(tracer);
Review comment:
I'd suggest you to browse the Jaeger's sampling doc:
https://www.jaegertracing.io/docs/1.14/sampling/#collector-sampling-configuration
You can use env variable to control the sampling strategy. Or
programmatically (at init). Finally, you can also define the sampling strategy
at the central trace collector.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern
Apache-HBase commented on issue #1016: HBASE-23656 [MERGETOOL] HBASE Support Merge region by pattern URL: https://github.com/apache/hbase/pull/1016#issuecomment-591225685 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1.4 Compile Tests _ | | +0 :ok: | mvndep | 1m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 22s | branch-1.4 passed | | +1 :green_heart: | compile | 1m 50s | branch-1.4 passed with JDK v1.8.0_242 | | +1 :green_heart: | compile | 1m 53s | branch-1.4 passed with JDK v1.7.0_252 | | +1 :green_heart: | checkstyle | 7m 57s | branch-1.4 passed | | +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 39s | branch-1.4 passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 3m 50s | branch-1.4 passed with JDK v1.7.0_252 | | +0 :ok: | spotbugs | 2m 42s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 14m 16s | branch-1.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 2m 13s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javac | 2m 13s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | javac | 3m 12s | the patch passed | | +1 :green_heart: | checkstyle | 9m 34s | the patch passed | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 2m 59s | Patch does not cause any errors with Hadoop 2.7.7. | | +1 :green_heart: | javadoc | 2m 43s | the patch passed with JDK v1.8.0_242 | | +1 :green_heart: | javadoc | 3m 47s | the patch passed with JDK v1.7.0_252 | | +1 :green_heart: | findbugs | 15m 8s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 166m 31s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 59s | The patch does not generate ASF License warnings. | | | | 273m 23s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.client.TestAdmin2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1016 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e6f933d0c6ca 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1016/out/precommit/personality/provided.sh | | git revision | branch-1.4 / 38bf65a | | Default Java | 1.7.0_252 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_242 /usr/lib/jvm/zulu-7-amd64:1.7.0_252 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/testReport/ | | Max. process+thread count | 4095 (vs. ulimit of 1) | | modules | C: hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1016/1/console | | versions | git=1.9.1 maven=3.0.5 shellcheck=0.7.0 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated.
[jira] [Commented] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job
[
https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045115#comment-17045115
]
Hudson commented on HBASE-23876:
Results for branch HBASE-23876/jdk11-nightly-master
[build #13 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/13//console].
> Add JDK11 compilation and unit test support to nightly job
> --
>
> Key: HBASE-23876
> URL: https://issues.apache.org/jira/browse/HBASE-23876
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> We already test against multiple JDK versions in a handful of places. Let's
> get JDK11 added to the mix. Applies to nightly job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23146) Support CheckAndMutate with multiple conditions
[ https://issues.apache.org/jira/browse/HBASE-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045108#comment-17045108 ] Toshihiro Suzuki commented on HBASE-23146: -- Created a PR for branch-2. Waiting for QA. > Support CheckAndMutate with multiple conditions > --- > > Key: HBASE-23146 > URL: https://issues.apache.org/jira/browse/HBASE-23146 > Project: HBase > Issue Type: New Feature >Reporter: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Currently, checkAndMutate supports only single condition. Supporting > checkAndMutate with multi conditions is useful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] brfrn169 opened a new pull request #1209: HBASE-23146 Support CheckAndMutate with multiple conditions
brfrn169 opened a new pull request #1209: HBASE-23146 Support CheckAndMutate with multiple conditions URL: https://github.com/apache/hbase/pull/1209 Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23864: --- Attachment: HBASE-23864.branch-2.1.003.patch > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10 > > Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG, > HBASE-23864.branch-2.1.002.patch, HBASE-23864.branch-2.1.003.patch, > HBASE-23864.branch-2.1.patch > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1179: HBASE-23861. Reconcile Hadoop version.
Apache-HBase commented on issue #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#issuecomment-591206638 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 48s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 44s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 49s | master passed | | +1 :green_heart: | compile | 3m 20s | master passed | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 30s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 35s | the patch passed | | +1 :green_heart: | compile | 3m 21s | the patch passed | | +1 :green_heart: | javac | 3m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 34s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 4s | root in the patch passed. | | +1 :green_heart: | asflicense | 0m 59s | The patch does not generate ASF License warnings. | | | | 213m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1179 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux a121848a666d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1179/out/precommit/personality/provided.sh | | git revision | master / 9f223c2236 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/3/testReport/ | | Max. process+thread count | 7493 (vs. ulimit of 1) | | modules | C: hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1179/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job URL: https://github.com/apache/hbase/pull/1195#issuecomment-591199907 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job URL: https://github.com/apache/hbase/pull/1195#issuecomment-591199260 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 1s | master passed | | +1 :green_heart: | compile | 4m 4s | master passed with JDK v2020-01-14 | | +1 :green_heart: | compile | 3m 25s | master passed with JDK v1.8.0_232 | | +1 :green_heart: | shadedjars | 5m 11s | branch has no errors when building our shaded downstream artifacts. | | -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. | | +1 :green_heart: | javadoc | 2m 49s | master passed with JDK v1.8.0_232 | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 42s | the patch passed | | +1 :green_heart: | compile | 4m 1s | the patch passed with JDK v2020-01-14 | | +1 :green_heart: | javac | 4m 1s | the patch passed | | +1 :green_heart: | compile | 3m 19s | the patch passed with JDK v1.8.0_232 | | +1 :green_heart: | javac | 3m 19s | the patch passed | | +1 :green_heart: | hadolint | 0m 1s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 5m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 18s | root in the patch failed with JDK v2020-01-14. | | +1 :green_heart: | javadoc | 2m 50s | the patch passed with JDK v1.8.0_232 | ||| _ Other Tests _ | | -1 :x: | unit | 116m 52s | root in the patch failed with JDK v1.8.0_232. | | +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | | | | 188m 31s | | | Reason | Tests | |---:|:--| | JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils | | JDK v1.8.0_232 Failed junit tests | hadoop.hbase.replication.regionserver.TestWALEntryStream | | | hadoop.hbase.io.hfile.bucket.TestBucketCache | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1195 | | Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 08b53edc990e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh | | git revision | master / 9f223c2236 | | Default Java | 1.8.0_232 | | Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/branch-javadoc-root-jdk2020-01-14.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/patch-javadoc-root-jdk2020-01-14.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/patch-unit-root-jdk1.8.0_232.txt | | JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/testReport/ | | Max. process+thread count | 5304 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.11.1
[GitHub] [hbase] Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase
Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase URL: https://github.com/apache/hbase/pull/1165#issuecomment-591197401 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 5s | https://github.com/apache/hbase/pull/1165 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/1165 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/6/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1208: HBASE-23845 Removed deprecated setMaxVersions from Scan
Apache-HBase commented on issue #1208: HBASE-23845 Removed deprecated setMaxVersions from Scan URL: https://github.com/apache/hbase/pull/1208#issuecomment-591196252 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 14 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 48s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 12s | master passed | | +1 :green_heart: | compile | 1m 51s | master passed | | +1 :green_heart: | checkstyle | 2m 1s | master passed | | +1 :green_heart: | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | master passed | | +0 :ok: | spotbugs | 4m 38s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 6m 40s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 21s | the patch passed | | +1 :green_heart: | compile | 1m 50s | the patch passed | | +1 :green_heart: | javac | 1m 50s | the patch passed | | +1 :green_heart: | checkstyle | 1m 57s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 27s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 16s | the patch passed | | +1 :green_heart: | findbugs | 6m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 53s | hbase-client in the patch passed. | | -1 :x: | unit | 59m 13s | hbase-server in the patch failed. | | -1 :x: | unit | 6m 28s | hbase-mapreduce in the patch failed. | | +1 :green_heart: | asflicense | 1m 23s | The patch does not generate ASF License warnings. | | | | 137m 48s | | | Reason | Tests | |---:|:--| | Unreaped Processes | hbase-server:8 | | Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshotV2NoCluster | | | hadoop.hbase.snapshot.TestExportSnapshotV1NoCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1208 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a62bf551d78c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1208/out/precommit/personality/provided.sh | | git revision | master / ecbed33092 | | Default Java | 1.8.0_181 | | Unreaped Processes Log | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/artifact/out/patch-unit-hbase-server-reaper.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/artifact/out/patch-unit-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/artifact/out/patch-unit-hbase-mapreduce.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/testReport/ | | Max. process+thread count | 8525 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server hbase-mapreduce U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1208/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards,
[jira] [Work started] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job
[ https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23876 started by Nick Dimiduk. > Add JDK11 compilation and unit test support to nightly job > -- > > Key: HBASE-23876 > URL: https://issues.apache.org/jira/browse/HBASE-23876 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > We already test against multiple JDK versions in a handful of places. Let's > get JDK11 added to the mix. Applies to nightly job. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job URL: https://github.com/apache/hbase/pull/1195#issuecomment-591173981 Okay, looks like it's running. At least I'm getting as far in Jenkins as I get trying to build and test locally. Check out: https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job
[
https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045047#comment-17045047
]
Hudson commented on HBASE-23876:
Results for branch HBASE-23876/jdk11-nightly-master
[build #12 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12//console].
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/12//console].
> Add JDK11 compilation and unit test support to nightly job
> --
>
> Key: HBASE-23876
> URL: https://issues.apache.org/jira/browse/HBASE-23876
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> We already test against multiple JDK versions in a handful of places. Let's
> get JDK11 added to the mix. Applies to nightly job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job
[
https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045032#comment-17045032
]
Hudson commented on HBASE-23876:
Results for branch HBASE-23876/jdk11-nightly-master
[build #11 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/11/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/11//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/11//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/11//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/11//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add JDK11 compilation and unit test support to nightly job
> --
>
> Key: HBASE-23876
> URL: https://issues.apache.org/jira/browse/HBASE-23876
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> We already test against multiple JDK versions in a handful of places. Let's
> get JDK11 added to the mix. Applies to nightly job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23894) [JDK11] build with -PerrorProne silently fails
[
https://issues.apache.org/jira/browse/HBASE-23894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045028#comment-17045028
]
Nick Dimiduk commented on HBASE-23894:
--
Attaching a log from a build failure found on HBASE-23876.
> [JDK11] build with -PerrorProne silently fails
> --
>
> Key: HBASE-23894
> URL: https://issues.apache.org/jira/browse/HBASE-23894
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: patch-compile-root.txt
>
>
> Build seems to simply stop at {{hbase-protocol-shaded}} without explanation.
> Dropping {{-PerrorProne}} allows the build to proceed, at least locally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23894) [JDK11] build with -PerrorProne silently fails
[
https://issues.apache.org/jira/browse/HBASE-23894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-23894:
-
Attachment: patch-compile-root.txt
> [JDK11] build with -PerrorProne silently fails
> --
>
> Key: HBASE-23894
> URL: https://issues.apache.org/jira/browse/HBASE-23894
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
> Attachments: patch-compile-root.txt
>
>
> Build seems to simply stop at {{hbase-protocol-shaded}} without explanation.
> Dropping {{-PerrorProne}} allows the build to proceed, at least locally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23894) [JDK11] build with -PerrorProne silently fails
Nick Dimiduk created HBASE-23894:
Summary: [JDK11] build with -PerrorProne silently fails
Key: HBASE-23894
URL: https://issues.apache.org/jira/browse/HBASE-23894
Project: HBase
Issue Type: Sub-task
Components: build
Reporter: Nick Dimiduk
Build seems to simply stop at {{hbase-protocol-shaded}} without explanation.
Dropping {{-PerrorProne}} allows the build to proceed, at least locally.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23146) Support CheckAndMutate with multiple conditions
[ https://issues.apache.org/jira/browse/HBASE-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045023#comment-17045023 ] Toshihiro Suzuki commented on HBASE-23146: -- Pushed to master. Working on branch-2. Will create another PR for branch-2. > Support CheckAndMutate with multiple conditions > --- > > Key: HBASE-23146 > URL: https://issues.apache.org/jira/browse/HBASE-23146 > Project: HBase > Issue Type: New Feature >Reporter: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Currently, checkAndMutate supports only single condition. Supporting > checkAndMutate with multi conditions is useful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23309) Add support in ChainWalEntryFilter to filter Entry if all cells get filtered through WalCellFilter
[
https://issues.apache.org/jira/browse/HBASE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-23309:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add support in ChainWalEntryFilter to filter Entry if all cells get filtered
> through WalCellFilter
> --
>
> Key: HBASE-23309
> URL: https://issues.apache.org/jira/browse/HBASE-23309
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.3.6, 2.3.3
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Attachments: HBASE-23309.branch-1.patch, HBASE-23309.branch-2.patch,
> HBASE-23309.patch
>
>
> ChainWalEntryFilter applies the filter on entry followed by filter on cells.
> If filter on cells remove all the cells from the entry, we should add an
> option in chainwalentryfilter to filter the entry as well.
> Here is the snippet for ChainWalEntryFilter filter. After filterCells we
> should check if there is any cells remaining in the entry.
> {code:java}
> @Override
> public Entry filter(Entry entry) {
> for (WALEntryFilter filter : filters) {
> if (entry == null) {
> return null;
> }
> entry = filter.filter(entry);
> }
> filterCells(entry);
> return entry;
> }{code}
> Customer replication endpoints may use this flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23845) Remove deprecated setMaxVersions() from Scan
[
https://issues.apache.org/jira/browse/HBASE-23845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23845:
--
Status: Patch Available (was: In Progress)
> Remove deprecated setMaxVersions() from Scan
>
>
> Key: HBASE-23845
> URL: https://issues.apache.org/jira/browse/HBASE-23845
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{setMaxVersions()}} in {{Scan}} was deprecated back in 2.0.0 and should be
> removed for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] HorizonNet opened a new pull request #1208: HBASE-23845 Removed deprecated setMaxVersions from Scan
HorizonNet opened a new pull request #1208: HBASE-23845 Removed deprecated setMaxVersions from Scan URL: https://github.com/apache/hbase/pull/1208 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job
[
https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044998#comment-17044998
]
Hudson commented on HBASE-23876:
Results for branch HBASE-23876/jdk11-nightly-master
[build #10 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/10/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/10//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/10//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/10//console].
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/10//console].
> Add JDK11 compilation and unit test support to nightly job
> --
>
> Key: HBASE-23876
> URL: https://issues.apache.org/jira/browse/HBASE-23876
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> We already test against multiple JDK versions in a handful of places. Let's
> get JDK11 added to the mix. Applies to nightly job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] brfrn169 merged pull request #1114: HBASE-23146 Support CheckAndMutate with multiple conditions
brfrn169 merged pull request #1114: HBASE-23146 Support CheckAndMutate with multiple conditions URL: https://github.com/apache/hbase/pull/1114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #1114: HBASE-23146 Support CheckAndMutate with multiple conditions
brfrn169 commented on a change in pull request #1114: HBASE-23146 Support
CheckAndMutate with multiple conditions
URL: https://github.com/apache/hbase/pull/1114#discussion_r384181018
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTable.java
##
@@ -289,6 +290,60 @@ default CheckAndMutateBuilder ifEquals(byte[] value) {
CompletableFuture thenMutate(RowMutations mutation);
}
+ /**
+ * Atomically checks if a row matches the specified filter. If it does, it
adds the
+ * Put/Delete/RowMutations.
+ *
+ * Use the returned {@link CheckAndMutateWithFilterBuilder} to construct
your request and then
+ * execute it. This is a fluent style API, the code is like:
+ *
+ *
+ *
+ * table.checkAndMutate(row, filter).thenPut(put)
+ * .thenAccept(succ -> {
+ * if (succ) {
+ * System.out.println("Check and put succeeded");
+ * } else {
+ * System.out.println("Check and put failed");
+ * }
+ * });
+ *
+ *
+ */
+ CheckAndMutateWithFilterBuilder checkAndMutate(byte[] row, Filter filter);
+
+ /**
+ * A helper class for sending checkAndMutate request with a filter.
+ */
+ interface CheckAndMutateWithFilterBuilder {
+
+/**
+ * @param timeRange time range to check.
+ */
+CheckAndMutateWithFilterBuilder timeRange(TimeRange timeRange);
Review comment:
Thank you very much for your review and approval!
> Maybe we could have a CheckAndMutateBuilderBase to palce these methods.
Yes, we can have CheckAndMutateBuilderBase to place the methods.
> Anyway, can be a follow issue as I do not think the change will break
compatibility.
Sure. Maybe we can do it at a next time when we add another type of
checkAndMutate.
Will commit this. Thank you very much again!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] jojochuang commented on a change in pull request #1179: HBASE-23861. Reconcile Hadoop version.
jojochuang commented on a change in pull request #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#discussion_r384179860 ## File path: hbase-server/pom.xml ## @@ -723,9 +723,6 @@ org.apache.hadoop hadoop-distcp - Review comment: Ah. good catch. Updated the PR to add this. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job URL: https://github.com/apache/hbase/pull/1195#issuecomment-591119825 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23893) Remove Hadoop 2.8 support
Wei-Chiu Chuang created HBASE-23893: --- Summary: Remove Hadoop 2.8 support Key: HBASE-23893 URL: https://issues.apache.org/jira/browse/HBASE-23893 Project: HBase Issue Type: Task Reporter: Wei-Chiu Chuang See HADOOP-16880 for the details. It looks like the time to drop Hadoop 2.8 support. Create this jira as the placeholder for this task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23888) PreCommit-HBASE-Build ignores the `ATTACHMENT_ID` provided by PreCommit-Admin
[
https://issues.apache.org/jira/browse/HBASE-23888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044968#comment-17044968
]
Nick Dimiduk commented on HBASE-23888:
--
{quote}IIRC, the suggest in the old time is, do not upload multiple patches at
the same time, upload one, wait until the pre commit job is started, and then
upload the next one.
{quote}
Really?? :( things have gotten worse since I was last around.
> PreCommit-HBASE-Build ignores the `ATTACHMENT_ID` provided by PreCommit-Admin
> -
>
> Key: HBASE-23888
> URL: https://issues.apache.org/jira/browse/HBASE-23888
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Priority: Major
>
> It appears that we've dropped the parameter {{ATTACHMENT_ID}} from our
> {{PreCommit-HBASE-Build}} job. (It may be the case that Yetus's
> {{test-patch}} doesn't support specifying what attachment id to test.) The
> result, I believe, is that when a Jira issue has patches attached for more
> than one branch, we only get precommit testing on the patch for master.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #1179: HBASE-23861. Reconcile Hadoop version.
ndimiduk commented on a change in pull request #1179: HBASE-23861. Reconcile Hadoop version. URL: https://github.com/apache/hbase/pull/1179#discussion_r384168804 ## File path: hbase-server/pom.xml ## @@ -723,9 +723,6 @@ org.apache.hadoop hadoop-distcp - Review comment: Looking at the `hadoop-2.0` profile in `hbase-server/pom.xml`, I see there's a version specified there for the `hadoop-distcp` dependency. You can apply the same change to this profile that you did to the `hadoop-3.0` profile, so that it will resolve the version from the parent pom's profile instead of the hard-coded version. https://github.com/ndimiduk/hbase/blob/00fc46756abb99de6f833997499505f89c9752e8/hbase-server/pom.xml#L612-L618 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] pustota2009 commented on a change in pull request #1200: HBASE 23887 BlockCache performance improve
pustota2009 commented on a change in pull request #1200: HBASE 23887 BlockCache
performance improve
URL: https://github.com/apache/hbase/pull/1200#discussion_r383691217
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
##
@@ -400,16 +413,24 @@ private Cacheable asReferencedHeapBlock(Cacheable buf) {
*/
@Override
public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
inMemory) {
+if (cacheDataBlockPercent != 100 && buf.getBlockType().isData()) {
+ // Don't cache this DATA block if we have limit on BlockCache,
+ // good for performance (HBASE-23887)
+ if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent) {
Review comment:
Yes, I think you got the general idea - when data much more then the cache
we have overhead. Lets provide some details. Why we do a lot of job:
1. Put some key-value into the map
2. Delete it quite soon, because there are quite big queue at entrance.
3. Clean the garbage
But anyway probability to hit into cache definite by size of cache. That is
why we can put into cache some random blocks and skip 1-3 steps for the rest
blocks. How it works:
Imagine we have little cache. Just for 1 block and trying to read 3 blocks
with offsets (into files):
124
198
223
Original way - we put the block 124, then put 198, evict 124, put 223, evict
198. A lot of work (5 actions).
With the feature - last few digits evenly distributed from 0 to 99. When we
divide by modulus we got:
124 -> 24
198 -> 98
223 -> 23
It helps to sort them. Some part, for example below 50 (if we set
cacheDataBlockPercent = 50) go into the cache. And skip others. It means we
will not try work with the block 198 and save CPU for other job. In the result
- we put block 124, then put 223, evict 124 (3 actions). Thats the idea.
If all is ok, later we can do some little improvments:
1. Don't even try get from the cache DATA blocks which can't be there (198
in the case above).
2. Populate the cache by data blocks anyway when the cache is empty.
And looks as good improvment provide the same logic for L2.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve
[ https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044194#comment-17044194 ] Danil Lipovoy edited comment on HBASE-23887 at 2/25/20 9:17 PM: [~reidchan] Yes, of course, you do great job as maintainers and you have to khow all details about it. I just now provided some details into comments PR, let copy part of it here and answer for your questions with that example. Imagine we have little cache. Just for 1 block and trying to read 3 blocks with offsets: 124 198 223 Original old way - we put the block 124, then put 198, evict 124, put 223, evict 198. A lot of work (5 actions). With the feature - last few digits evenly distributed from 0 to 99. When we divide by modulus we got: 124 -> 24 198 -> 98 223 -> 23 It helps to sort them. Some part, for example below 50 (if we set cacheDataBlockPercent = 50) go into the cache. And skip others. It means we will not try to handle the block 198 and save CPU for other job. In the result - we put block 124, then put 223, evict 124 (3 actions). About the random point. You are absolutely right, it is not trully random and this is good, because open the door for next improovments. We could don't even try to get data blocks witch can't be in the cache. It is 119 in our case above. If we use something like Random functions and skip really random blocks we close this opportunity (because can't know is it in the cache or not). I collect few first 10 offset from real requests: 1028307681 52897173 52615009 52988632 53586349 1534493828 651126 1595437225 457603970 1018982096 We can see that last 2 digits evenly distributed and not concentrate around 00 or somewhere else. Thats why we don't face the problems where some hot area will missed, because they are all different. About documentation - I would propose give some explanation like: use that parameter if you read much more then could fit into the cache and you can't use L2. Let me know please, what points not enough explained, I will try to provide more details. was (Author: pustota): [~reidchan] Yes, of course, you do great job as maintainers and you have to khow all details about it. I just now provided some details into comments PR, let copy part of it here and answer for your questions with that example. Imagine we have little cache. Just for 1 block and trying to read 3 blocks with offsets: 124 198 223 Original old way - we put the block 124, then put 198, evict 124, put 223, evict 198. A lot of work (5 actions). With the patch - last few digits evenly distributed from 0 to 99. When we divide by modulus we got: 124 -> 24 198 -> 98 223 -> 23 It helps to sort them. Some part, for example below 50 (if we set cacheDataBlockPercent = 50) go into the cache. And skip others. It means we will not try to handle the block 198 and save CPU for other job. In the result - we put block 124, then put 223, evict 124 (3 actions). About the random point. You are absolutely right, it is not trully random and this is good, because open the door for next improovments. We could don't even try to get data blocks witch can't be in the cache. It is 119 in our case above. If we use something like Random functions and skip really random blocks we close this opportunity (because can't know is it in the cache or not). I collect few first 10 offset from real requests: 1028307681 52897173 52615009 52988632 53586349 1534493828 651126 1595437225 457603970 1018982096 We can see that last 2 digits evenly distributed and not concentrate around 00 or somewhere else. Thats why we don't face the problems where some hot area will missed, because they are all different. About documentation - I would propose give some explanation like: use that parameter if you read much more then could fit into the cache and you can't use L2. Let me know please, what points not enough explained, I will try to provide more details. > BlockCache performance improve > -- > > Key: HBASE-23887 > URL: https://issues.apache.org/jira/browse/HBASE-23887 > Project: HBase > Issue Type: Improvement > Components: BlockCache, Performance >Reporter: Danil Lipovoy >Priority: Minor > Attachments: cmp.png > > > Hi! > I first time here, correct me please if something wrong. > I want propose how to improve performance when data in HFiles much more than > BlockChache (usual story in BigData). The idea - caching only part of DATA > blocks. It is good becouse LruBlockCache starts to work and save huge amount > of GC. See the picture in attachment with test below. Requests per second is > higher, GC is lower. > > The key point of the code: > Added the parameter: *hbase.lru.cache.data.block.percent* which by default = > 100 >
[jira] [Commented] (HBASE-23834) HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch
[ https://issues.apache.org/jira/browse/HBASE-23834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044885#comment-17044885 ] Wei-Chiu Chuang commented on HBASE-23834: - The consensus in the community (https://s.apache.org/HBaseShadeJetty) is to shade jetty in hbase-thirdparty. Will do that. > HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch > > > Key: HBASE-23834 > URL: https://issues.apache.org/jira/browse/HBASE-23834 > Project: HBase > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > HBase master branch is currently on Jetty 9.3, and latest Hadoop 3 > (unreleased branches trunk, branch-3.2 and branch-3.1) bumped Jetty to 9.4 to > address a vulnerability CVE-2017-9735. > (1) Jetty 9.3 and 9.4 are quite different (there are incompatible API > changes) and HBase won't start on the latest Hadoop 3. > (2) In any case, HBase should update its Jetty dependency to address the > vulnerability. > Fortunately for HBase, updating to Jetty 9.4 requires no code change other > than the maven version string. > More tests are needed to verify if HBase can run on older Hadoop versions if > its Jetty is updated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044740#comment-17044740
]
Danil Lipovoy edited comment on HBASE-23887 at 2/25/20 8:49 PM:
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
I agree, we use this perfect tool for tests. You are right about random access,
it is exactly our case in production environment (about 300 Tb HFiles) and
that's why we started to think how to optimize this case.
Now I loaded 1 bln records into 1 table 64 regions all in on 1 RS and read it
uniform by YCSB. Below the results.
1. Current version:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[READ], 99thPercentileLatency(us), 949
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 1560.15
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 61599
[CLEANUP], 95thPercentileLatency(us), 32
[CLEANUP], 99thPercentileLatency(us), 61599
2. With the feature:
[OVERALL], RunTime(ms), 34467
[OVERALL], Throughput(ops/sec), 29013.25905939014
[TOTAL_GCS_PS_Scavenge], Count, 9
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 104
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.3017378942176575
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 32
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.09284242899004845
[TOTAL_GCs], Count, 10
[TOTAL_GC_TIME], Time(ms), 136
[TOTAL_GC_TIME_%], Time(%), 0.3945803232077059
[READ], Operations, 100
[READ], AverageLatency(us), 661.87457
[READ], MinLatency(us), 145
[READ], MaxLatency(us), 261759
[READ], 95thPercentileLatency(us), 808
[READ], 99thPercentileLatency(us), 955
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2626.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 104511
[CLEANUP], 95thPercentileLatency(us), 26
[CLEANUP], 99thPercentileLatency(us), 104511
was (Author: pustota):
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, we use this perfect tool for tests. You are right about random access,
it is exactly our case in production environment (about 300 Tb HFiles) and
that's why we started to think how to optimize this case.
Now I loaded 1 bln records into 1 table 64 regions all in on 1 RS and read it
uniform by YCSB. Below the results.
1. Current version:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%),
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044740#comment-17044740
]
Danil Lipovoy edited comment on HBASE-23887 at 2/25/20 8:48 PM:
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, we use this perfect tool for tests. You are right about random access,
it is exactly our case in production environment (about 300 Tb HFiles) and
that's why we started to think how to optimize this case.
Now I loaded 1 bln records into 1 table 64 regions all in on 1 RS and read it
uniform by YCSB. Below the results.
1. Current version:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[READ], 99thPercentileLatency(us), 949
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 1560.15
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 61599
[CLEANUP], 95thPercentileLatency(us), 32
[CLEANUP], 99thPercentileLatency(us), 61599
2. With the feature:
[OVERALL], RunTime(ms), 34467
[OVERALL], Throughput(ops/sec), 29013.25905939014
[TOTAL_GCS_PS_Scavenge], Count, 9
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 104
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.3017378942176575
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 32
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.09284242899004845
[TOTAL_GCs], Count, 10
[TOTAL_GC_TIME], Time(ms), 136
[TOTAL_GC_TIME_%], Time(%), 0.3945803232077059
[READ], Operations, 100
[READ], AverageLatency(us), 661.87457
[READ], MinLatency(us), 145
[READ], MaxLatency(us), 261759
[READ], 95thPercentileLatency(us), 808
[READ], 99thPercentileLatency(us), 955
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2626.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 104511
[CLEANUP], 95thPercentileLatency(us), 26
[CLEANUP], 99thPercentileLatency(us), 104511
was (Author: pustota):
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. Current version:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME],
[GitHub] [hbase] Apache-HBase commented on issue #1203: HBASE-23878 : Backport HBASE-22040 to branch-2.1
Apache-HBase commented on issue #1203: HBASE-23878 : Backport HBASE-22040 to branch-2.1 URL: https://github.com/apache/hbase/pull/1203#issuecomment-591053398 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-2.1 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 3s | branch-2.1 passed | | +1 :green_heart: | compile | 1m 27s | branch-2.1 passed | | +1 :green_heart: | checkstyle | 2m 6s | branch-2.1 passed | | +1 :green_heart: | shadedjars | 4m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | branch-2.1 passed | | +0 :ok: | spotbugs | 2m 48s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 57s | branch-2.1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 42s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | -1 :x: | checkstyle | 0m 39s | hbase-client: The patch generated 7 new + 223 unchanged - 4 fixed = 230 total (was 227) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 20m 13s | Patch does not cause any errors with Hadoop 2.7.7 2.8.5 or 3.0.3 3.1.2. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | | +1 :green_heart: | findbugs | 4m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 32s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 142m 37s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 3s | The patch does not generate ASF License warnings. | | | | 209m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1203/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1203 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8a762c79b734 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1203/out/precommit/personality/provided.sh | | git revision | branch-2.1 / b81a1c1cb5 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1203/2/artifact/out/diff-checkstyle-hbase-client.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1203/2/testReport/ | | Max. process+thread count | 5090 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1203/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-22834) Remove deprecated methods from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel resolved HBASE-22834.
---
Fix Version/s: 3.0.0
Release Note: (was: The following methods were removed from
HBaseTestingUtility (due to HBASE-13893, HBASE-19410 & HBASE-19841):
createLocalHTU(): Use #HBaseTestingUtility() instead.
createLocalHTU(Configuration): Use #HBaseTestingUtility(Configuration) instead.
createTableDescriptor(String, int, int, int, KeepDeletedCells): Use
#createTableDescriptor(TableName, int, int, int, KeepDeletedCells) instead.
createTableDescriptor(String): Use #createTableDescriptor(TableName, int, int,
int, KeepDeletedCells) instead.
createLocalHRegion(byte[], byte[], byte[], String, Configuration, boolean,
Durability, WAL, byte[]...): Use #createLocalHRegion(TableName, byte[],
byte[], boolean, Durability, WAL, byte[]...) instead.)
Resolution: Fixed
All sub-tasked are resolved.
> Remove deprecated methods from HBaseTestingUtility
> --
>
> Key: HBASE-22834
> URL: https://issues.apache.org/jira/browse/HBASE-22834
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> {{HBaseTestingUtility}} has some deprecated methods, which should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23781) Remove deprecated createTableDescriptor(String) from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-23781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23781:
--
Fix Version/s: 3.0.0
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
> Remove deprecated createTableDescriptor(String) from HBaseTestingUtility
>
>
> Key: HBASE-23781
> URL: https://issues.apache.org/jira/browse/HBASE-23781
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> {{createTableDescriptor(String)}} in {{HBaseTestingUtility}} was deprecated
> back in 2.0.0 and should be removed for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] HorizonNet merged pull request #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility
HorizonNet merged pull request #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility URL: https://github.com/apache/hbase/pull/1146 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[
https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044784#comment-17044784
]
Sean Busbey commented on HBASE-23639:
-
{quote}
Sean Busbey by making the classes IA.LimitedPrivate, they can only be used by a
group of projects and can't be used if someone creates a new project.
{quote}
This is part of our overloaded use of IA.LimitedPrivate. We essentially define
audiences for our LimitedPrivate definitions in {{HBaseInterfaceAudience}}. One
of those is a downstream project (i.e. Phoenix), but several are about
downstream folks who are working in the grey area between things we expressly
encourage use of and things we are likely to want the prerogative to break at
will.
For example {{HIA.COPROC}} and {{HIA.REPLICATION}} are both for folks who want
to make use of some customization points that are very much still internals of
HBase. I think similarly having a {{HIA.CHAOS}} or {{HIA.INTEGRATION_TEST}}
would serve an analogous purpose for what you're trying to accomplish here.
{quote}
In HBaseClusterManager, CommandProvider is present which gives back commands to
execute the operations, which can be used by downstream , which will enable
anyone to use these functionalities without breaking current implementation by
ssh.
We have also created a Jira for making CommandProvider Public as well, as if
someone wants to create a CommandProvider for YARN and wanted
HBaseClusterManager to pick that up right now it can't be done.
{quote}
I'll try to set aside time to look at the specifics of CommandProvider to see
how it fits with the question I'm asking. Your comment on "without breaking the
current implementation by ssh" has things backwards. In general we should be
picking interface annotations that aim for the most restrictive answer to "what
can I change in this current implementation without breaking someone else".
From an interface perspective we need not worry about breaking the current
implementation via ssh because we control that code and the code that executes
it.
> Change InterfaceAudience for ClusterManager interface and HBaseClusterManager
> class public
> ---
>
> Key: HBASE-23639
> URL: https://issues.apache.org/jira/browse/HBASE-23639
> Project: HBase
> Issue Type: Improvement
> Components: API, integration tests, test
>Affects Versions: master
>Reporter: Mihir Monani
>Assignee: Lokesh Khurana
>Priority: Minor
> Attachments: HBASE-23639-master.patch
>
>
> In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager
> which has some part of implementation code for Chaos Action.
>
> If any user wants to create private implementation which resembles
> HBaseClusterManager, then it has to be merged with hbase-it package as
> ClusterManager Interface and it's extended implementations are private.
>
> We should make them public so anyone can have their own implementation and
> need to be merged with hbase-it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22978) Online slow response log
[
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044779#comment-17044779
]
Viraj Jasani commented on HBASE-22978:
--
Planning to commit this after ~2 days unless anyone has any objections.
> Online slow response log
>
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0, 2.3.0, 1.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.7.0
>
> Attachments: Screen Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot
> 2019-10-19 at 2.32.54 AM.png, Screen Shot 2019-10-19 at 2.34.11 AM.png,
> Screen Shot 2019-10-19 at 2.36.14 AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer -
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value:
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above
> example. We do this because the human readable representation is verbose, the
> rate of too slow warnings may be high, and the combination of these things
> can overwhelm the log capture system. The truncation is unfortunate because
> it eliminates much of the utility of the warnings. For example, the region
> name, the start and end keys, and the filter hierarchy are all important
> clues for debugging performance problems caused by moderate to low
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be
> too slow in addition to the responseTooSlow logging. The in-memory
> representation can be complete and compressed. A new admin API and shell
> command can provide access to the ring buffer for online performance
> debugging. A modest sizing of the ring buffer will prevent excessive memory
> utilization for a minor performance debugging feature by limiting the total
> number of retained records. There is some chance a high rate of requests will
> cause information on other interesting requests to be overwritten before it
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted
> in the request. We don't need to retain all key-values in the mutation, which
> may be too large to comfortably retain. We only need a unique set of row
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to
> apply fast compression to ring buffer entries (if codec support is
> available), something like snappy or zstandard, and decompress on the fly
> when servicing the retrieval API request. This will minimize the impact of
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same
> information provided by responseTooSlow warnings. Total size of response
> serialization, possibly also total cell or row counts, should be sufficient
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls
> behind and we are unable to persist an entry before it is overwritten, that
> is fine. Response too slow logging is best effort. If we can detect this make
> a note of it in the log file. Provide a tool for parsing, dumping, filtering,
> and pretty printing the slow logs written to HDFS. The tool and the shell can
> share and reuse some utility classes and methods for accomplishing that.
> —
> New shell commands:
> {{get_slow_responses [ ... , ] [ , \{
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring
> buffer maintained by the given list of servers; or all servers in the cluster
> if no list is provided. Optionally provide a map of parameters for filtering
> as additional argument. The TABLE filter, which expects a string containing a
> table name, will include only entries pertaining to that table. The REGION
> filter, which expects
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044740#comment-17044740
]
Danil Lipovoy edited comment on HBASE-23887 at 2/25/20 6:58 PM:
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. Current version:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[READ], 99thPercentileLatency(us), 949
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 1560.15
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 61599
[CLEANUP], 95thPercentileLatency(us), 32
[CLEANUP], 99thPercentileLatency(us), 61599
2. With the feature:
[OVERALL], RunTime(ms), 34467
[OVERALL], Throughput(ops/sec), 29013.25905939014
[TOTAL_GCS_PS_Scavenge], Count, 9
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 104
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.3017378942176575
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 32
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.09284242899004845
[TOTAL_GCs], Count, 10
[TOTAL_GC_TIME], Time(ms), 136
[TOTAL_GC_TIME_%], Time(%), 0.3945803232077059
[READ], Operations, 100
[READ], AverageLatency(us), 661.87457
[READ], MinLatency(us), 145
[READ], MaxLatency(us), 261759
[READ], 95thPercentileLatency(us), 808
[READ], 99thPercentileLatency(us), 955
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2626.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 104511
[CLEANUP], 95thPercentileLatency(us), 26
[CLEANUP], 99thPercentileLatency(us), 104511
was (Author: pustota):
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. The old way:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[GitHub] [hbase] Apache-HBase commented on issue #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte…
Apache-HBase commented on issue #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte… URL: https://github.com/apache/hbase/pull/1207#issuecomment-591012955 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 33s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | checkstyle | 0m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | master passed | | +0 :ok: | spotbugs | 0m 41s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 17s | the patch passed | | +1 :green_heart: | findbugs | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 47s | hbase-http in the patch passed. | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 49m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1207/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1207 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 745ed911f0a4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1207/out/precommit/personality/provided.sh | | git revision | master / 3e21dc73de | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1207/1/testReport/ | | Max. process+thread count | 617 (vs. ulimit of 1) | | modules | C: hbase-http U: hbase-http | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1207/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044740#comment-17044740
]
Danil Lipovoy edited comment on HBASE-23887 at 2/25/20 6:57 PM:
[~reidchan]
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. The old way:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[READ], 99thPercentileLatency(us), 949
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 1560.15
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 61599
[CLEANUP], 95thPercentileLatency(us), 32
[CLEANUP], 99thPercentileLatency(us), 61599
2. The new way:
[OVERALL], RunTime(ms), 34467
[OVERALL], Throughput(ops/sec), 29013.25905939014
[TOTAL_GCS_PS_Scavenge], Count, 9
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 104
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.3017378942176575
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 32
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.09284242899004845
[TOTAL_GCs], Count, 10
[TOTAL_GC_TIME], Time(ms), 136
[TOTAL_GC_TIME_%], Time(%), 0.3945803232077059
[READ], Operations, 100
[READ], AverageLatency(us), 661.87457
[READ], MinLatency(us), 145
[READ], MaxLatency(us), 261759
[READ], 95thPercentileLatency(us), 808
[READ], 99thPercentileLatency(us), 955
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2626.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 104511
[CLEANUP], 95thPercentileLatency(us), 26
[CLEANUP], 99thPercentileLatency(us), 104511
was (Author: pustota):
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. The old way:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[jira] [Commented] (HBASE-23887) BlockCache performance improve
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044740#comment-17044740
]
Danil Lipovoy commented on HBASE-23887:
---
>>Then the naming hbase.lru.cache.data.block.percent is not good, it is
>>confusing.
Ok, what would be better?
>>One case: if client happens to read 198 many times, then 198 will never get
>>cached because of the rule.
It is good point! We can easy provide special condition for that extreme cases
- just skip the new logic for IN_MEMORY tables:
{code:java}
if (cacheDataBlockPercent != 100 && buf.getBlockType().isData() && !inMemory)
{{code}
If someone read really often (in our case block evicts by few seconds because
heavy read) and need full cached data, he can use this option.
>>Another one extreme case: all offset last 2 digits just between [00, 85]
>>(assuming 85 set), then BC will cache them all anyway...
Please clarify, do you mean that blocks could be not evenly distributed?
If it is the point, I could collect statistics for millions requests and we
will know it for sure.
>>YCSB is a good benchmark tool,
Agree, I use it for tests. Now I loaded 1 bln records into 1 table 64 regions
all in on 1 RS and read it by YCSB. Below the results.
1. The old way:
[OVERALL], RunTime(ms), 45470
[OVERALL], Throughput(ops/sec), 21992.522542335606
[TOTAL_GCS_PS_Scavenge], Count, 8
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 740
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 1.627446668132835
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 31
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.06817681988124037
[TOTAL_GCs], Count, 9
[TOTAL_GC_TIME], Time(ms), 771
[TOTAL_GC_TIME_%], Time(%), 1.6956234880140753
[READ], Operations, 100
[READ], AverageLatency(us), 882.532253
[READ], MinLatency(us), 137
[READ], MaxLatency(us), 599039
[READ], 95thPercentileLatency(us), 765
[READ], 99thPercentileLatency(us), 949
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 1560.15
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 61599
[CLEANUP], 95thPercentileLatency(us), 32
[CLEANUP], 99thPercentileLatency(us), 61599
2. The new way:
[OVERALL], RunTime(ms), 34467
[OVERALL], Throughput(ops/sec), 29013.25905939014
[TOTAL_GCS_PS_Scavenge], Count, 9
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 104
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.3017378942176575
[TOTAL_GCS_PS_MarkSweep], Count, 1
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 32
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.09284242899004845
[TOTAL_GCs], Count, 10
[TOTAL_GC_TIME], Time(ms), 136
[TOTAL_GC_TIME_%], Time(%), 0.3945803232077059
[READ], Operations, 100
[READ], AverageLatency(us), 661.87457
[READ], MinLatency(us), 145
[READ], MaxLatency(us), 261759
[READ], 95thPercentileLatency(us), 808
[READ], 99thPercentileLatency(us), 955
[READ], Return=OK, 100
[CLEANUP], Operations, 40
[CLEANUP], AverageLatency(us), 2626.25
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 104511
[CLEANUP], 95thPercentileLatency(us), 26
[CLEANUP], 99thPercentileLatency(us), 104511
> BlockCache performance improve
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Priority: Minor
> Attachments: cmp.png
>
>
> Hi!
> I first time here, correct me please if something wrong.
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC. See the picture in attachment with test below. Requests per second is
> higher, GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 0-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
>
> Descriptions of the test:
> 4 nodes E5-2698 v4 @ 2.20GHz, 700 Gb Mem.
> 4 RegionServers
> 4 tables by 64 regions by 1.88 Gb data in each = 600 Gb total (only FAST_DIFF)
> Total BlockCache Size = 48 Gb (8 % of data in HFiles)
> Random read in 20 threads
>
> I am going to make Pull Request, hope it is right way to make some
> contribution in this cool product.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ramkrish86 commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
ramkrish86 commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin… URL: https://github.com/apache/hbase/pull/1187#issuecomment-590990341 @Reidd - I think what you mean is that if blocks are cached on compaction is there any impact the cache usage ? Ya it is. Since this is aggressive caching probably we need bigger caches avialable for this feature to work. Say for eg if cacheOnWrite is also enabled then blocks are cached during flushes and also during compactions. So at any point of time there could be duplicate blocks in the cache thus bloating the cache. LRU will remove the old blocks anyway but still the usage is going to be more. but in clould usages (as you can see in parent JIRa) a compaction will clear all the blocks from the cache. So any scans that happens after a compaction will have high latencies. So in such cases with bigger caches this new config will help to have the blocks in cache and thus having more predictable scan latencies. Hence it was later decided (if you see the subjiras in the parent JIRA) that lets cache the index blocks also if this property and cacheOnWrite is set to true. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on issue #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte…
wchevreuil commented on issue #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte… URL: https://github.com/apache/hbase/pull/1207#issuecomment-590989997 Idea here is that we need to figure out a valid path from resources within the classpath, but that's only possible if we refer a file that's not contained with a jar, so thought of checking for possible existence of a file named "hbase-test-util-dependent" file that extending tests could then define in its classpath. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[
https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044708#comment-17044708
]
HBase QA commented on HBASE-23639:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
2s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 79 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
25s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} hbase-it: The patch generated 3 new + 129 unchanged -
1 fixed = 132 total (was 130) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 47s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
17s{color} | {color:red} hbase-it generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
46s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1146/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23639 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994569/HBASE-23639-master.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck
xml compile spotbugs findbugs hbaseanti checkstyle |
| uname | Linux b24080e31cba 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3e21dc73de |
|
[GitHub] [hbase] wchevreuil opened a new pull request #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte…
wchevreuil opened a new pull request #1207: HBASE-23892 KeyStoreTestUtil.getClasspathDir NPE when running UT exte… URL: https://github.com/apache/hbase/pull/1207 …nds SecureTestCluster and hbase-server is referred as a jar file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23889) Switch back to ZKConnectionRegistry by default at least in test
[
https://issues.apache.org/jira/browse/HBASE-23889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044704#comment-17044704
]
Bharath Vissapragada commented on HBASE-23889:
--
I think there are two separate issues here.
bq. after mering back the feature branch, the flaky list for master branch
became really big.
I think the flakiness started after HBASE-23779 and related patches. That
increased the fork count and that exposed a lot of flakes. I'm happy to dig
into failures that you think are caused by the feature branch.
bq. And also, remove the invalidateConnection method in HBTU, just reset the
property in Configuration to let the client load the new configuration
automatically.
I do see your point, but I think there are some finer issues here.
1. This pattern of picking random ports after every role restart is very
specific to the unit-tests to avoid port conflicts. This is not reflective of a
real world usage pattern. Because every restart picks a totally different
, the older master: becomes useless. In a normal world where
restarts happen (and the master: remains the same), the connections are
resilient.
2. Getting rid of "invalidateConnection()" is an enormous task (like you
already noted). Our HBTU usage is pretty erratic and all the tests directly
operate on the underlying LocalHBaseCluster. If you see my comment in the
code.. This pretty much means that one needs to rewrite most of the tests
written over the past 10 (?) years.
{noformat}
* TODO: There should be a more coherent way of doing this. Unfortunately the
way tests are
* written, not all start() stop() calls go through this class. Most tests
directly operate on
* the underlying mini/local hbase cluster. That makes it difficult for
this wrapper class to
* maintain the connection state automatically. Cleaning this is a much
bigger refactor.
{noformat}
3. The whole premise of the feature is that the new bootstrap set of nodes for
clients is the HMaster group. Earlier ZK had this role. This means that client
expects the masters to be generally available to perform basic operations.
This resulted in outdated test scenarios (like
test-connection-when-cluster-is-not-up etc). This is the reason, we have some
test utilities like {{AlwaysStandByHMaster}} to help transition the tests into
the newer world.
That said, I still agree with you that having dynamic reconfiguration on the
client side will nice-to-have feature, but IMHO that alone shouldn't be the
reason to switch the default registry back to ZK based. WDYT?
> Switch back to ZKConnectionRegistry by default at least in test
> ---
>
> Key: HBASE-23889
> URL: https://issues.apache.org/jira/browse/HBASE-23889
> Project: HBase
> Issue Type: Bug
> Components: Client, rpc, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> For now, MasterRegistry can not deal with master restart, as it can not load
> the new master address automatically.
> I see there is a invalidateConnection method in HBaseTestingUtilities but it
> needs a very big refactoring to make all the UTs work like this.
> So here I suggest we switch back to ZKConnectionRegistry by default, and open
> a new feature branch to finish the TODOs and the refactoring on UTs.
> As now it is already a big problem for me as I want to merge a feature branch
> back to master but the state of UTs are a mess.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23892) KeyStoreTestUtil.getClasspathDir NPE when running UT extends SecureTestCluster and hbase-server is referred as a jar file.
[ https://issues.apache.org/jira/browse/HBASE-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-23892: - Affects Version/s: 3.0.0 > KeyStoreTestUtil.getClasspathDir NPE when running UT extends > SecureTestCluster and hbase-server is referred as a jar file. > -- > > Key: HBASE-23892 > URL: https://issues.apache.org/jira/browse/HBASE-23892 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > > Was working on a third party project that relies on hbase-server as a > dependency and defines a UT class that extends *SecureTestCluster*. In this > project, hbase-server jar is added on the test classpath, and it relies on > *KeyStoreTestUtil* to decide where to place related ssl files. Current > *KeyStoreTestUtil* code assumes related class files would be under an > existing local FS path, but when those are loaded from a jar, related class > URI path returns null and causes an NPE that errors out the test execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23892) KeyStoreTestUtil.getClasspathDir NPE when running UT extends SecureTestCluster and hbase-server is referred as a jar file.
Wellington Chevreuil created HBASE-23892: Summary: KeyStoreTestUtil.getClasspathDir NPE when running UT extends SecureTestCluster and hbase-server is referred as a jar file. Key: HBASE-23892 URL: https://issues.apache.org/jira/browse/HBASE-23892 Project: HBase Issue Type: Bug Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil Was working on a third party project that relies on hbase-server as a dependency and defines a UT class that extends *SecureTestCluster*. In this project, hbase-server jar is added on the test classpath, and it relies on *KeyStoreTestUtil* to decide where to place related ssl files. Current *KeyStoreTestUtil* code assumes related class files would be under an existing local FS path, but when those are loaded from a jar, related class URI path returns null and causes an NPE that errors out the test execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Khurana updated HBASE-23639: --- Attachment: (was: HBASE-23639-master.patch) > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Affects Versions: master >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Khurana updated HBASE-23639: --- Attachment: HBASE-23639-master.patch Affects Version/s: master Status: Patch Available (was: In Progress) > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Affects Versions: master >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044653#comment-17044653 ] Lokesh Khurana commented on HBASE-23639: [~busbey] by making the classes IA.LimitedPrivate, they can only be used by a group of projects and can't be used if someone creates a new project. In HBaseClusterManager, CommandProvider is present which gives back commands to execute the operations, which can be used by downstream , which will enable anyone to use these functionalities without breaking current implementation by ssh. We have also created a Jira for making CommandProvider Public as well, as if someone wants to create a CommandProvider for YARN and wanted HBaseClusterManager to pick that up right now it can't be done. > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22827) Expose multi-region merge in shell and Admin API
[ https://issues.apache.org/jira/browse/HBASE-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044638#comment-17044638 ] Sakthi commented on HBASE-22827: Merged in 2.2 > Expose multi-region merge in shell and Admin API > > > Key: HBASE-22827 > URL: https://issues.apache.org/jira/browse/HBASE-22827 > Project: HBase > Issue Type: Improvement > Components: Admin, shell >Reporter: Michael Stack >Assignee: Sakthi >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4 > > Attachments: hbase-22827.branch-2.001.patch > > > HBASE-22777 adds being able to merge more than two regions at once. It is > only available internally currently for use by hbck2 doing fixup of overlaps > in hbase:meta. This issue is about exposing it via the Admin Interface and in > turn, via the shell. Probably best if old two region merge method is > deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22827) Expose multi-region merge in shell and Admin API
[ https://issues.apache.org/jira/browse/HBASE-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-22827: --- Fix Version/s: 2.2.4 > Expose multi-region merge in shell and Admin API > > > Key: HBASE-22827 > URL: https://issues.apache.org/jira/browse/HBASE-22827 > Project: HBase > Issue Type: Improvement > Components: Admin, shell >Reporter: Michael Stack >Assignee: Sakthi >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4 > > Attachments: hbase-22827.branch-2.001.patch > > > HBASE-22777 adds being able to merge more than two regions at once. It is > only available internally currently for use by hbck2 doing fixup of overlaps > in hbase:meta. This issue is about exposing it via the Admin Interface and in > turn, via the shell. Probably best if old two region merge method is > deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] jatsakthi merged pull request #1204: HBASE-22827 Expose multi-region merge in shell and Admin API
jatsakthi merged pull request #1204: HBASE-22827 Expose multi-region merge in shell and Admin API URL: https://github.com/apache/hbase/pull/1204 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1165: HBASE-22514 Move rsgroup feature into core of HBase
saintstack commented on a change in pull request #1165: HBASE-22514 Move
rsgroup feature into core of HBase
URL: https://github.com/apache/hbase/pull/1165#discussion_r383988388
##
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
##
@@ -2262,4 +2265,104 @@ boolean snapshotCleanupSwitch(final boolean on, final
boolean synchronous)
*/
boolean isSnapshotCleanupEnabled() throws IOException;
+ /**
+ * Creates a new RegionServer group with the given name
+ * @param groupName the name of the group
+ * @throws IOException if a remote or network exception occurs
+ */
+ void addRSGroup(String groupName) throws IOException;
+
+ /**
+ * Get group info for the given group name
+ * @param groupName the group name
+ * @return group info
+ * @throws IOException if a remote or network exception occurs
+ */
+ RSGroupInfo getRSGroup(String groupName) throws IOException;
+
+ /**
+ * Get group info for the given hostPort
+ * @param hostPort HostPort to get RSGroupInfo for
+ * @throws IOException if a remote or network exception occurs
+ */
+ RSGroupInfo getRSGroup(Address hostPort) throws IOException;
+
+ /**
+ * Get group info for the given table
+ * @param tableName table name to get RSGroupInfo for
+ * @throws IOException if a remote or network exception occurs
+ */
+ RSGroupInfo getRSGroup(TableName tableName) throws IOException;
+
+ /**
+ * Lists current set of RegionServer groups
+ * @throws IOException if a remote or network exception occurs
+ */
+ List listRSGroups() throws IOException;
+
+ /**
+ * Get all tables in this RegionServer group.
+ * @param groupName the group name
+ * @throws IOException if a remote or network exception occurs
+ * @see #getConfiguredNamespacesAndTablesInRSGroup(String)
+ */
+ List listTablesInRSGroup(String groupName) throws IOException;
+
+ /**
+ * Get the namespaces and tables which have this RegionServer group in
descriptor.
+ *
+ * The difference between this method and {@link
#listTablesInRSGroup(String)} is that, this
+ * method will not include the table which is actually in this RegionServr
group but without the
+ * RegionServer group configuration in its {@link TableDescriptor}. For
example, we have a group
+ * 'A', and we make namespace 'nsA' in this group, then all the tables under
this namespace will
+ * in the group 'A', but this method will not return these tables but only
the namespace 'nsA',
+ * while the {@link #listTablesInRSGroup(String)} will return all these
tables.
+ * @param groupName the group name
+ * @throws IOException if a remote or network exception occurs
+ * @see #listTablesInRSGroup(String)
+ */
+ Pair, List>
getConfiguredNamespacesAndTablesInRSGroup(String groupName)
Review comment:
The 'Configure' seems redundant to me given the suffix on method name is
'InRSGroup'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1165: HBASE-22514 Move rsgroup feature into core of HBase
saintstack commented on a change in pull request #1165: HBASE-22514 Move rsgroup feature into core of HBase URL: https://github.com/apache/hbase/pull/1165#discussion_r383986883 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java ## @@ -2262,4 +2265,104 @@ boolean snapshotCleanupSwitch(final boolean on, final boolean synchronous) */ boolean isSnapshotCleanupEnabled() throws IOException; + /** + * Creates a new RegionServer group with the given name + * @param groupName the name of the group + * @throws IOException if a remote or network exception occurs + */ + void addRSGroup(String groupName) throws IOException; Review comment: Ok. So, your argument is that folks using current data structures and methods will have an easier time if the names of methods, etc., have same basic pattern in the new context. Ok. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383932417 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the +rsgroup for a table is stored in TableDescriptor, instead of in RSGroupInfo, +so the getTables method of RSGroupInfo has been deprecated. And if you use the +Admin methods to get the RSGroupInfo, its getTables method will always return +empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, +we will fill the tables field of the RSGroupInfo before returning, to make it +compatible with old hbase client/shell. + +When upgrading, the migration between the RSGroupInfo and TableDescriptor will Review comment: excellent. please add a sentence here that expressly calls out that enforcement will keep working during the migration and running new admin commands is fine. probably I will get folks asking me how to tell when the migration is complete, so if there's an easy answer to that we should include it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383931523 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the Review comment: please include that information in this paragraph. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383930944 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to Review comment: okay good. Presuming there's no harm in still having the balancer customized I think this is fine. If having the balancer customized still is a problem, then we should include a sentence that says "you should remove the custom balancer config" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase
Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase URL: https://github.com/apache/hbase/pull/1165#issuecomment-590907554 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 47 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 49s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 19s | master passed | | +1 :green_heart: | compile | 3m 3s | master passed | | +1 :green_heart: | checkstyle | 2m 36s | master passed | | +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 19s | master passed | | +0 :ok: | spotbugs | 1m 22s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 15s | branch/hbase-annotations no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 32s | branch/hbase-assembly no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 2m 56s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 3m 2s | the patch passed | | +1 :green_heart: | cc | 3m 2s | the patch passed | | +1 :green_heart: | javac | 3m 2s | the patch passed | | +1 :green_heart: | checkstyle | 2m 32s | root: The patch generated 0 new + 733 unchanged - 22 fixed = 733 total (was 755) | | +1 :green_heart: | rubocop | 0m 9s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 45s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 10m 13s | the patch passed | | +1 :green_heart: | javadoc | 6m 5s | the patch passed | | +0 :ok: | findbugs | 0m 13s | hbase-annotations has no data from findbugs | | +0 :ok: | findbugs | 0m 31s | hbase-assembly has no data from findbugs | ||| _ Other Tests _ | | -1 :x: | unit | 69m 2s | root in the patch failed. | | +1 :green_heart: | asflicense | 5m 37s | The patch does not generate ASF License warnings. | | | | 209m 44s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.rsgroup.TestRSGroupsBasics | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1165 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml cc hbaseprotoc prototool rubocop | | uname | Linux 310d05e52bd6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1165/out/precommit/personality/provided.sh | | git revision | master / 3e21dc73de | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/5/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/5/testReport/ | | Max. process+thread count | 8897 (vs. ulimit of 1) | | modules | C: hbase-annotations hbase-protocol-shaded hbase-common hbase-protocol hbase-client hbase-server hbase-thrift hbase-shell hbase-it hbase-assembly . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/5/console | | versions |
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383929445 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3347,20 +3346,9 @@ Here is example using a few of the rsgroup commands. To add a group, do as foll .RegionServer Groups must be Enabled [NOTE] -If you have not enabled the rsgroup Coprocessor Endpoint in the master and -you run the any of the rsgroup shell commands, you will see an error message -like the below: - -[source,java] - -ERROR: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered master coprocessor service found for name RSGroupAdminService -at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:604) -at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) -at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:1140) -at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257) - +If you have not enabled the rsgroup in the master and you call the any of +rsgroup admin methods, or run the any of the rsgroup shell commands, you may +see a `DoNotRetryIOException` which tells that 'RSGroup is disabled' Review comment: that message would be great! instructions on how to turn it on would be even better, but I'd be happy with a clear statement that the feature is off as the cause. I see now why I misunderstood. I took "see a X which tells that Y" to mean "you will see X and you should interpret it to mean Y" rather than the intended "you will see X and it will contain details that Y". suggested edit for this sentence: > If you have not enabled the rsgroup feature and you call any of the rsgroup admin methods or shell commands the call will fail with a `DoNotRetryIOException` with a detail message that says the rsgroup feature is disabled. wdyt? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.
Apache-HBase commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos. URL: https://github.com/apache/hbase/pull/1201#issuecomment-590903837 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 79 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 58s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 22s | master passed | | +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | master passed | | +0 :ok: | spotbugs | 5m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 19s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | +1 :green_heart: | javac | 0m 32s | the patch passed | | -1 :x: | checkstyle | 0m 22s | hbase-it: The patch generated 3 new + 129 unchanged - 1 fixed = 132 total (was 130) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 37s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | -1 :x: | javadoc | 0m 19s | hbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | findbugs | 0m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 53s | hbase-it in the patch passed. | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 49m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1201 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile spotbugs findbugs hbaseanti checkstyle | | uname | Linux f103fbfa9b6f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1201/out/precommit/personality/provided.sh | | git revision | master / 3e21dc73de | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/2/artifact/out/diff-checkstyle-hbase-it.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/2/artifact/out/diff-javadoc-javadoc-hbase-it.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/2/testReport/ | | Max. process+thread count | 404 (vs. ulimit of 1) | | modules | C: hbase-it U: hbase-it | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383920424 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3347,20 +3346,9 @@ Here is example using a few of the rsgroup commands. To add a group, do as foll .RegionServer Groups must be Enabled [NOTE] -If you have not enabled the rsgroup Coprocessor Endpoint in the master and -you run the any of the rsgroup shell commands, you will see an error message -like the below: - -[source,java] - -ERROR: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered master coprocessor service found for name RSGroupAdminService -at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:604) -at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) -at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:1140) -at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257) - +If you have not enabled the rsgroup in the master and you call the any of +rsgroup admin methods, or run the any of the rsgroup shell commands, you may +see a `DoNotRetryIOException` which tells that 'RSGroup is disabled' Review comment: I think the message "RSGroup is disabled" is clear enough? You mean add a "set 'hbase.balancer.rsgroup.enabled' to true if you want to use RSGroup"? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383919516 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to Review comment: No they don't. Actually I think there are two requirements. One is for communicating with pre 3.0.0 hbase client/shell, then you have to set the coprocessor. Another is that you upgrade from pre 3.0.0 to 3.0.0, then we will set the flag to true automatically for you if you have already enabled rs group by the old way. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383917827 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the +rsgroup for a table is stored in TableDescriptor, instead of in RSGroupInfo, +so the getTables method of RSGroupInfo has been deprecated. And if you use the +Admin methods to get the RSGroupInfo, its getTables method will always return +empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, +we will fill the tables field of the RSGroupInfo before returning, to make it +compatible with old hbase client/shell. + +When upgrading, the migration between the RSGroupInfo and TableDescriptor will Review comment: Just update the table descriptor, no other effect, you can see the comment in RSGroupInfoManagerImpl.migrate method for more details why this could work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache9 commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383916533 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the Review comment: The old implementation is a bit broken, as we can set rsgroup for a namespace, and you can never get this information from the RSGroupInfo. In the new implementation there are two Admin methods for getting the information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383906368 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to Review comment: if they're relying on this side effect of configuring the coprocessor, do they also have to be sure to set the custom balancer (as in the old implementation) or does the flag flipping to true take care of that? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383904939 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3347,20 +3346,9 @@ Here is example using a few of the rsgroup commands. To add a group, do as foll .RegionServer Groups must be Enabled [NOTE] -If you have not enabled the rsgroup Coprocessor Endpoint in the master and -you run the any of the rsgroup shell commands, you will see an error message -like the below: - -[source,java] - -ERROR: org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered master coprocessor service found for name RSGroupAdminService -at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:604) -at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) -at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:1140) -at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277) -at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257) - +If you have not enabled the rsgroup in the master and you call the any of +rsgroup admin methods, or run the any of the rsgroup shell commands, you may +see a `DoNotRetryIOException` which tells that 'RSGroup is disabled' Review comment: Does the `DoNotRetryIOException` make clear that the feature is off? or does it talk about some other thing that we can give an example of here to distinguish from some coincidental failure? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383907721 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the +rsgroup for a table is stored in TableDescriptor, instead of in RSGroupInfo, +so the getTables method of RSGroupInfo has been deprecated. And if you use the +Admin methods to get the RSGroupInfo, its getTables method will always return +empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, +we will fill the tables field of the RSGroupInfo before returning, to make it +compatible with old hbase client/shell. + +When upgrading, the migration between the RSGroupInfo and TableDescriptor will Review comment: what happens wrt group enforcement during the migration? Will regions for grouped tables/namespaces go outside their assigned servers? Will rsgroup administrative commands I run wait to handle moving things in or out of a rsgroup until the migration is done? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383905794 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation Review comment: the section on Upgrade Paths needs a call out that points down here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
busbey commented on a change in pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#discussion_r383908543 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -3479,6 +3478,27 @@ To enable ACL, add the following to your hbase-site.xml and restart your Master: +=== Migrating From Old Implementation +The coprocessor `org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint` is +deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell +to communicate with the new hbase cluster, you still need to add this +coprocessor to master. And if this coprocessor is specified, the +`hbase.balancer.rsgroup.enabled` flag will be set to true automatically to +enable rs group feature. + +The main difference comparing to the old implementation is that, now the Review comment: is there an implementation reason we don't fill in the tables for RSGroupInfo in this case? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache-HBase commented on issue #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206#issuecomment-590885908 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-22514 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 36s | HBASE-22514 passed | | +0 :ok: | refguide | 4m 51s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 4m 45s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 22m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1206/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1206 | | Optional Tests | dupname asflicense refguide | | uname | Linux 61a3f8d1b8e6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1206/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 77b24fd007 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1206/1/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1206/1/artifact/out/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1206/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[
https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044465#comment-17044465
]
Sean Busbey commented on HBASE-23639:
-
I have concerns making the chaos and cluster management stuff IA.Public. These
have historically been for our project's internal testing. Making them
IA.Public means now we have to consider the impact to them e.g. during upgrades.
[~mihir6692]and [~lokiore] would your concerns be addressed by making the
needed classes IA.LimitedPrivate? that would mean e.g. we wouldn't break them
on maintenance releases, but we might still break them on minor releases.
For {{HBaseClusterManager}} can y'all get more specific on what things are
needed for a downstream implementor of {{ClusterManager}}? I'd rather we
separate out those things instead of making the implementation by ssh something
we tell downstream folks we'll avoid breaking.
> Change InterfaceAudience for ClusterManager interface and HBaseClusterManager
> class public
> ---
>
> Key: HBASE-23639
> URL: https://issues.apache.org/jira/browse/HBASE-23639
> Project: HBase
> Issue Type: Improvement
> Components: API, integration tests, test
>Reporter: Mihir Monani
>Assignee: Lokesh Khurana
>Priority: Minor
> Attachments: HBASE-23639-master.patch
>
>
> In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager
> which has some part of implementation code for Chaos Action.
>
> If any user wants to create private implementation which resembles
> HBaseClusterManager, then it has to be merged with hbase-it package as
> ClusterManager Interface and it's extended implementations are private.
>
> We should make them public so anyone can have their own implementation and
> need to be merged with hbase-it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23891) Add an option to Actions to filter out meta RS
Tamas Adami created HBASE-23891: --- Summary: Add an option to Actions to filter out meta RS Key: HBASE-23891 URL: https://issues.apache.org/jira/browse/HBASE-23891 Project: HBase Issue Type: Sub-task Components: integration tests Affects Versions: 3.0.0 Reporter: Tamas Adami Fix For: 3.0.0, 2.3.0, 2.2.3 Add an option to Actions to be able to filter meta server out. Some ITs rely on meta RS and have timeout errors if this RS is killed. (e.g. IntegrationTestTimeBoundedRequestsWithRegionReplicas) For the time being there is no option for removing meta server from server list to kill or configuring these actions properly. The following chaos monkey actions are affected: GracefulRollingRestartRsAction, RollingBatchSuspendResumeRsAction -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23639 started by Lokesh Khurana. -- > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Khurana updated HBASE-23639: --- Attachment: (was: HBASE-23639-master.patch) > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23639) Change InterfaceAudience for ClusterManager interface and HBaseClusterManager class public
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Khurana updated HBASE-23639: --- Attachment: HBASE-23639-master.patch > Change InterfaceAudience for ClusterManager interface and HBaseClusterManager > class public > --- > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 opened a new pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide
Apache9 opened a new pull request #1206: HBASE-23890 Update the rsgroup section in our ref guide URL: https://github.com/apache/hbase/pull/1206 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1200: HBASE 23887 BlockCache performance improve
Apache-HBase commented on issue #1200: HBASE 23887 BlockCache performance improve URL: https://github.com/apache/hbase/pull/1200#issuecomment-590854368 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | yamllint | 0m 0s | yamllint was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 52s | master passed | | +1 :green_heart: | compile | 3m 21s | master passed | | +1 :green_heart: | checkstyle | 2m 31s | master passed | | +1 :green_heart: | shadedjars | 5m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 35s | master passed | | +0 :ok: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 19m 41s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 35s | the patch passed | | +1 :green_heart: | compile | 3m 22s | the patch passed | | +1 :green_heart: | javac | 3m 22s | the patch passed | | +1 :green_heart: | checkstyle | 2m 29s | root: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | shadedjars | 0m 11s | patch has 7 errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 34s | the patch passed | | +1 :green_heart: | findbugs | 20m 4s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 76m 15s | root in the patch failed. | | -1 :x: | asflicense | 0m 49s | The patch generated 1 ASF License warnings. | | | | 179m 35s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.wal.TestFSHLog | | | hadoop.hbase.TestJMXConnectorServer | | | hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | | | hadoop.hbase.regionserver.TestHRegionServerBulkLoad | | | hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay | | | hadoop.hbase.regionserver.TestHeapMemoryManager | | | hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | | | hadoop.hbase.TestMultiVersions | | | hadoop.hbase.regionserver.wal.TestAsyncWALReplay | | | hadoop.hbase.regionserver.wal.TestWALReplayCompressed | | | hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod | | | hadoop.hbase.regionserver.wal.TestAsyncLogRolling | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1200 | | Optional Tests | dupname asflicense yamllint javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 222a3d1edc00 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1200/out/precommit/personality/provided.sh | | git revision | master / 3e21dc73de | | Default Java | 1.8.0_181 | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/artifact/out/patch-shadedjars.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/testReport/ | | asflicense | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 6439 (vs. ulimit of 1) | | modules | C: hbase-server . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1200/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[
https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044411#comment-17044411
]
HBase QA commented on HBASE-23864:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
38s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
57s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-2.1 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
45s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 1 new + 170
unchanged - 0 fixed = 171 total (was 170) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 29m 44s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1145/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23864 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994527/HBASE-23864.branch-2.1.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8409fa8f9c73 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / b81a1c1cb5 |
| Default Java | 1.8.0_181 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/1145/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
|
[GitHub] [hbase] virajjasani commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.
virajjasani commented on issue #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos. URL: https://github.com/apache/hbase/pull/1201#issuecomment-590837094 Overall, looks good This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.
virajjasani commented on a change in pull request #1201: HBASE-23639 : Moving
classes out of hbase-it /test for direct API use of chaos.
URL: https://github.com/apache/hbase/pull/1201#discussion_r383837705
##
File path:
hbase-it/src/main/java/org/apache/hadoop/hbase/HBaseClusterManager.java
##
@@ -43,8 +43,9 @@
* servers on the remote machines (for example, the test user could be the
same user as the
* user the daemon is running as)
*/
-@InterfaceAudience.Private
-public class HBaseClusterManager extends Configured implements ClusterManager {
+@InterfaceAudience.Public
+public class HBaseClusterManager extends Configured implements
+ org.apache.hadoop.hbase.ClusterManager {
Review comment:
nit: you can import it and then use
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani edited a comment on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
virajjasani edited a comment on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin… URL: https://github.com/apache/hbase/pull/1187#issuecomment-590827126 FYI @ramkrish86 @chenxu14 in case Reid's question is discussed before in parent or other relevant Jira. It is interesting and I don't have data/ans from perf POV. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin…
virajjasani commented on issue #1187: HBASE-23631 : Allow cache on write during compactions when prefetchin… URL: https://github.com/apache/hbase/pull/1187#issuecomment-590827126 FYI @ramkrish86 @chenxu14 in case Reid's question is discussed before in parent or other relevant Jira. It is interesting and I don't have data from perf POV. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23874) Move Jira-attached file precommit definition from script in Jenkins config to dev-support
[
https://issues.apache.org/jira/browse/HBASE-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044355#comment-17044355
]
Hudson commented on HBASE-23874:
Results for branch master
[build #1643 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1643/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1643//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1643//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1643//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/1643//console].
> Move Jira-attached file precommit definition from script in Jenkins config to
> dev-support
> -
>
> Key: HBASE-23874
> URL: https://issues.apache.org/jira/browse/HBASE-23874
> Project: HBase
> Issue Type: Task
> Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HBASE-23874.0.patch, HBASE-23874.1.branch-1.3.patch,
> HBASE-23874.1.branch-1.4.patch, HBASE-23874.1.branch-1.patch,
> HBASE-23874.1.branch-2.1.patch, HBASE-23874.1.branch-2.2.patch,
> HBASE-23874.1.branch-2.patch
>
>
> Right now the precommit job that scans Jira for attachments is defined
> entirely in the Jenkins job configuration. Let's move this into git so that
> changes can be tracked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
