[jira] [Created] (HBASE-26752) Fix flappy test TestSimpleRegionNormalizerOnCluster.java
Aman Poonia created HBASE-26752: --- Summary: Fix flappy test TestSimpleRegionNormalizerOnCluster.java Key: HBASE-26752 URL: https://issues.apache.org/jira/browse/HBASE-26752 Project: HBase Issue Type: Bug Components: Normalizer Affects Versions: 1.7.1 Reporter: Aman Poonia Assignee: Aman Poonia TestSimpleRegionNormalizerOnCluster.java can hang after HBASE-26744 The assumption that order of HTable list is sorted is wrong so depending on that order can cause the test to hang or be inaccurate -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4092: HBASE-26712 Balancer encounters NPE in rare case
Apache-HBase commented on pull request #4092: URL: https://github.com/apache/hbase/pull/4092#issuecomment-1035894849 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 51s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 43s | master passed | | +1 :green_heart: | compile | 1m 48s | master passed | | +1 :green_heart: | checkstyle | 0m 34s | master passed | | +1 :green_heart: | spotbugs | 1m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 21s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 32m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4092/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4092 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2250362cd574 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ccd908f218 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4092/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4106: HBASE-26545 Implement tracing of scan
Apache-HBase commented on pull request #4106: URL: https://github.com/apache/hbase/pull/4106#issuecomment-1035893774 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 42s | master passed | | +1 :green_heart: | compile | 2m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 53s | master passed | | +1 :green_heart: | spotbugs | 1m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 25s | the patch passed | | +1 :green_heart: | compile | 2m 31s | the patch passed | | -0 :warning: | javac | 0m 37s | hbase-client generated 1 new + 121 unchanged - 0 fixed = 122 total (was 121) | | -0 :warning: | checkstyle | 0m 17s | hbase-client: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) | | -0 :warning: | checkstyle | 0m 36s | hbase-server: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 35m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4106 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3d8a9463ea8d 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ccd908f218 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-client.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26696) Migrate our jenkins jobs from ci-hadoop to ci-hbase
[ https://issues.apache.org/jira/browse/HBASE-26696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490660#comment-17490660 ] Duo Zhang commented on HBASE-26696: --- OK, still another problem... We do not have zip command on hbase11-20... > Migrate our jenkins jobs from ci-hadoop to ci-hbase > --- > > Key: HBASE-26696 > URL: https://issues.apache.org/jira/browse/HBASE-26696 > Project: HBase > Issue Type: Task > Components: jenkins >Reporter: Duo Zhang >Priority: Major > > We have our own jenkins controller now. > Let's do the migration. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26749) Migrate HBase main pre commit job to ci-hbase
[ https://issues.apache.org/jira/browse/HBASE-26749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26749. --- Fix Version/s: 2.5.0 1.7.2 3.0.0-alpha-3 2.4.10 Hadoop Flags: Reviewed Resolution: Fixed Merged to all ative branches. Thanks [~busbey] for reviewing! > Migrate HBase main pre commit job to ci-hbase > - > > Key: HBASE-26749 > URL: https://issues.apache.org/jira/browse/HBASE-26749 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10 > > > We used to run this on the hadoop label so first we need to change the label > in jenkins file. > And before migration, we need to move the 10 hbase nodes from ci-hadoop to > ci-hbase as there are no jobs on them any more. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26697) Migrate HBase Nightly HBase-Flaky-Tests and HBase-Find-Flaky-Tests to ci-hbase
[ https://issues.apache.org/jira/browse/HBASE-26697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26697. --- Fix Version/s: 2.5.0 1.7.2 3.0.0-alpha-3 2.4.10 Hadoop Flags: Reviewed Resolution: Fixed The work is done. Thanks all for helping! > Migrate HBase Nightly HBase-Flaky-Tests and HBase-Find-Flaky-Tests to ci-hbase > -- > > Key: HBASE-26697 > URL: https://issues.apache.org/jira/browse/HBASE-26697 > Project: HBase > Issue Type: Sub-task > Components: jenkins >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10 > > > They are tied together so let's do them at once. > We also need to modify the url for our pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035858980 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 2m 42s | master passed | | +1 :green_heart: | shadedjars | 9m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 2m 44s | the patch passed | | +1 :green_heart: | javac | 2m 44s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 49s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. | | -1 :x: | unit | 221m 26s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 8m 12s | hbase-shell in the patch passed. | | | | 272m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6451725581dc 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/testReport/ | | Max. process+thread count | 2241 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4108: test pr
Apache9 commented on pull request #4108: URL: https://github.com/apache/hbase/pull/4108#issuecomment-1035856847 OK, good, at least it works. Although it seems that we need to rely on scan repository to find new PRs, while on ci-hadoop the github will nofity us. Will take a look later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 closed pull request #4108: test pr
Apache9 closed pull request #4108: URL: https://github.com/apache/hbase/pull/4108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4108: test pr
Apache-HBase commented on pull request #4108: URL: https://github.com/apache/hbase/pull/4108#issuecomment-1035854491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035791349 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 47s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | +1 :green_heart: | shadedjars | 8m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 41s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 37s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed | | +1 :green_heart: | javac | 3m 12s | the patch passed | | +1 :green_heart: | shadedjars | 8m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 3s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 26s | hbase-client in the patch passed. | | -1 :x: | unit | 139m 59s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 7m 12s | hbase-shell in the patch passed. | | | | 192m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 719835405ee4 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/testReport/ | | Max. process+thread count | 3644 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #4107: HBASE-26749 Migrate HBase main pre commit job to ci-hbase
Apache9 merged pull request #4107: URL: https://github.com/apache/hbase/pull/4107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26741) Incorrect exception handling in shell
[
https://issues.apache.org/jira/browse/HBASE-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490628#comment-17490628
]
Hudson commented on HBASE-26741:
Results for branch master
[build #515 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect exception handling in shell
> -
>
> Key: HBASE-26741
> URL: https://issues.apache.org/jira/browse/HBASE-26741
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.10
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> The exception handling changed in the shell compared to 2.2.
> {noformat}
> ➜ hbase-upstream git:(branch-2.4) ✗ cat commands.txt
> scan 'foo'
> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ bin/hbase shell -n commands.txt
> 2022-02-07 16:21:33,654 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> hbase:001:0> scan 'foo'
> ROW COLUMN+CELL
> Took 0.3890 seconds
> Traceback (most
> recent call last):
> RuntimeError (Unknown table foo!)
> hbase:002:0> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ echo $?
> 0 {noformat}
> The execution continues even after an exception is thrown. In 2.2.7 the
> execution stops when an exception is thrown and the shell exits with error
> code.
> {noformat}
> ➜ hbase-2.2.7 bin/hbase shell -n commands.txt
> 2022-02-07 16:33:54,930 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW COLUMN+CELL
> Took 0.3082 seconds
> RuntimeError:
> Unknown table foo!
> translate_hbase_exceptions at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:130
> command_safe at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:49
> internal_command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:148
> command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:140
> scan at (eval):2
> at commands.txt:1
> load at org/jruby/RubyKernel.java:973
> at
> /Users/petersomogyi/tmp/hbase-2.2.7/bin/../bin/hirb.rb:186
> ➜ hbase-2.2.7 echo $?
> 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26747) Use python2 instead of python in our python scripts
[
https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490629#comment-17490629
]
Hudson commented on HBASE-26747:
Results for branch master
[build #515 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/515/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use python2 instead of python in our python scripts
> ---
>
> Key: HBASE-26747
> URL: https://issues.apache.org/jira/browse/HBASE-26747
> Project: HBase
> Issue Type: Sub-task
> Components: jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10
>
>
> According to the suggestion from INFRA team, let's specify python2 directly.
> Later we could see if we can migrate to python3.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (HBASE-26644) Spurious compaction failures with file tracker
[
https://issues.apache.org/jira/browse/HBASE-26644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser resolved HBASE-26644.
Resolution: Not A Problem
Yep, all good. I believe you fixed this in HBASE-26675
> Spurious compaction failures with file tracker
> --
>
> Key: HBASE-26644
> URL: https://issues.apache.org/jira/browse/HBASE-26644
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
>
> Noticed when running a basic {{{}hbase pe randomWrite{}}}, we'll see
> compactions failing at various points.
> One example:
> {noformat}
> 2022-01-03 17:41:18,319 ERROR
> [regionserver/localhost:16020-shortCompactions-0]
> regionserver.CompactSplit(670): Compaction failed
> region=TestTable,0004054490,1641249249856.2dc7251c6eceb660b9c7bb0b587db913.,
> storeName=2dc7251c6eceb660b9c7bb0b587db913/info0, priority=6,
> startTime=1641249666161
> java.io.IOException: Root-level entries already added in single-level mode
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexWriter.writeSingleLevelIndex(HFileBlockIndex.java:1136)
> at
> org.apache.hadoop.hbase.io.hfile.CompoundBloomFilterWriter$MetaWriter.write(CompoundBloomFilterWriter.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterImpl$1.writeToBlock(HFileWriterImpl.java:713)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.writeBlock(HFileBlock.java:1205)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:660)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:377)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:70)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:386)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:62)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:125)
> at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1141)
> at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2388)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:654)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:697)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) {noformat}
> This isn't a super-critical issue because compactions will be retried
> automatically and they appear to eventually succeed. However, when the max
> storefiles limit is reaching, this does cause ingest to hang (as I was doing
> with my modest configuration).
> We had seen a similar kind of problem in our testing when backporting to
> HBase 2.4 (not upstream as the decision was to not do this) which we
> eventually tracked down to a bad merge-conflict resolution to the new HFile
> Cleaner. However, initial investigations don't have the same exact problem.
> It seems that we have some kind of generic race condition. Would be good to
> add more logging to catch this in the future (since we have two separate
> instances of this category of bug already).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26521) Name RPC spans as `$package.$service/$method`
[
https://issues.apache.org/jira/browse/HBASE-26521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490614#comment-17490614
]
Hudson commented on HBASE-26521:
Results for branch branch-2.5
[build #46 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Name RPC spans as `$package.$service/$method`
> -
>
> Key: HBASE-26521
> URL: https://issues.apache.org/jira/browse/HBASE-26521
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> As per
> https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/rpc.md#span-name
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26741) Incorrect exception handling in shell
[
https://issues.apache.org/jira/browse/HBASE-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490613#comment-17490613
]
Hudson commented on HBASE-26741:
Results for branch branch-2.5
[build #46 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/46/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect exception handling in shell
> -
>
> Key: HBASE-26741
> URL: https://issues.apache.org/jira/browse/HBASE-26741
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.10
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> The exception handling changed in the shell compared to 2.2.
> {noformat}
> ➜ hbase-upstream git:(branch-2.4) ✗ cat commands.txt
> scan 'foo'
> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ bin/hbase shell -n commands.txt
> 2022-02-07 16:21:33,654 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> hbase:001:0> scan 'foo'
> ROW COLUMN+CELL
> Took 0.3890 seconds
> Traceback (most
> recent call last):
> RuntimeError (Unknown table foo!)
> hbase:002:0> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ echo $?
> 0 {noformat}
> The execution continues even after an exception is thrown. In 2.2.7 the
> execution stops when an exception is thrown and the shell exits with error
> code.
> {noformat}
> ➜ hbase-2.2.7 bin/hbase shell -n commands.txt
> 2022-02-07 16:33:54,930 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW COLUMN+CELL
> Took 0.3082 seconds
> RuntimeError:
> Unknown table foo!
> translate_hbase_exceptions at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:130
> command_safe at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:49
> internal_command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:148
> command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:140
> scan at (eval):2
> at commands.txt:1
> load at org/jruby/RubyKernel.java:973
> at
> /Users/petersomogyi/tmp/hbase-2.2.7/bin/../bin/hirb.rb:186
> ➜ hbase-2.2.7 echo $?
> 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26741) Incorrect exception handling in shell
[
https://issues.apache.org/jira/browse/HBASE-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490598#comment-17490598
]
Hudson commented on HBASE-26741:
Results for branch branch-2.4
[build #286 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/286/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/286/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/286/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/286/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/286/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect exception handling in shell
> -
>
> Key: HBASE-26741
> URL: https://issues.apache.org/jira/browse/HBASE-26741
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.10
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> The exception handling changed in the shell compared to 2.2.
> {noformat}
> ➜ hbase-upstream git:(branch-2.4) ✗ cat commands.txt
> scan 'foo'
> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ bin/hbase shell -n commands.txt
> 2022-02-07 16:21:33,654 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> hbase:001:0> scan 'foo'
> ROW COLUMN+CELL
> Took 0.3890 seconds
> Traceback (most
> recent call last):
> RuntimeError (Unknown table foo!)
> hbase:002:0> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ echo $?
> 0 {noformat}
> The execution continues even after an exception is thrown. In 2.2.7 the
> execution stops when an exception is thrown and the shell exits with error
> code.
> {noformat}
> ➜ hbase-2.2.7 bin/hbase shell -n commands.txt
> 2022-02-07 16:33:54,930 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW COLUMN+CELL
> Took 0.3082 seconds
> RuntimeError:
> Unknown table foo!
> translate_hbase_exceptions at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:130
> command_safe at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:49
> internal_command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:148
> command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:140
> scan at (eval):2
> at commands.txt:1
> load at org/jruby/RubyKernel.java:973
> at
> /Users/petersomogyi/tmp/hbase-2.2.7/bin/../bin/hirb.rb:186
> ➜ hbase-2.2.7 echo $?
> 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26521) Name RPC spans as `$package.$service/$method`
[
https://issues.apache.org/jira/browse/HBASE-26521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490595#comment-17490595
]
Hudson commented on HBASE-26521:
Results for branch branch-2
[build #462 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Name RPC spans as `$package.$service/$method`
> -
>
> Key: HBASE-26521
> URL: https://issues.apache.org/jira/browse/HBASE-26521
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> As per
> https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/rpc.md#span-name
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26747) Use python2 instead of python in our python scripts
[
https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490596#comment-17490596
]
Hudson commented on HBASE-26747:
Results for branch branch-2
[build #462 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use python2 instead of python in our python scripts
> ---
>
> Key: HBASE-26747
> URL: https://issues.apache.org/jira/browse/HBASE-26747
> Project: HBase
> Issue Type: Sub-task
> Components: jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10
>
>
> According to the suggestion from INFRA team, let's specify python2 directly.
> Later we could see if we can migrate to python3.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26741) Incorrect exception handling in shell
[
https://issues.apache.org/jira/browse/HBASE-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490594#comment-17490594
]
Hudson commented on HBASE-26741:
Results for branch branch-2
[build #462 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/462/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect exception handling in shell
> -
>
> Key: HBASE-26741
> URL: https://issues.apache.org/jira/browse/HBASE-26741
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.10
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> The exception handling changed in the shell compared to 2.2.
> {noformat}
> ➜ hbase-upstream git:(branch-2.4) ✗ cat commands.txt
> scan 'foo'
> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ bin/hbase shell -n commands.txt
> 2022-02-07 16:21:33,654 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> hbase:001:0> scan 'foo'
> ROW COLUMN+CELL
> Took 0.3890 seconds
> Traceback (most
> recent call last):
> RuntimeError (Unknown table foo!)
> hbase:002:0> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ echo $?
> 0 {noformat}
> The execution continues even after an exception is thrown. In 2.2.7 the
> execution stops when an exception is thrown and the shell exits with error
> code.
> {noformat}
> ➜ hbase-2.2.7 bin/hbase shell -n commands.txt
> 2022-02-07 16:33:54,930 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW COLUMN+CELL
> Took 0.3082 seconds
> RuntimeError:
> Unknown table foo!
> translate_hbase_exceptions at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:130
> command_safe at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:49
> internal_command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:148
> command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:140
> scan at (eval):2
> at commands.txt:1
> load at org/jruby/RubyKernel.java:973
> at
> /Users/petersomogyi/tmp/hbase-2.2.7/bin/../bin/hirb.rb:186
> ➜ hbase-2.2.7 echo $?
> 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035676112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 52s | master passed | | +1 :green_heart: | compile | 6m 3s | master passed | | +1 :green_heart: | checkstyle | 2m 2s | master passed | | +1 :green_heart: | spotbugs | 7m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 6m 6s | the patch passed | | +1 :green_heart: | cc | 6m 6s | the patch passed | | -0 :warning: | javac | 0m 57s | hbase-client generated 5 new + 116 unchanged - 5 fixed = 121 total (was 121) | | -0 :warning: | checkstyle | 0m 32s | hbase-client: The patch generated 1 new + 181 unchanged - 0 fixed = 182 total (was 181) | | -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 1 new + 87 unchanged - 0 fixed = 88 total (was 87) | | -0 :warning: | rubocop | 0m 11s | The patch generated 33 new + 371 unchanged - 7 fixed = 404 total (was 378) | | -0 :warning: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 19m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | hbaseprotoc | 2m 35s | the patch passed | | +1 :green_heart: | spotbugs | 8m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 72m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 08b305ca1489 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035612106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 32s | master passed | | +1 :green_heart: | compile | 2m 42s | master passed | | +1 :green_heart: | shadedjars | 9m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 20s | the patch passed | | +1 :green_heart: | compile | 2m 42s | the patch passed | | +1 :green_heart: | javac | 2m 42s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 50s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 37s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 217m 39s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 8m 4s | hbase-shell in the patch passed. | | | | 267m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6c28ecd5f88d 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/testReport/ | | Max. process+thread count | 2402 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4094: URL: https://github.com/apache/hbase/pull/4094#issuecomment-1035557111 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 52s | branch-2 passed | | +1 :green_heart: | compile | 2m 39s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 35s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 2m 37s | the patch passed | | +1 :green_heart: | javac | 2m 37s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 43s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 44s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 147m 39s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 7m 28s | hbase-shell in the patch passed. | | | | 191m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4094 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1fdadf409d65 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 83f890fe0e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/testReport/ | | Max. process+thread count | 3943 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4094: URL: https://github.com/apache/hbase/pull/4094#issuecomment-1035552675 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 26s | branch-2 passed | | +1 :green_heart: | compile | 3m 7s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 50s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 33s | the patch passed | | +1 :green_heart: | compile | 3m 22s | the patch passed | | +1 :green_heart: | javac | 3m 22s | the patch passed | | +1 :green_heart: | shadedjars | 7m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 2s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 47s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 137m 16s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 7m 11s | hbase-shell in the patch passed. | | | | 187m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4094 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 34779ec2a57f 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 83f890fe0e | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/testReport/ | | Max. process+thread count | 4013 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035539517 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 32s | master passed | | +1 :green_heart: | compile | 3m 10s | master passed | | +1 :green_heart: | shadedjars | 8m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 32s | the patch passed | | +1 :green_heart: | compile | 3m 9s | the patch passed | | +1 :green_heart: | javac | 3m 9s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 1s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 25s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 139m 35s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 7m 9s | hbase-shell in the patch passed. | | | | 188m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5e443e780b99 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/testReport/ | | Max. process+thread count | 3559 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-26745) MetricsStochasticBalancerSource metrics don't render in /jmx endpoint
[
https://issues.apache.org/jira/browse/HBASE-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Briana Augenreich reassigned HBASE-26745:
-
Assignee: Briana Augenreich
> MetricsStochasticBalancerSource metrics don't render in /jmx endpoint
> -
>
> Key: HBASE-26745
> URL: https://issues.apache.org/jira/browse/HBASE-26745
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Briana Augenreich
>Priority: Minor
>
> The check in JSONBean
> [here|https://github.com/apache/hbase/blob/master/hbase-http/src/main/java/org/apache/hadoop/hbase/util/JSONBean.java#L266]
> causes the balancer cost metrics to be excluded from the {{/jmx}} endpoint.
> I think we should just remove the check for ":". I don't think including
> metrics with a colon in them poses any problem, since the strings are quoted.
> We removed this check in our cdh5 fork almost a year ago and have not seen
> any issues.
> Alternatively we can fix the MetricsStochasticBalancerSource metrics to emit
> metrics that adhere to these filters, but I think that would be a backwards
> incompatible change since this is only a json rendering issue and people
> might be consuming JMX directly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4094: URL: https://github.com/apache/hbase/pull/4094#issuecomment-1035411691 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 7s | branch-2 passed | | +1 :green_heart: | compile | 6m 9s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 13s | branch-2 passed | | +1 :green_heart: | spotbugs | 6m 22s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 6m 14s | the patch passed | | +1 :green_heart: | cc | 6m 14s | the patch passed | | +1 :green_heart: | javac | 6m 14s | the patch passed | | -0 :warning: | checkstyle | 0m 37s | hbase-client: The patch generated 1 new + 178 unchanged - 0 fixed = 179 total (was 178) | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89) | | -0 :warning: | rubocop | 0m 11s | The patch generated 33 new + 379 unchanged - 8 fixed = 412 total (was 387) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 1s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 25s | the patch passed | | +1 :green_heart: | spotbugs | 6m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 43s | The patch does not generate ASF License warnings. | | | | 66m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4094 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 2bd8953404db 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 83f890fe0e | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 86 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4094/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
Apache-HBase commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035399913 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 51s | master passed | | +1 :green_heart: | compile | 6m 2s | master passed | | +1 :green_heart: | checkstyle | 2m 1s | master passed | | +1 :green_heart: | spotbugs | 7m 22s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 6m 2s | the patch passed | | +1 :green_heart: | cc | 6m 2s | the patch passed | | -0 :warning: | javac | 0m 56s | hbase-client generated 5 new + 116 unchanged - 5 fixed = 121 total (was 121) | | -0 :warning: | checkstyle | 0m 32s | hbase-client: The patch generated 1 new + 181 unchanged - 0 fixed = 182 total (was 181) | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 1 new + 87 unchanged - 0 fixed = 88 total (was 87) | | -0 :warning: | rubocop | 0m 11s | The patch generated 37 new + 371 unchanged - 7 fixed = 408 total (was 378) | | -0 :warning: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 19m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | hbaseprotoc | 2m 35s | the patch passed | | +1 :green_heart: | spotbugs | 7m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 73m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4095 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 58644b58a61b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0c9ec2e030 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4095/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on pull request #4094: URL: https://github.com/apache/hbase/pull/4094#issuecomment-1035298545 Rebased and addresses @gjacoby126 's feedback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #4095: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on pull request #4095: URL: https://github.com/apache/hbase/pull/4095#issuecomment-1035284469 Address @gjacoby126 's feedback from #4094 and rebased. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4106: HBASE-26545 Implement tracing of scan
Apache-HBase commented on pull request #4106: URL: https://github.com/apache/hbase/pull/4106#issuecomment-1035283440 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 8m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 150m 12s | hbase-server in the patch passed. | | | | 185m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4106 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e384629f1ac7 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 85fadfde26 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/testReport/ | | Max. process+thread count | 3654 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4106: HBASE-26545 Implement tracing of scan
Apache-HBase commented on pull request #4106: URL: https://github.com/apache/hbase/pull/4106#issuecomment-1035280847 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 30s | master passed | | +1 :green_heart: | compile | 2m 5s | master passed | | +1 :green_heart: | shadedjars | 9m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 20s | the patch passed | | +1 :green_heart: | compile | 2m 2s | the patch passed | | +1 :green_heart: | javac | 2m 2s | the patch passed | | +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-client in the patch passed. | | -1 :x: | unit | 142m 17s | hbase-server in the patch failed. | | | | 184m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4106 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 821ae282038a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 85fadfde26 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/testReport/ | | Max. process+thread count | 3740 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803969249
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
I did take up your suggestion to D.R.Y this and was able to remove some
lines, it looks good @gjacoby126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
I suppose it is possible to collapse this into a single code block,
would that be better? E.g.
boolean doTasks = false;
case TASKS:
doTasks = true;
// fall through
case LIVE_SERVERS: {
...
// do stuff
if (doTasks) {
// do tasks stuff too, like the master monitoredtask collection
...
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
You mean here in this compilation unit? It's a little different in each
location and pretty well handled, I think. We use a map to collect the results
from the processing of either Option alternative and a boolean to avoid double
processing during the processing of the Option alternatives. Does it improve
readability to make a two or three line private method that is called twice?
I suppose it is possible to collapse this into a single code block, would
that be better? E.g.
boolean doTasks = false;
case TASKS:
doTasks = true;
// fall through
case LIVE_SERVERS: {
...
// do stuff
if (doTasks) {
// do tasks stuff too, like the master monitoredtask collection
...
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
You mean here in this compilation unit? It's a little different in each
location and pretty well handled, I think. We use a map to collect the results
from the processing of either Option alternative and a boolean to avoid double
processing during the processing of the Option alternatives. Does it improve
readability to make a two or three line private method that is called twice?
I suppose it is possible to collapse this into a single code block, would
that be better? E.g.
boolean doTasks = false;
case TASKS:
doTasks = true;
// fall through
case LIVE_SERVERS: {
...
// do stuff
if (doTasks) {
// do tasks stuff too, like the master monitoredtask collection
...
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26751) Tests for shell exit behavior
[ https://issues.apache.org/jira/browse/HBASE-26751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490405#comment-17490405 ] Sean Busbey commented on HBASE-26751: - we should do this as integration tests with the nightly test cluster instance rather than try to do it under junit. > Tests for shell exit behavior > - > > Key: HBASE-26751 > URL: https://issues.apache.org/jira/browse/HBASE-26751 > Project: HBase > Issue Type: Test > Components: shell, test >Reporter: Peter Somogyi >Priority: Major > > HBase shell exit behavior and exception handling is not covered with tests. > The tests should cover the clean exits and also when a command throws > exceptions in interactive and non-interactive modes. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
You mean here in this compilation unit? It's a little different in each
location and pretty well handled, I think. We use a shared map to collect the
results and a boolean to avoid double processing during the processing of the
Option alternatives. Does it improve readability to make a two or three line
private method that is called twice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
What duplication? You mean here in this compilation unit? It's a little
different in each location and pretty well handled, I think. We use a shared
map to collect the results and a boolean to avoid double processing during the
processing of the Option alternatives. Does it improve readability to make a
two or three line private method that is called twice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803942597
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2712,16 +2714,52 @@ public ClusterMetrics
getClusterMetricsWithoutCoprocessor(EnumSet option
options = EnumSet.allOf(Option.class);
}
+// TASKS and/or LIVE_SERVERS will populate this map, which will be given
to the builder if
+// not null after option processing completes.
+Map serverMetricsMap = null;
+boolean processedLiveServers = false;
+
for (Option opt : options) {
switch (opt) {
case HBASE_VERSION: builder.setHBaseVersion(VersionInfo.getVersion());
break;
case CLUSTER_ID: builder.setClusterId(getClusterId()); break;
case MASTER: builder.setMasterName(getServerName()); break;
case BACKUP_MASTERS: builder.setBackerMasterNames(getBackupMasters());
break;
+case TASKS: {
+ // Master tasks
+ builder.setMasterTasks(TaskMonitor.get().getTasks().stream()
+.map(task -> ServerTaskBuilder.newBuilder()
+ .setDescription(task.getDescription())
+ .setStatus(task.getStatus())
+ .setState(ServerTask.State.valueOf(task.getState().name()))
+ .setStartTime(task.getStartTime())
+ .setCompletionTime(task.getCompletionTimestamp())
+ .build())
+.collect(Collectors.toList()));
+ // TASKS is also synonymous with LIVE_SERVERS for now because task
information for
+ // regionservers is carried in ServerLoad.
+ // Add entries to serverMetricsMap for all live servers
+ if (serverManager != null && !processedLiveServers) {
+if (serverMetricsMap == null) {
+ serverMetricsMap = new HashMap<>();
+}
+final Map map = serverMetricsMap;
+serverManager.getOnlineServers().entrySet()
Review comment:
What duplication? The code is different in each location.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4094: HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks'
apurtell commented on a change in pull request #4094:
URL: https://github.com/apache/hbase/pull/4094#discussion_r803941792
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2699,11 +2701,11 @@ public void checkTableModifiable(final TableName
tableName)
}
}
- public ClusterMetrics getClusterMetricsWithoutCoprocessor() throws
InterruptedIOException {
-return getClusterMetricsWithoutCoprocessor(EnumSet.allOf(Option.class));
+ public ClusterMetrics getClusterMetricsInternal() throws
InterruptedIOException {
+return getClusterMetricsInternal(EnumSet.allOf(Option.class));
}
- public ClusterMetrics getClusterMetricsWithoutCoprocessor(EnumSet
options)
+ public ClusterMetrics getClusterMetricsInternal(EnumSet options)
Review comment:
This isn't a required change, I just did it because everywhere else in
the code the convention is fooInternal, not fooWithoutCoprocessor, and I find
it weird and worth changing. I doubt there are any TOOLS that require this
method but this change isn't necessary either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-26750) The branch-1 nightly jenkins job fails with "No such DSL method 'jiraComment' found"
[ https://issues.apache.org/jira/browse/HBASE-26750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26750. --- Assignee: Duo Zhang Resolution: Fixed I installed jira plugin on ci-hbase and also configured the jira site. Now it works, see HBASE-26747. Resolved. > The branch-1 nightly jenkins job fails with "No such DSL method 'jiraComment' > found" > > > Key: HBASE-26750 > URL: https://issues.apache.org/jira/browse/HBASE-26750 > Project: HBase > Issue Type: Sub-task > Components: jenkins >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26709) Ban the usage of junit 3 TestCase
[
https://issues.apache.org/jira/browse/HBASE-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490356#comment-17490356
]
Hudson commented on HBASE-26709:
Results for branch branch-2.5
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Ban the usage of junit 3 TestCase
> -
>
> Key: HBASE-26709
> URL: https://issues.apache.org/jira/browse/HBASE-26709
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> We should move up all our UTs to junit 4 style.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-24510) Remove HBaseTestCase and GenericTestUtils
[
https://issues.apache.org/jira/browse/HBASE-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490355#comment-17490355
]
Hudson commented on HBASE-24510:
Results for branch branch-2.5
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove HBaseTestCase and GenericTestUtils
> -
>
> Key: HBASE-24510
> URL: https://issues.apache.org/jira/browse/HBASE-24510
> Project: HBase
> Issue Type: Task
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> It is still a junit3 style test base, let's remove it.
> GenericTestUtils is also useless, remove it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26724) Backport the UT changes in HBASE-24510 to branch-2.x
[
https://issues.apache.org/jira/browse/HBASE-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490354#comment-17490354
]
Hudson commented on HBASE-26724:
Results for branch branch-2.5
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Backport the UT changes in HBASE-24510 to branch-2.x
>
>
> Key: HBASE-26724
> URL: https://issues.apache.org/jira/browse/HBASE-26724
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0
>
>
> We could still keep HBaseTestCase but we can modify our own UTs to not rely
> on it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26747) Use python2 instead of python in our python scripts
[
https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490353#comment-17490353
]
Hudson commented on HBASE-26747:
Results for branch branch-2.5
[build #45 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/45/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use python2 instead of python in our python scripts
> ---
>
> Key: HBASE-26747
> URL: https://issues.apache.org/jira/browse/HBASE-26747
> Project: HBase
> Issue Type: Sub-task
> Components: jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10
>
>
> According to the suggestion from INFRA team, let's specify python2 directly.
> Later we could see if we can migrate to python3.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26747) Use python2 instead of python in our python scripts
[ https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26747: -- Component/s: jenkins > Use python2 instead of python in our python scripts > --- > > Key: HBASE-26747 > URL: https://issues.apache.org/jira/browse/HBASE-26747 > Project: HBase > Issue Type: Sub-task > Components: jenkins >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10 > > > According to the suggestion from INFRA team, let's specify python2 directly. > Later we could see if we can migrate to python3. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26747) Use python2 instead of python in our python scripts
[ https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490350#comment-17490350 ] Duo Zhang commented on HBASE-26747: --- OK, good. It works. Let me push to all branches and resolve this issue. > Use python2 instead of python in our python scripts > --- > > Key: HBASE-26747 > URL: https://issues.apache.org/jira/browse/HBASE-26747 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10 > > > According to the suggestion from INFRA team, let's specify python2 directly. > Later we could see if we can migrate to python3. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26747) Use python2 instead of python in our python scripts
[
https://issues.apache.org/jira/browse/HBASE-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490347#comment-17490347
]
Hudson commented on HBASE-26747:
Results for branch HBASE-26747
[build #6 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26747/6/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26747/6/General_20Nightly_20Build_20Report/]
> Use python2 instead of python in our python scripts
> ---
>
> Key: HBASE-26747
> URL: https://issues.apache.org/jira/browse/HBASE-26747
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.10
>
>
> According to the suggestion from INFRA team, let's specify python2 directly.
> Later we could see if we can migrate to python3.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4106: HBASE-26545 Implement tracing of scan
Apache-HBase commented on pull request #4106: URL: https://github.com/apache/hbase/pull/4106#issuecomment-1035108068 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | compile | 4m 17s | master passed | | +1 :green_heart: | checkstyle | 1m 33s | master passed | | +1 :green_heart: | spotbugs | 3m 7s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 4m 12s | the patch passed | | -0 :warning: | javac | 0m 56s | hbase-client generated 1 new + 121 unchanged - 0 fixed = 122 total (was 121) | | -0 :warning: | checkstyle | 0m 28s | hbase-client: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) | | -0 :warning: | checkstyle | 1m 1s | hbase-server: The patch generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 3m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 56m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4106 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8bd4401ed656 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 85fadfde26 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4106/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-26648) Improve fidelity of RegionLocator spans
[
https://issues.apache.org/jira/browse/HBASE-26648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-26648 started by Nick Dimiduk.
> Improve fidelity of RegionLocator spans
> ---
>
> Key: HBASE-26648
> URL: https://issues.apache.org/jira/browse/HBASE-26648
> Project: HBase
> Issue Type: Sub-task
> Components: tracing
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> While backporting HBASE-26474 to branch-2, I realized that the nuance of
> tracing of region locator calls is lacking. In the current implementation,
> all calls to the region locator produce {{SpanKind.CLIENT}} spans. This is
> correct only for the code path where the client reaches out to a remote host
> to populate cache locations. This doesn't happen every time.
> Instead, the top-level span should be {{SpanKind.INTERNAL}} and only when a
> region location must be requested remotely should we create a child span with
> {{SpanKind.CLIENT}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26545) Implement tracing of scan
[ https://issues.apache.org/jira/browse/HBASE-26545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-26545: - Status: Patch Available (was: In Progress) > Implement tracing of scan > - > > Key: HBASE-26545 > URL: https://issues.apache.org/jira/browse/HBASE-26545 > Project: HBase > Issue Type: Sub-task > Components: tracing >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Naive tracing of a long scan, where each back-and-forth between client and > server gets its own Span, can result in many 100's or 1000's of Spans. This > is easily overwhelming for tracing infrastructure. Identify some solution > that allows for a reasonable out-of-the-box behavior when a scan is to be > traced. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk opened a new pull request #4106: HBASE-26545 Implement tracing of scan
ndimiduk opened a new pull request #4106: URL: https://github.com/apache/hbase/pull/4106 * on `AsyncTable`, both `scan` and `scanAll` methods should result in `SCAN` table operations. * the span of the `SCAN` table operation should have children representing all the RPC calls involved in servicing the scan. * when a user provides custom implementation of `AdvancedScanResultConsumer`, any spans emitted from the callback methods should also be tied to the span that represents the `SCAN` table operation. This is easily done because these callbacks are executed on the RPC thread. * when a user provides a custom implementation of `ScanResultConsumer`, any spans emitted from the callback methods are tied to the context of user application code. We could `link` them to the `SCAN` table operation, except that we have no means of passing that span along. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26073) Avoid merging regions if tables has not reached the state where regions are not split because of number of regions
[ https://issues.apache.org/jira/browse/HBASE-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490298#comment-17490298 ] Nick Dimiduk commented on HBASE-26073: -- Hey there [~mnpoonia], is this still an issue for you with the current state of the normalizer? Thanks. > Avoid merging regions if tables has not reached the state where regions are > not split because of number of regions > -- > > Key: HBASE-26073 > URL: https://issues.apache.org/jira/browse/HBASE-26073 > Project: HBase > Issue Type: Improvement > Components: Normalizer >Affects Versions: 1.7.0, 2.4.4 >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > > we have a table on a cluster with 100 regions with default split policy > (SteppingSplitPolicy). This is a small table and will not get loaded with too > much data. Now if region size of table is smaller than the normalizer target > region size than there are chances that normalizer will consider the regions > for merges. But since the number of regions are small split policy will > trigger split on next flush. This is a continuous loop and our cluster will > be busy in these two actions. > We plan to consider number of regions and number of regions in creating plans > for normalizer. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-24146) Run a perf comparison of 2.3.x with 2.2.x
[ https://issues.apache.org/jira/browse/HBASE-24146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24146. -- Resolution: Won't Do > Run a perf comparison of 2.3.x with 2.2.x > - > > Key: HBASE-24146 > URL: https://issues.apache.org/jira/browse/HBASE-24146 > Project: HBase > Issue Type: Task > Components: community, Performance >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Priority: Major > > Kick the tires on a recent build. Compare results with a run on the latest > 2.2.x. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4038: HBASE-26552 Introduce retry to logroller to avoid abort
Apache-HBase commented on pull request #4038: URL: https://github.com/apache/hbase/pull/4038#issuecomment-1034931034 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 7s | master passed | | +1 :green_heart: | shadedjars | 9m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 21s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 9m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 211m 47s | hbase-server in the patch passed. | | | | 245m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4038 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cf47b58532a4 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6c3c53a81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/testReport/ | | Max. process+thread count | 2739 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26521) Name RPC spans as `$package.$service/$method`
[ https://issues.apache.org/jira/browse/HBASE-26521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-26521: - Resolution: Fixed Status: Resolved (was: Patch Available) > Name RPC spans as `$package.$service/$method` > - > > Key: HBASE-26521 > URL: https://issues.apache.org/jira/browse/HBASE-26521 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > As per > https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/rpc.md#span-name -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26521) Name RPC spans as `$package.$service/$method`
[ https://issues.apache.org/jira/browse/HBASE-26521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-26521: - Fix Version/s: 2.5.0 2.6.0 > Name RPC spans as `$package.$service/$method` > - > > Key: HBASE-26521 > URL: https://issues.apache.org/jira/browse/HBASE-26521 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > As per > https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/rpc.md#span-name -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk merged pull request #4104: Backport "HBASE-26521 Name RPC spans as `$package.$service/$method` (#4024)" to branch-2.5
ndimiduk merged pull request #4104: URL: https://github.com/apache/hbase/pull/4104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #4103: Backport "HBASE-26521 Name RPC spans as `$package.$service/$method` (#4024)" to branch-2
ndimiduk merged pull request #4103: URL: https://github.com/apache/hbase/pull/4103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26741) Incorrect exception handling in shell
[
https://issues.apache.org/jira/browse/HBASE-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi resolved HBASE-26741.
---
Fix Version/s: 2.5.0
3.0.0-alpha-3
2.4.10
Resolution: Fixed
Merged to branch-2.4+. Thanks for the review [~elserj].
Filed HBASE-26751 to cover the shell exit behavior with tests.
> Incorrect exception handling in shell
> -
>
> Key: HBASE-26741
> URL: https://issues.apache.org/jira/browse/HBASE-26741
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.10
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.10
>
>
> The exception handling changed in the shell compared to 2.2.
> {noformat}
> ➜ hbase-upstream git:(branch-2.4) ✗ cat commands.txt
> scan 'foo'
> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ bin/hbase shell -n commands.txt
> 2022-02-07 16:21:33,654 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> hbase:001:0> scan 'foo'
> ROW COLUMN+CELL
> Took 0.3890 seconds
> Traceback (most
> recent call last):
> RuntimeError (Unknown table foo!)
> hbase:002:0> exit
> ➜ hbase-upstream git:(branch-2.4) ✗ echo $?
> 0 {noformat}
> The execution continues even after an exception is thrown. In 2.2.7 the
> execution stops when an exception is thrown and the shell exits with error
> code.
> {noformat}
> ➜ hbase-2.2.7 bin/hbase shell -n commands.txt
> 2022-02-07 16:33:54,930 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> ROW COLUMN+CELL
> Took 0.3082 seconds
> RuntimeError:
> Unknown table foo!
> translate_hbase_exceptions at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:130
> command_safe at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell/commands.rb:49
> internal_command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:148
> command at
> /Users/petersomogyi/tmp/hbase-2.2.7/lib/ruby/shell.rb:140
> scan at (eval):2
> at commands.txt:1
> load at org/jruby/RubyKernel.java:973
> at
> /Users/petersomogyi/tmp/hbase-2.2.7/bin/../bin/hirb.rb:186
> ➜ hbase-2.2.7 echo $?
> 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26751) Tests for shell exit behavior
Peter Somogyi created HBASE-26751: - Summary: Tests for shell exit behavior Key: HBASE-26751 URL: https://issues.apache.org/jira/browse/HBASE-26751 Project: HBase Issue Type: Test Components: shell, test Reporter: Peter Somogyi HBase shell exit behavior and exception handling is not covered with tests. The tests should cover the clean exits and also when a command throws exceptions in interactive and non-interactive modes. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] petersomogyi merged pull request #4101: HBASE-26741 Incorrect exception handling in shell
petersomogyi merged pull request #4101: URL: https://github.com/apache/hbase/pull/4101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4038: HBASE-26552 Introduce retry to logroller to avoid abort
Apache-HBase commented on pull request #4038: URL: https://github.com/apache/hbase/pull/4038#issuecomment-1034742993 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 3m 23s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 3m 10s | the patch passed | | +1 :green_heart: | javac | 3m 10s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 50m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4038 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux acc3e063c78c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6c3c53a81c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4038/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26744) Normalizer exits without normalizing all the tables
[
https://issues.apache.org/jira/browse/HBASE-26744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha resolved HBASE-26744.
Fix Version/s: 1.8.0
Resolution: Fixed
Merged to branch-1, thanks [~mnpoonia] for contributing.
> Normalizer exits without normalizing all the tables
> ---
>
> Key: HBASE-26744
> URL: https://issues.apache.org/jira/browse/HBASE-26744
> Project: HBase
> Issue Type: Bug
> Components: Normalizer
>Affects Versions: 1.7.1
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 1.8.0
>
>
> Currently if there are multiple tables to normalize, normalizer is exiting
> even before iterating all the table if there is a table that doesnot require
> normalization
> here is the offending code
> [https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1736]
>
> {code:java}
> List plans = this.normalizer.computePlansForTable(table);
>
> if (plans == null || plans.isEmpty()) {
>return true;
> } {code}
> this is running inside loop over tables
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] sunhelly merged pull request #4100: HBASE-26744 Normalizer exits without normalizing all the tables
sunhelly merged pull request #4100: URL: https://github.com/apache/hbase/pull/4100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26552) Introduce retry to logroller to avoid abort
[
https://issues.apache.org/jira/browse/HBASE-26552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26552:
---
Summary: Introduce retry to logroller to avoid abort (was: Introduce retry
to logroller when encounters IOException)
> Introduce retry to logroller to avoid abort
> ---
>
> Key: HBASE-26552
> URL: https://issues.apache.org/jira/browse/HBASE-26552
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> When calling RollController#rollWal in AbstractWALRoller, the regionserver
> may abort when encounters exception,
> {code:java}
> ...
> } catch (FailedLogCloseException | ConnectException e) {
> abort("Failed log close in log roller", e);
> } catch (IOException ex) {
> // Abort if we get here. We probably won't recover an IOE. HBASE-1132
> abort("IOE in log roller",
> ex instanceof RemoteException ? ((RemoteException)
> ex).unwrapRemoteException() : ex);
> } catch (Exception ex) {
> LOG.error("Log rolling failed", ex);
> abort("Log rolling failed", ex);
> } {code}
> I think we should support retry of rollWal here to avoid recovering the
> service by killing regionserver. The restart of regionserver is costly and
> very not friendly to the availability.
> I find that when creating new writer for the WAL in
> FanOutOneBlockAsyncDFSOutputHelper#createOutput, it supports retry to
> addBlock by setting this config "hbase.fs.async.create.retries". The idea of
> retry to roll WAL is similar to it, they both try best to make roll WAL
> succeed.
> But the initialization of new WAL writer also includes flushing the write
> buffer flush and waiting until it is completed by
> AsyncProtobufLogWriter#writeMagicAndWALHeader, which can also fail by some
> hardware reasons. The regionserver connected to the datanodes after addBlock,
> but that not means the magic and header can be flushed successfully.
> {code:java}
> protected long writeMagicAndWALHeader(byte[] magic, WALHeader header) throws
> IOException {
> return write(future -> {
> output.write(magic);
> try {
> header.writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> // should not happen
> throw new AssertionError(e);
> }
> addListener(output.flush(false), (len, error) -> {
> if (error != null) {
> future.completeExceptionally(error);
> } else {
> future.complete(len);
> }
> });
> });
> }{code}
> We have found that in our production clusters, there exists aborting of
> regionservers that caused by "IOE in log roller". And the practice in our
> clusters is that just one more retry of rollWal can make the WAL roll
> complete and continue serving.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26734) FanOutOneBlockAsyncDFSOutputHelper stuck when run against hadoop-3.3.1
[
https://issues.apache.org/jira/browse/HBASE-26734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490051#comment-17490051
]
Wei-Chiu Chuang commented on HBASE-26734:
-
I wrote that part of the code so I'll take a closer look. slf4j-log4j2 doesn't
seem related to this issue.
Is it reproducible? Could you run it at debug log level? Just like to confirm
it's not the older Hadoop artifacts that got slipped in accidentally.
> FanOutOneBlockAsyncDFSOutputHelper stuck when run against hadoop-3.3.1
> --
>
> Key: HBASE-26734
> URL: https://issues.apache.org/jira/browse/HBASE-26734
> Project: HBase
> Issue Type: Sub-task
> Environment: JDK: jdk1.8.0_221
> Hadoop: hadoop-3.3.1
> Hbase: hbase-2.3.1 / hbase-2.3.7
>Reporter: chen qing
>Priority: Major
> Attachments: hbase-root-master-master.log,
> hbase-root-regionserver-slave01.log
>
>
> I just had the same problem when i started the hbase cluster. HRegionServers
> were started and HMaster threw an exception.
> This is HMaster's log:
> {code:java}
> 2022-02-05 18:07:51,323 WARN [RS-EventLoopGroup-1-1]
> concurrent.DefaultPromise: An exception was thrown by
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete()
> java.lang.IllegalArgumentException: object is not an instance of declaring
> class
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:69)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:343)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:425)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:183)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:419)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:477)
> at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:472)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:570)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:549)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:615)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:653)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:529)
> at
>
