[GitHub] [hbase] bsglz opened a new pull request #2182: HBASE-24704 Make the Table Schema easier to view even there are multi…
bsglz opened a new pull request #2182: URL: https://github.com/apache/hbase/pull/2182 …ple families This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on pull request #2018: HBASE-24659 Calculate FIXED_OVERHEAD automatically
nyl3532016 commented on pull request #2018: URL: https://github.com/apache/hbase/pull/2018#issuecomment-667476083 > Let's run a YCSB or at least a PE to see the performance impact? @Apache9 sir,I have done the performance test, and attaches on jira page, please take a look ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2181: HBASE-24808 cleaner.CleanerChore: Can NOT create CleanerDelegate= Cla…
Apache-HBase commented on pull request #2181: URL: https://github.com/apache/hbase/pull/2181#issuecomment-667474946 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 8s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 59s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 15s | the patch passed | | +1 :green_heart: | checkstyle | 1m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 33m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2181/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2181 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 63d16f2ecdec 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 271a3cbf13 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2181/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24803) Unify hbase-shell ::Shell::Commands::Command#help behavior
[
https://issues.apache.org/jira/browse/HBASE-24803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169231#comment-17169231
]
Hudson commented on HBASE-24803:
Results for branch branch-2
[build #2764 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2764/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2764/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2742/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2757/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unify hbase-shell ::Shell::Commands::Command#help behavior
> --
>
> Key: HBASE-24803
> URL: https://issues.apache.org/jira/browse/HBASE-24803
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 2.3.0
>Reporter: Elliot Miller
>Assignee: Elliot Miller
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> This is a small patch to unify shell help:
> * Many of the commands use different indentation levels to style the help
> text.
> * A few commands printed help instead of returning it, which breaks our
> pattern.
> I also added a test to make sure the help text stays consistent after this
> patch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24404) Support flush a single column family of region
[
https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169230#comment-17169230
]
Hudson commented on HBASE-24404:
Results for branch branch-2
[build #2764 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2764/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2764/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2756/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2742/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2757/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Support flush a single column family of region
> --
>
> Key: HBASE-24404
> URL: https://issues.apache.org/jira/browse/HBASE-24404
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-1, 2.4.0
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> It is useful when we set many column families for a table, could avoid
> generate too many small files.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1592: HBASE-24244 Move Region Command fails to move region of a table to another RS which is a part of different RSGroup without giving error.
Apache-HBase commented on pull request #1592: URL: https://github.com/apache/hbase/pull/1592#issuecomment-667465902 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | checkstyle | 1m 10s | master passed | | +1 :green_heart: | spotbugs | 2m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | -0 :warning: | checkstyle | 1m 8s | hbase-server: The patch generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 15s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 33m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1592/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1592 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux f61242355b1a 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1592/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1592/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2177: HBASE-24799 Do not call make_binary_release for hbase-thirdparty in r…
Apache-HBase commented on pull request #2177: URL: https://github.com/apache/hbase/pull/2177#issuecomment-667465523 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 18s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2177 | | Optional Tests | | | uname | Linux c3085fdead48 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Max. process+thread count | 52 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2177: HBASE-24799 Do not call make_binary_release for hbase-thirdparty in r…
Apache-HBase commented on pull request #2177: URL: https://github.com/apache/hbase/pull/2177#issuecomment-667465469 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 1m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2177 | | Optional Tests | | | uname | Linux 432fb5058ac4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Max. process+thread count | 45 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2177: HBASE-24799 Do not call make_binary_release for hbase-thirdparty in r…
Apache-HBase commented on pull request #2177: URL: https://github.com/apache/hbase/pull/2177#issuecomment-667465241 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 51s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 3s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2177 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 660c3c42fa29 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Max. process+thread count | 52 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2177/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
Apache-HBase commented on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667463449 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 5s | master passed | | +1 :green_heart: | compile | 1m 27s | master passed | | +1 :green_heart: | shadedjars | 8m 0s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 52s | hbase-server in master failed. | | -0 :warning: | patch | 9m 2s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 8m 2s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 47s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 33s | hbase-server in the patch passed. | | | | 182m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2172 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d89b95a5e414 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/testReport/ | | Max. process+thread count | 4190 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2042: HBASE-24704 Make the Table Schema easier to view even there are multi…
bsglz commented on pull request #2042: URL: https://github.com/apache/hbase/pull/2042#issuecomment-667462640 rebase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
Apache-HBase commented on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667462543 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | shadedjars | 5m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | -0 :warning: | patch | 6m 20s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | shadedjars | 5m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 13s | hbase-server in the patch passed. | | | | 172m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2172 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b062846db5cc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/testReport/ | | Max. process+thread count | 4576 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a change in pull request #2042: HBASE-24704 Make the Table Schema easier to view even there are multi…
bsglz commented on a change in pull request #2042: URL: https://github.com/apache/hbase/pull/2042#discussion_r463915997 ## File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp ## @@ -765,40 +767,45 @@ %> Table Schema + +<% + Collection families = new HTableDescriptor(table.getDescriptor()).getFamilies(); Review comment: Fixed, remove the unnecessary new HTableDescriptor and use ColumnFamilyDescriptor instead of HColumnDescriptor. Thanks. ## File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp ## @@ -765,40 +767,45 @@ %> Table Schema + +<% + Collection families = new HTableDescriptor(table.getDescriptor()).getFamilies(); + Set familyKeySet = new HashSet(); Review comment: Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
Apache-HBase commented on pull request #2180: URL: https://github.com/apache/hbase/pull/2180#issuecomment-667460579 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 41s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 5m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 5m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 25s | hbase-common in the patch passed. | | -1 :x: | unit | 157m 16s | hbase-server in the patch failed. | | | | 185m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2180 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d242fb1ceff8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/testReport/ | | Max. process+thread count | 4418 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2133: Cherry pick HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2133: URL: https://github.com/apache/hbase/pull/2133#issuecomment-667458934 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 0s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 10s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 6m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | branch-2.3 passed | | -0 :warning: | patch | 7m 44s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 7m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 156m 50s | hbase-server in the patch passed. | | | | 187m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2133 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4b87590de35f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 696dff4e9e | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/testReport/ | | Max. process+thread count | 3810 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2133: Cherry pick HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2133: URL: https://github.com/apache/hbase/pull/2133#issuecomment-667456980 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 35s | branch-2.3 passed | | +1 :green_heart: | compile | 1m 26s | branch-2.3 passed | | +1 :green_heart: | shadedjars | 8m 43s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 52s | hbase-server in branch-2.3 failed. | | -0 :warning: | patch | 9m 46s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 7m 56s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 48s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 135m 23s | hbase-server in the patch passed. | | | | 170m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2133 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 674d100e2015 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 696dff4e9e | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/testReport/ | | Max. process+thread count | 4201 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on a change in pull request #2181: HBASE-24808 cleaner.CleanerChore: Can NOT create CleanerDelegate= Cla…
anoopsjohn commented on a change in pull request #2181:
URL: https://github.com/apache/hbase/pull/2181#discussion_r463908201
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
##
@@ -151,7 +151,11 @@ private void initCleanerChain(String confKey) {
this.cleanersChain = new LinkedList<>();
String[] logCleaners = conf.getStrings(confKey);
Review comment:
nit: We can even rename this and related var names also.. This is not
just logCleaner.. Hfile cleaner also.. may be call just cleaner or fileCleaner.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] anoopsjohn commented on pull request #2125: HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
anoopsjohn commented on pull request #2125: URL: https://github.com/apache/hbase/pull/2125#issuecomment-667452832 >Hasn't the NPE been fixed by: HBASE-24034 [Flakey Tests] A couple of fixes and cleanups In FSHLog.java, I see the FSHLog.java:582 mentioned in a comment. No.. Here the NPE comes as the writer is null. As part of the initial writer create (that happens via rollWriter call), we ended up attaining a safe point and so publish a sync. This in turn try to call writer.sync() and so the NPE. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2125: HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2125: URL: https://github.com/apache/hbase/pull/2125#issuecomment-667450130 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 5m 43s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 12s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 5m 49s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 150m 30s | hbase-server in the patch failed. | | | | 176m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2125 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 082941c710d1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/testReport/ | | Max. process+thread count | 4114 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2125: HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2125: URL: https://github.com/apache/hbase/pull/2125#issuecomment-667448966 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 29s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 6m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 152m 36s | hbase-server in the patch failed. | | | | 180m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2125 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e3ff6843dcca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/testReport/ | | Max. process+thread count | 4031 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
Apache-HBase commented on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667445602 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 25s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 1m 57s | master passed | | -0 :warning: | patch | 2m 6s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 5 new + 12 unchanged - 2 fixed = 17 total (was 14) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 8s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 32m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2172 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 2d279defa66d 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2172/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2179: HBASE-24694 Support flush a single column family of table
Apache-HBase commented on pull request #2179: URL: https://github.com/apache/hbase/pull/2179#issuecomment-667440459 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | compile | 2m 57s | master passed | | +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 28s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 18s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | | -0 :warning: | javadoc | 0m 51s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 57s | the patch passed | | +1 :green_heart: | compile | 2m 57s | the patch passed | | +1 :green_heart: | javac | 2m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 26s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 51s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 137m 28s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 58s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 5s | hbase-shell in the patch passed. | | | | 187m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2179 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ba3e0495dad1 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/testReport/ | | Max. process+thread count | 4470 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | |
[GitHub] [hbase] Apache-HBase commented on pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
Apache-HBase commented on pull request #2180: URL: https://github.com/apache/hbase/pull/2180#issuecomment-667440310 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 36s | master passed | | +1 :green_heart: | spotbugs | 2m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 185 unchanged - 0 fixed = 186 total (was 185) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 39m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2180 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 966f99a63380 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2180/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2179: HBASE-24694 Support flush a single column family of table
Apache-HBase commented on pull request #2179: URL: https://github.com/apache/hbase/pull/2179#issuecomment-667440259 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | master passed | | +1 :green_heart: | compile | 2m 40s | master passed | | +1 :green_heart: | shadedjars | 5m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 2m 37s | the patch passed | | +1 :green_heart: | javac | 2m 37s | the patch passed | | +1 :green_heart: | shadedjars | 5m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 4s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 140m 54s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 5m 18s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 10s | hbase-shell in the patch passed. | | | | 187m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2179 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dce63f904741 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/testReport/ | | Max. process+thread count | 4414 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24794) hbase.rowlock.wait.duration should not be <= 0
[
https://issues.apache.org/jira/browse/HBASE-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169205#comment-17169205
]
Hudson commented on HBASE-24794:
Results for branch branch-2.2
[build #923 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/923/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/923//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/923//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/923//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> hbase.rowlock.wait.duration should not be <= 0
> --
>
> Key: HBASE-24794
> URL: https://issues.apache.org/jira/browse/HBASE-24794
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.0, 1.5.0, 2.2.0, 2.3.0, 1.6.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 1.4.14, 2.2.6
>
>
> had a cluster fail after upgrade from hbase 1 because all writes to meta
> failed.
> master started in maintenance mode looks like (RS hosting meta in non-maint
> would look similar starting with {{HRegion.doBatchMutate}}):
> {code}
> 2020-07-28 17:52:56,553 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed getting lock, row=some_user_table
> java.io.IOException: Timed out waiting for lock for row: some_user_table in
> region 1588230740
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lockRowsAndBuildMiniBatch(HRegion.java:3322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3992)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3923)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3914)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4255)
> at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3047)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2827)
> at
> org.apache.hadoop.hbase.client.ClientServiceCallable.doMutate(ClientServiceCallable.java:55)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:538)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:533)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:127)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:542)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1339)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1329)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1672)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1112)
> at
> org.apache.hadoop.hbase.master.TableStateManager.fixTableStates(TableStateManager.java:296)
> at
> org.apache.hadoop.hbase.master.TableStateManager.start(TableStateManager.java:269)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1004)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2274)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> logging roughly 6k times /second.
> failure was caused by a change in behavior for
> {{hbase.rowlock.wait.duration}} in HBASE-17210 (so 1.4.0+, 2.0.0+). Prior to
> that change setting the config <= 0 meant that row locks would succeed only
> if they were immediately available. After the change we fail the lock attempt
> without checking the lock at all.
>
[GitHub] [hbase] Apache-HBase commented on pull request #2133: Cherry pick HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2133: URL: https://github.com/apache/hbase/pull/2133#issuecomment-667431033 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | branch-2.3 passed | | +1 :green_heart: | checkstyle | 1m 9s | branch-2.3 passed | | +1 :green_heart: | spotbugs | 2m 0s | branch-2.3 passed | | -0 :warning: | patch | 2m 9s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 19s | the patch passed | | +1 :green_heart: | checkstyle | 1m 24s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 29s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 47m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2133 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 9b9d59fc3a6b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 696dff4e9e | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2133/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2125: HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
Apache-HBase commented on pull request #2125: URL: https://github.com/apache/hbase/pull/2125#issuecomment-667418265 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 41s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 2m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 32m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2125 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 7638cb2ce033 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2125/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] tedyu commented on pull request #2125: HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751
tedyu commented on pull request #2125: URL: https://github.com/apache/hbase/pull/2125#issuecomment-667417667 Hasn't the NPE been fixed by: HBASE-24034 [Flakey Tests] A couple of fixes and cleanups In FSHLog.java, I see the FSHLog.java:582 mentioned in a comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2179: HBASE-24694 Support flush a single column family of table
Apache-HBase commented on pull request #2179: URL: https://github.com/apache/hbase/pull/2179#issuecomment-667410483 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | checkstyle | 2m 44s | master passed | | +1 :green_heart: | spotbugs | 4m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | -0 :warning: | checkstyle | 1m 5s | hbase-server: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) | | -0 :warning: | rubocop | 0m 9s | The patch generated 2 new + 339 unchanged - 1 fixed = 341 total (was 340) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 1s | The patch does not generate ASF License warnings. | | | | 46m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.12 Server=19.03.12 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2179 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle rubocop | | uname | Linux f3c49858e50b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7c4d66a277 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server hbase-thrift hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2179/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack opened a new pull request #2181: HBASE-24808 cleaner.CleanerChore: Can NOT create CleanerDelegate= Cla…
saintstack opened a new pull request #2181: URL: https://github.com/apache/hbase/pull/2181 …ssNotFoundException This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-24808) cleaner.CleanerChore: Can NOT create CleanerDelegate= ClassNotFoundException
Michael Stack created HBASE-24808: - Summary: cleaner.CleanerChore: Can NOT create CleanerDelegate= ClassNotFoundException Key: HBASE-24808 URL: https://issues.apache.org/jira/browse/HBASE-24808 Project: HBase Issue Type: Bug Reporter: Michael Stack 2020-07-31 00:19:49,839 WARN [master/ps0753:16000:becomeActiveMaster] cleaner.CleanerChore: Can NOT create CleanerDelegate= java.lang.ClassNotFoundException: at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.newFileCleaner(CleanerChore.java:173) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.initCleanerChain(CleanerChore.java:155) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.(CleanerChore.java:105) at org.apache.hadoop.hbase.master.cleaner.HFileCleaner.(HFileCleaner.java:139) at org.apache.hadoop.hbase.master.cleaner.HFileCleaner.(HFileCleaner.java:120) at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1424) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1025) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2189) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:609) at java.lang.Thread.run(Thread.java:748) This is the config: hbase.master.hfilecleaner.plugins -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24802) Please fix CVEs by removing reference to htrace-core4
[ https://issues.apache.org/jira/browse/HBASE-24802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169162#comment-17169162 ] Michael Stack commented on HBASE-24802: --- What you need? A release w/o 'Jackson databind version 2.4.0 ' ? > Please fix CVEs by removing reference to htrace-core4 > - > > Key: HBASE-24802 > URL: https://issues.apache.org/jira/browse/HBASE-24802 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.3.0 >Reporter: Rodney Aaron Stainback >Priority: Major > > htrace-core4 is a retired project and even on the latest version they Shade > Jackson databind version 2.4.0 which has the following CVEs: > |cve|severity|cvss| > |CVE-2017-15095|critical|9.8| > |CVE-2018-1000873|medium|6.5| > |CVE-2018-14718|critical|9.8| > |CVE-2018-5968|high|8.1| > |CVE-2018-7489|critical|9.8| > |CVE-2019-14540|critical|9.8| > |CVE-2019-14893|critical|9.8| > |CVE-2019-16335|critical|9.8| > |CVE-2019-16942|critical|9.8| > |CVE-2019-16943|critical|9.8| > |CVE-2019-17267|critical|9.8| > |CVE-2019-17531|critical|9.8| > |CVE-2019-20330|critical|9.8| > |CVE-2020-10672|high|8.8| > |CVE-2020-10673|high|8.8| > |CVE-2020-10968|high|8.8| > |CVE-2020-10969|high|8.8| > |CVE-2020-1|high|8.8| > |CVE-2020-2|high|8.8| > |CVE-2020-3|high|8.8| > |CVE-2020-11619|critical|9.8| > |CVE-2020-11620|critical|9.8| > |CVE-2020-14060|high|8.1| > |CVE-2020-14061|high|8.1| > |CVE-2020-14062|high|8.1| > |CVE-2020-14195|high|8.1| > |CVE-2020-8840|critical|9.8| > |CVE-2020-9546|critical|9.8| > |CVE-2020-9547|critical|9.8| > |CVE-2020-9548|critical|9.8| > > Our security team is trying to block us from using hbase because of this -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24794) hbase.rowlock.wait.duration should not be <= 0
[
https://issues.apache.org/jira/browse/HBASE-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169129#comment-17169129
]
Hudson commented on HBASE-24794:
Results for branch branch-1.4
[build #1231 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1231/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1231//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1231//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1231//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> hbase.rowlock.wait.duration should not be <= 0
> --
>
> Key: HBASE-24794
> URL: https://issues.apache.org/jira/browse/HBASE-24794
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.0, 1.5.0, 2.2.0, 2.3.0, 1.6.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 1.4.14, 2.2.6
>
>
> had a cluster fail after upgrade from hbase 1 because all writes to meta
> failed.
> master started in maintenance mode looks like (RS hosting meta in non-maint
> would look similar starting with {{HRegion.doBatchMutate}}):
> {code}
> 2020-07-28 17:52:56,553 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed getting lock, row=some_user_table
> java.io.IOException: Timed out waiting for lock for row: some_user_table in
> region 1588230740
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lockRowsAndBuildMiniBatch(HRegion.java:3322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3992)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3923)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3914)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4255)
> at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3047)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2827)
> at
> org.apache.hadoop.hbase.client.ClientServiceCallable.doMutate(ClientServiceCallable.java:55)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:538)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:533)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:127)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:542)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1339)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1329)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1672)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1112)
> at
> org.apache.hadoop.hbase.master.TableStateManager.fixTableStates(TableStateManager.java:296)
> at
> org.apache.hadoop.hbase.master.TableStateManager.start(TableStateManager.java:269)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1004)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2274)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> logging roughly 6k times /second.
> failure was caused by a change in behavior for
> {{hbase.rowlock.wait.duration}} in HBASE-17210 (so 1.4.0+, 2.0.0+). Prior to
> that change setting the config <= 0 meant that row locks would succeed only
> if they were immediately available. After the change we fail the lock attempt
> without checking the lock at all.
> workaround: set {{hbase.rowlock.wait.duration}} to a small positive number,
> e.g. 1,
[GitHub] [hbase] virajjasani commented on a change in pull request #2042: HBASE-24704 Make the Table Schema easier to view even there are multi…
virajjasani commented on a change in pull request #2042: URL: https://github.com/apache/hbase/pull/2042#discussion_r463768549 ## File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp ## @@ -765,40 +767,45 @@ %> Table Schema + +<% + Collection families = new HTableDescriptor(table.getDescriptor()).getFamilies(); + Set familyKeySet = new HashSet(); Review comment: nit: `HashSet<>()` ## File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp ## @@ -765,40 +767,45 @@ %> Table Schema + +<% + Collection families = new HTableDescriptor(table.getDescriptor()).getFamilies(); Review comment: Please use `ModifyableTableDescriptor` as `HTableDescriptor` is deprecated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] arshadmohammad commented on a change in pull request #1592: HBASE-24244 Move Region Command fails to move region of a table to another RS which is a part of different RSGroup without
arshadmohammad commented on a change in pull request #1592:
URL: https://github.com/apache/hbase/pull/1592#discussion_r463734520
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -2085,6 +2085,10 @@ public void move(final byte[] encodedRegionName, byte[]
destServerName) throws I
return;
}
}
+if (dest.equals(LoadBalancer.BOGUS_SERVER_NAME)) {
Review comment:
Added comment in code.
Force pushed to correct author name in previous commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #2145: HBASE-24476 release scripts should provide timing information
ndimiduk commented on pull request #2145: URL: https://github.com/apache/hbase/pull/2145#issuecomment-667237096 Thanks @busbey , this will be helpful. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24804) Follow up work to add client side scan metrics for read replica
[ https://issues.apache.org/jira/browse/HBASE-24804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-24804: - Summary: Follow up work to add client side scan metrics for read replica (was: Follow up work add client side scan metrics for read replica) > Follow up work to add client side scan metrics for read replica > --- > > Key: HBASE-24804 > URL: https://issues.apache.org/jira/browse/HBASE-24804 > Project: HBase > Issue Type: New Feature > Components: read replicas >Affects Versions: 2.4.0 >Reporter: Huaxiang Sun >Assignee: Huaxiang Sun >Priority: Minor > > This is a followup work for HBASE-18436, which adds client metrics for read > replica get. Will add metrics for scan as well. This metrics will be used in > PE and any interested applications. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-15916) incorrect work formating for shell scan
[ https://issues.apache.org/jira/browse/HBASE-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot Miller updated HBASE-15916: -- Description: this commit change behavior of 'scan' command https://github.com/apache/hbase/commit/e1e8434340f02907976f20566c3e55d8d627d4c4 old behavior: 1. call table._scan_internal with args 2. table._scan_internal 2.1. call @converters.clear() 2.2. create scan and fill @converters 2.3. return result 3. scan print result with formatters new behavior issue: 1. scan command prepare Scan and fill @converters 2. table._scan_internal 2.1. call @converters.clear() 2.2. if we have scan != nil use his 2.3. return result 3. scan print result WITHOUT formatters p.s. this example dosn't work [http://blog.cloudera.com/blog/2016/01/how-to-create-and-use-a-custom-formatter-in-the-apache-hbase-shell/|https://web.archive.org/web/20160326074917/http://blog.cloudera.com/blog/2016/01/how-to-create-and-use-a-custom-formatter-in-the-apache-hbase-shell/] was: this commit change behavior of 'scan' command https://github.com/apache/hbase/commit/e1e8434340f02907976f20566c3e55d8d627d4c4 old behavior: 1. call table._scan_internal with args 2. table._scan_internal 2.1. call @converters.clear() 2.2. create scan and fill @converters 2.3. return result 3. scan print result with formatters new behavior issue: 1. scan command prepare Scan and fill @converters 2. table._scan_internal 2.1. call @converters.clear() 2.2. if we have scan != nil use his 2.3. return result 3. scan print result WITHOUT formatters p.s. this example dosn't work http://blog.cloudera.com/blog/2016/01/how-to-create-and-use-a-custom-formatter-in-the-apache-hbase-shell/ > incorrect work formating for shell scan > > > Key: HBASE-15916 > URL: https://issues.apache.org/jira/browse/HBASE-15916 > Project: HBase > Issue Type: Bug >Reporter: Alexey Diomin >Priority: Minor > Attachments: > 0001-HBASE-15916-incorrect-work-formating-for-shell-scan.patch > > > this commit change behavior of 'scan' command > https://github.com/apache/hbase/commit/e1e8434340f02907976f20566c3e55d8d627d4c4 > old behavior: > 1. call table._scan_internal with args > 2. table._scan_internal > 2.1. call @converters.clear() > 2.2. create scan and fill @converters > 2.3. return result > 3. scan print result with formatters > new behavior issue: > 1. scan command prepare Scan and fill @converters > 2. table._scan_internal > 2.1. call @converters.clear() > 2.2. if we have scan != nil use his > 2.3. return result > 3. scan print result WITHOUT formatters > p.s. this example dosn't work > [http://blog.cloudera.com/blog/2016/01/how-to-create-and-use-a-custom-formatter-in-the-apache-hbase-shell/|https://web.archive.org/web/20160326074917/http://blog.cloudera.com/blog/2016/01/how-to-create-and-use-a-custom-formatter-in-the-apache-hbase-shell/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24794) hbase.rowlock.wait.duration should not be <= 0
[
https://issues.apache.org/jira/browse/HBASE-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168928#comment-17168928
]
Hudson commented on HBASE-24794:
Results for branch master
[build #1799 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/master/1799//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> hbase.rowlock.wait.duration should not be <= 0
> --
>
> Key: HBASE-24794
> URL: https://issues.apache.org/jira/browse/HBASE-24794
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.0, 1.5.0, 2.2.0, 2.3.0, 1.6.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 1.4.14, 2.2.6
>
>
> had a cluster fail after upgrade from hbase 1 because all writes to meta
> failed.
> master started in maintenance mode looks like (RS hosting meta in non-maint
> would look similar starting with {{HRegion.doBatchMutate}}):
> {code}
> 2020-07-28 17:52:56,553 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed getting lock, row=some_user_table
> java.io.IOException: Timed out waiting for lock for row: some_user_table in
> region 1588230740
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lockRowsAndBuildMiniBatch(HRegion.java:3322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3992)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3923)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3914)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4255)
> at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3047)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2827)
> at
> org.apache.hadoop.hbase.client.ClientServiceCallable.doMutate(ClientServiceCallable.java:55)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:538)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:533)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:127)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:542)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1339)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1329)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1672)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1112)
> at
> org.apache.hadoop.hbase.master.TableStateManager.fixTableStates(TableStateManager.java:296)
> at
> org.apache.hadoop.hbase.master.TableStateManager.start(TableStateManager.java:269)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1004)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2274)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> logging roughly 6k times /second.
> failure was caused by a change in behavior for
> {{hbase.rowlock.wait.duration}} in
[GitHub] [hbase] virajjasani commented on a change in pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
virajjasani commented on a change in pull request #2172:
URL: https://github.com/apache/hbase/pull/2172#discussion_r463671614
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionMover1.java
##
@@ -80,23 +83,18 @@ public static void tearDownAfterClass() throws Exception {
TEST_UTIL.shutdownMiniCluster();
}
- @Before
- public void setUp() throws Exception {
-// Create a pre-split table just to populate some regions
-TableName tableName = TableName.valueOf("testRegionMover");
-Admin admin = TEST_UTIL.getAdmin();
-if (admin.tableExists(tableName)) {
- TEST_UTIL.deleteTable(tableName);
-}
+ private void initTableRegions(TableName tableName) throws IOException {
Review comment:
Oh, I wasn't aware of this. Sounds really great. I was also not happy to
keep repeating the same code for each method.
Thanks, will update the PR shortly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24797) Move log code out of loop
[
https://issues.apache.org/jira/browse/HBASE-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168927#comment-17168927
]
Hudson commented on HBASE-24797:
Results for branch master
[build #1799 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1799/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/master/1799//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Move log code out of loop
> -
>
> Key: HBASE-24797
> URL: https://issues.apache.org/jira/browse/HBASE-24797
> Project: HBase
> Issue Type: Bug
> Components: Normalizer
>Affects Versions: 3.0.0-alpha-1
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
>
> In HMaster#normalizeRegions, maybe we shoule move the log code about
> submittedPlanProcIds out of loop.
>
> {code:java}
> public boolean normalizeRegions() throws IOException {
> ...
> final List submittedPlanProcIds = new ArrayList<>();
> for (TableName table : allEnabledTables) {
> ...
> for (NormalizationPlan plan : plans) {
> long procId = plan.submit(this);
> submittedPlanProcIds.add(procId);
> ...
> }
> int totalPlansSubmitted = submittedPlanProcIds.size();
> if (totalPlansSubmitted > 0 && LOG.isDebugEnabled()) {
> LOG.debug("Normalizer plans submitted. Total plans count: {} , procID
> list: {}",
> totalPlansSubmitted, submittedPlanProcIds);
> }
> }
> ...
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on a change in pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
busbey commented on a change in pull request #2172:
URL: https://github.com/apache/hbase/pull/2172#discussion_r463661594
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionMover1.java
##
@@ -80,23 +83,18 @@ public static void tearDownAfterClass() throws Exception {
TEST_UTIL.shutdownMiniCluster();
}
- @Before
- public void setUp() throws Exception {
-// Create a pre-split table just to populate some regions
-TableName tableName = TableName.valueOf("testRegionMover");
-Admin admin = TEST_UTIL.getAdmin();
-if (admin.tableExists(tableName)) {
- TEST_UTIL.deleteTable(tableName);
-}
+ private void initTableRegions(TableName tableName) throws IOException {
Review comment:
the `name.getMethodName()` call is available in `Before` annotated
methods, so you should still do this in a shared setup method instead of call
this in each test.
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionMover2.java
##
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.MiniHBaseCluster;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegionServer;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TestName;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Tests for Region Mover Load/Unload functionality with and without ack mode
and also to test
+ * exclude functionality useful for rack decommissioning
+ */
+@Category({ MiscTests.class, LargeTests.class})
+public class TestRegionMover2 {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionMover2.class);
+
+ @Rule
+ public TestName name = new TestName();
+
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionMover2.class);
+
+ private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+TEST_UTIL.startMiniCluster(3);
+TEST_UTIL.getAdmin().balancerSwitch(false, true);
+ }
+
+ @AfterClass
+ public static void tearDownAfterClass() throws Exception {
+TEST_UTIL.shutdownMiniCluster();
+ }
+
+ private void initTableRegions(TableName tableName) throws IOException {
+TableDescriptor tableDesc = TableDescriptorBuilder.newBuilder(tableName)
+ .setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam1")).build();
+int startKey = 0;
+int endKey = 8;
+TEST_UTIL.getAdmin().createTable(tableDesc, Bytes.toBytes(startKey),
Bytes.toBytes(endKey), 9);
+ }
+
+ @Test
+ public void testWithMergedRegions() throws Exception {
+final TableName tableName = TableName.valueOf(name.getMethodName());
+initTableRegions(tableName);
+MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
+Admin admin = TEST_UTIL.getAdmin();
+Table table = TEST_UTIL.getConnection().getTable(tableName);
+List puts = new ArrayList<>();
+for (int i = 0; i < 1; i++) {
+ puts.add(new Put(Bytes.toBytes("rowkey_" + i))
+.addColumn(Bytes.toBytes("fam1"), Bytes.toBytes("q1"),
[GitHub] [hbase] busbey edited a comment on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
busbey edited a comment on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667169370 > Precommit run is not getting scheduled, waiting logs are present for long time it's backlog waiting for executors to open up on the new ci host: https://ci-hadoop.apache.org/label/Hadoop/load-statistics looks like the queue size started to climb yesterday around midnight CDT. peaked about 5 hours ago at ~300. is now sitting around 280. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
busbey commented on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667169370 > Precommit run is not getting scheduled, waiting logs are present for long time it's backlog waiting for executors to open up on the new ci host: https://ci-hadoop.apache.org/label/Hadoop/load-statistics looks like the queue size started to climb yesterday around midnight. peaked about 5 hours ago at ~300. is now sitting around 280. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2177: HBASE-24799 Do not call make_binary_release for hbase-thirdparty in r…
Apache9 commented on pull request #2177: URL: https://github.com/apache/hbase/pull/2177#issuecomment-667163464 OK, I finally got the release script work for hbase-thirdparty... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24797) Move log code out of loop
[
https://issues.apache.org/jira/browse/HBASE-24797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168910#comment-17168910
]
Hudson commented on HBASE-24797:
Results for branch branch-2.3
[build #196 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move log code out of loop
> -
>
> Key: HBASE-24797
> URL: https://issues.apache.org/jira/browse/HBASE-24797
> Project: HBase
> Issue Type: Bug
> Components: Normalizer
>Affects Versions: 3.0.0-alpha-1
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0
>
>
> In HMaster#normalizeRegions, maybe we shoule move the log code about
> submittedPlanProcIds out of loop.
>
> {code:java}
> public boolean normalizeRegions() throws IOException {
> ...
> final List submittedPlanProcIds = new ArrayList<>();
> for (TableName table : allEnabledTables) {
> ...
> for (NormalizationPlan plan : plans) {
> long procId = plan.submit(this);
> submittedPlanProcIds.add(procId);
> ...
> }
> int totalPlansSubmitted = submittedPlanProcIds.size();
> if (totalPlansSubmitted > 0 && LOG.isDebugEnabled()) {
> LOG.debug("Normalizer plans submitted. Total plans count: {} , procID
> list: {}",
> totalPlansSubmitted, submittedPlanProcIds);
> }
> }
> ...
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24794) hbase.rowlock.wait.duration should not be <= 0
[
https://issues.apache.org/jira/browse/HBASE-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168912#comment-17168912
]
Hudson commented on HBASE-24794:
Results for branch branch-2.3
[build #196 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> hbase.rowlock.wait.duration should not be <= 0
> --
>
> Key: HBASE-24794
> URL: https://issues.apache.org/jira/browse/HBASE-24794
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.0, 1.5.0, 2.2.0, 2.3.0, 1.6.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 1.4.14, 2.2.6
>
>
> had a cluster fail after upgrade from hbase 1 because all writes to meta
> failed.
> master started in maintenance mode looks like (RS hosting meta in non-maint
> would look similar starting with {{HRegion.doBatchMutate}}):
> {code}
> 2020-07-28 17:52:56,553 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed getting lock, row=some_user_table
> java.io.IOException: Timed out waiting for lock for row: some_user_table in
> region 1588230740
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lockRowsAndBuildMiniBatch(HRegion.java:3322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3992)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3923)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3914)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4255)
> at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3047)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2827)
> at
> org.apache.hadoop.hbase.client.ClientServiceCallable.doMutate(ClientServiceCallable.java:55)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:538)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:533)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:127)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:542)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1339)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1329)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1672)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1112)
> at
> org.apache.hadoop.hbase.master.TableStateManager.fixTableStates(TableStateManager.java:296)
> at
> org.apache.hadoop.hbase.master.TableStateManager.start(TableStateManager.java:269)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1004)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2274)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> logging roughly 6k times /second.
> failure was caused by a change in behavior for
> {{hbase.rowlock.wait.duration}} in
[jira] [Commented] (HBASE-24788) Fix the connection leaks on getting hbase admin from unclosed connection
[
https://issues.apache.org/jira/browse/HBASE-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168911#comment-17168911
]
Hudson commented on HBASE-24788:
Results for branch branch-2.3
[build #196 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.3/196/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the connection leaks on getting hbase admin from unclosed connection
>
>
> Key: HBASE-24788
> URL: https://issues.apache.org/jira/browse/HBASE-24788
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.6.0
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> Observed the significant increase in ZK connection on performance testing on
> map reduce jobs. Turns out the
> [TableOutputFormat.checkOutputSpecs()|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182]
> is not closing the connection it uses to get the hbase admin. It closes the
> hbase admin but never close the connection to get the admin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24779) Improve insight into replication WAL readers hung on checkQuota
[
https://issues.apache.org/jira/browse/HBASE-24779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168909#comment-17168909
]
Wellington Chevreuil commented on HBASE-24779:
--
Had opened HBASE-24807 for the branch-1 backport.
> Improve insight into replication WAL readers hung on checkQuota
> ---
>
> Key: HBASE-24779
> URL: https://issues.apache.org/jira/browse/HBASE-24779
> Project: HBase
> Issue Type: Task
> Components: Replication
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
>
> Helped a customer this past weekend who, with a large number of
> RegionServers, has some RegionServers which replicated data to a peer without
> issues while other RegionServers did not.
> The number of queue logs varied over the past 24hrs in the same manner. Some
> spikes in queued logs into 100's of logs, but other times, only 1's-10's of
> logs were queued.
> We were able to validate that there were "good" and "bad" RegionServers by
> creating a test table, assigning it to a regionserver, enabling replication
> on that table, and validating if the local puts were replicated to a peer. On
> a good RS, data was replicated immediately. On a bad RS, data was never
> replicated (at least, on the order of 10's of minutes which we waited).
> On the "bad RS", we were able to observe that the \{{wal-reader}} thread(s)
> on that RS were spending time in a Thread.sleep() in a different location
> than the other. Specifically it was sitting in the
> {{ReplicationSourceWALReader#checkQuota()}}'s sleep call, _not_ the
> {{handleEmptyWALBatch()}} method on the same class.
> My only assumption is that, somehow, these RegionServers got into a situation
> where they "allocated" memory from the quota but never freed it. Then,
> because the WAL reader thinks it has no free memory, it blocks indefinitely
> and there are no pending edits to ship and (thus) free that memory. A cursory
> glance at the code gives me a _lot_ of anxiety around places where we don't
> properly clean it up (e.g. batches that fail to ship, dropping a peer). As a
> first stab, let me add some more debugging so we can actually track this
> state properly for the operators and their sanity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24807) Backport HBASE-20417 to branch-1
Wellington Chevreuil created HBASE-24807: Summary: Backport HBASE-20417 to branch-1 Key: HBASE-24807 URL: https://issues.apache.org/jira/browse/HBASE-24807 Project: HBase Issue Type: Bug Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil The wal reader shouldn't keep running with peer disabled. We need to backport HBASE-20417 to branch-1 or do something similar, if backport isn't possible due to differences in the code base. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24803) Unify hbase-shell ::Shell::Commands::Command#help behavior
[ https://issues.apache.org/jira/browse/HBASE-24803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24803. --- Fix Version/s: (was: 2.3.1) 2.4.0 Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-2+. Thanks for nice cleanup [~bitoffdev] > Unify hbase-shell ::Shell::Commands::Command#help behavior > -- > > Key: HBASE-24803 > URL: https://issues.apache.org/jira/browse/HBASE-24803 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.3.0 >Reporter: Elliot Miller >Assignee: Elliot Miller >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0 > > > This is a small patch to unify shell help: > * Many of the commands use different indentation levels to style the help > text. > * A few commands printed help instead of returning it, which breaks our > pattern. > I also added a test to make sure the help text stays consistent after this > patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2178: HBASE-24803 Unify hbase-shell ::Shell::Commands::Command#help behavior
saintstack merged pull request #2178: URL: https://github.com/apache/hbase/pull/2178 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on a change in pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
busbey commented on a change in pull request #2180: URL: https://github.com/apache/hbase/pull/2180#discussion_r463650086 ## File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java ## @@ -69,7 +69,7 @@ Compression.Algorithm.NONE, Compression.Algorithm.GZ }; - protected Configuration conf; + protected final Configuration conf; Review comment: s/hbase client code/code that uses hbase client This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on a change in pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
busbey commented on a change in pull request #2180: URL: https://github.com/apache/hbase/pull/2180#discussion_r463649812 ## File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java ## @@ -69,7 +69,7 @@ Compression.Algorithm.NONE, Compression.Algorithm.GZ }; - protected Configuration conf; + protected final Configuration conf; Review comment: > should it be changed to IA Private here? no, there 's a whole back and forth community discussion that has been on-going in bits and pieces for years. the short version is that we definitely need something downstream facing for running tests of hbase client code, but no one has provided a clean way to separate that out from how we test our own stuff. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24779) Improve insight into replication WAL readers hung on checkQuota
[
https://issues.apache.org/jira/browse/HBASE-24779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168890#comment-17168890
]
Josh Elser commented on HBASE-24779:
{quote}Had done further investigations and found HBASE-20417, which has fixed
the problem of wal reader running even with peer disabled for 2.1.0 onwards.
{quote}
Great! Thanks for searching and finding that. I didn't look hard enough, it
seems :)
{quote}So we should backport HBASE-20417 to branch-1 as well.
{quote}
+1
> Improve insight into replication WAL readers hung on checkQuota
> ---
>
> Key: HBASE-24779
> URL: https://issues.apache.org/jira/browse/HBASE-24779
> Project: HBase
> Issue Type: Task
> Components: Replication
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
>
> Helped a customer this past weekend who, with a large number of
> RegionServers, has some RegionServers which replicated data to a peer without
> issues while other RegionServers did not.
> The number of queue logs varied over the past 24hrs in the same manner. Some
> spikes in queued logs into 100's of logs, but other times, only 1's-10's of
> logs were queued.
> We were able to validate that there were "good" and "bad" RegionServers by
> creating a test table, assigning it to a regionserver, enabling replication
> on that table, and validating if the local puts were replicated to a peer. On
> a good RS, data was replicated immediately. On a bad RS, data was never
> replicated (at least, on the order of 10's of minutes which we waited).
> On the "bad RS", we were able to observe that the \{{wal-reader}} thread(s)
> on that RS were spending time in a Thread.sleep() in a different location
> than the other. Specifically it was sitting in the
> {{ReplicationSourceWALReader#checkQuota()}}'s sleep call, _not_ the
> {{handleEmptyWALBatch()}} method on the same class.
> My only assumption is that, somehow, these RegionServers got into a situation
> where they "allocated" memory from the quota but never freed it. Then,
> because the WAL reader thinks it has no free memory, it blocks indefinitely
> and there are no pending edits to ship and (thus) free that memory. A cursory
> glance at the code gives me a _lot_ of anxiety around places where we don't
> properly clean it up (e.g. batches that fail to ship, dropping a peer). As a
> first stab, let me add some more debugging so we can actually track this
> state properly for the operators and their sanity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-11686) Shell code should create a binding / irb workspace instead of polluting the root namespace
[
https://issues.apache.org/jira/browse/HBASE-11686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elliot Miller resolved HBASE-11686.
---
Resolution: Fixed
Resolving this. I created a sub-task to make a few small changes to the first
patch.
> Shell code should create a binding / irb workspace instead of polluting the
> root namespace
> --
>
> Key: HBASE-11686
> URL: https://issues.apache.org/jira/browse/HBASE-11686
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Sean Busbey
>Assignee: Elliot Miller
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
>
> Right now, the shell builds a list of commands and then injects them into the
> root exectution's context
> bin/hirb.rb
> {code}
> # Add commands to this namespace
> @shell.export_commands(self)
> {code}
> hbase-shell/src/main/ruby/shell.rb
> {code}
> def export_commands(where)
> ::Shell.commands.keys.each do |cmd|
> # here where is the IRB namespace
> # this method just adds the call to the specified command
> # which just references back to 'this' shell object
> # a decently extensible way to add commands
> where.send :instance_eval, <<-EOF
> def #{cmd}(*args)
> ret = @shell.command('#{cmd}', *args)
> puts
> return ret
> end
> EOF
> end
> end
> {code}
> This is an unclean abstraction. For one, it requires that there be an
> instance variable in the main namespace called '@shell' without making that
> clear in the docs. Additionally, it complicates maintenance by breaking
> isolation.
> We should update things so that shell can provide a binding for eval or a
> workspace for IRB execution and then use it directly when we construct our
> IRB session.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24806) Small Updates to Functionality of Shell IRB Workspace
Elliot Miller created HBASE-24806: - Summary: Small Updates to Functionality of Shell IRB Workspace Key: HBASE-24806 URL: https://issues.apache.org/jira/browse/HBASE-24806 Project: HBase Issue Type: Sub-task Components: shell Affects Versions: 3.0.0-alpha-1 Reporter: Elliot Miller Assignee: Elliot Miller Fix For: 3.0.0-alpha-1 There are a few amendments I want to make to the first patch for shell IRB workspaces: # Hide the new warning: "irb: warn: can't alias help from irb_help." # Split Shell::Shell#eval_io into eval_io and exception_handler. This will be a better separation of concerns for both usage and testing. ## Why is this change so important? At the moment, eval_io may raise SystemExit, which would cause the ruby test executor to quit without running all tests. The method eval_io also used to refer to a global variable $fullTraceback, which is a poor separation of concerns. # Allow finding script2run in the load path. While undocumented, the 2.x shell did this, so we may need to do this for compatibility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] anoopsjohn commented on pull request #2157: HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
anoopsjohn commented on pull request #2157: URL: https://github.com/apache/hbase/pull/2157#issuecomment-667111598 Seems no luck. Can u pls close this and raise a new PR for branch-2.2? Same for branch-1 also This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-24779) Improve insight into replication WAL readers hung on checkQuota
[
https://issues.apache.org/jira/browse/HBASE-24779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168660#comment-17168660
]
Wellington Chevreuil edited comment on HBASE-24779 at 7/31/20, 11:51 AM:
-
Had done further investigations and found HBASE-20417, which has fixed the
problem of wal reader running even with peer disabled for 2.1.0 onwards.
So we should backport HBASE-20417 to branch-1 as well.
was (Author: wchevreuil):
Had done further investigations and found HBASE-20417, which has fixed the
problem of wal reader running even with peer disabled for 2.1.0 onwards.
> Improve insight into replication WAL readers hung on checkQuota
> ---
>
> Key: HBASE-24779
> URL: https://issues.apache.org/jira/browse/HBASE-24779
> Project: HBase
> Issue Type: Task
> Components: Replication
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
>
> Helped a customer this past weekend who, with a large number of
> RegionServers, has some RegionServers which replicated data to a peer without
> issues while other RegionServers did not.
> The number of queue logs varied over the past 24hrs in the same manner. Some
> spikes in queued logs into 100's of logs, but other times, only 1's-10's of
> logs were queued.
> We were able to validate that there were "good" and "bad" RegionServers by
> creating a test table, assigning it to a regionserver, enabling replication
> on that table, and validating if the local puts were replicated to a peer. On
> a good RS, data was replicated immediately. On a bad RS, data was never
> replicated (at least, on the order of 10's of minutes which we waited).
> On the "bad RS", we were able to observe that the \{{wal-reader}} thread(s)
> on that RS were spending time in a Thread.sleep() in a different location
> than the other. Specifically it was sitting in the
> {{ReplicationSourceWALReader#checkQuota()}}'s sleep call, _not_ the
> {{handleEmptyWALBatch()}} method on the same class.
> My only assumption is that, somehow, these RegionServers got into a situation
> where they "allocated" memory from the quota but never freed it. Then,
> because the WAL reader thinks it has no free memory, it blocks indefinitely
> and there are no pending edits to ship and (thus) free that memory. A cursory
> glance at the code gives me a _lot_ of anxiety around places where we don't
> properly clean it up (e.g. batches that fail to ship, dropping a peer). As a
> first stab, let me add some more debugging so we can actually track this
> state properly for the operators and their sanity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24779) Improve insight into replication WAL readers hung on checkQuota
[
https://issues.apache.org/jira/browse/HBASE-24779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168660#comment-17168660
]
Wellington Chevreuil commented on HBASE-24779:
--
Had done further investigations and found HBASE-20417, which has fixed the
problem of wal reader running even with peer disabled for 2.1.0 onwards.
> Improve insight into replication WAL readers hung on checkQuota
> ---
>
> Key: HBASE-24779
> URL: https://issues.apache.org/jira/browse/HBASE-24779
> Project: HBase
> Issue Type: Task
> Components: Replication
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
>
> Helped a customer this past weekend who, with a large number of
> RegionServers, has some RegionServers which replicated data to a peer without
> issues while other RegionServers did not.
> The number of queue logs varied over the past 24hrs in the same manner. Some
> spikes in queued logs into 100's of logs, but other times, only 1's-10's of
> logs were queued.
> We were able to validate that there were "good" and "bad" RegionServers by
> creating a test table, assigning it to a regionserver, enabling replication
> on that table, and validating if the local puts were replicated to a peer. On
> a good RS, data was replicated immediately. On a bad RS, data was never
> replicated (at least, on the order of 10's of minutes which we waited).
> On the "bad RS", we were able to observe that the \{{wal-reader}} thread(s)
> on that RS were spending time in a Thread.sleep() in a different location
> than the other. Specifically it was sitting in the
> {{ReplicationSourceWALReader#checkQuota()}}'s sleep call, _not_ the
> {{handleEmptyWALBatch()}} method on the same class.
> My only assumption is that, somehow, these RegionServers got into a situation
> where they "allocated" memory from the quota but never freed it. Then,
> because the WAL reader thinks it has no free memory, it blocks indefinitely
> and there are no pending edits to ship and (thus) free that memory. A cursory
> glance at the code gives me a _lot_ of anxiety around places where we don't
> properly clean it up (e.g. batches that fail to ship, dropping a peer). As a
> first stab, let me add some more debugging so we can actually track this
> state properly for the operators and their sanity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
virajjasani commented on pull request #2172: URL: https://github.com/apache/hbase/pull/2172#issuecomment-667079180 Precommit run is not getting scheduled, waiting logs are present for long time: ``` Still waiting to schedule task Waiting for next available executor on ‘Hadoop’ ``` I tried manual trigger but the run is still not scheduled. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24794) hbase.rowlock.wait.duration should not be <= 0
[
https://issues.apache.org/jira/browse/HBASE-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168633#comment-17168633
]
Hudson commented on HBASE-24794:
Results for branch branch-1
[build #1335 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> hbase.rowlock.wait.duration should not be <= 0
> --
>
> Key: HBASE-24794
> URL: https://issues.apache.org/jira/browse/HBASE-24794
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.0, 1.5.0, 2.2.0, 2.3.0, 1.6.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 1.4.14, 2.2.6
>
>
> had a cluster fail after upgrade from hbase 1 because all writes to meta
> failed.
> master started in maintenance mode looks like (RS hosting meta in non-maint
> would look similar starting with {{HRegion.doBatchMutate}}):
> {code}
> 2020-07-28 17:52:56,553 WARN org.apache.hadoop.hbase.regionserver.HRegion:
> Failed getting lock, row=some_user_table
> java.io.IOException: Timed out waiting for lock for row: some_user_table in
> region 1588230740
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLockInternal(HRegion.java:5863)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lockRowsAndBuildMiniBatch(HRegion.java:3322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3992)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3923)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3914)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3928)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4255)
> at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3047)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2827)
> at
> org.apache.hadoop.hbase.client.ClientServiceCallable.doMutate(ClientServiceCallable.java:55)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:538)
> at org.apache.hadoop.hbase.client.HTable$3.rpcCall(HTable.java:533)
> at
> org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:127)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:542)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1339)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1329)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1672)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1112)
> at
> org.apache.hadoop.hbase.master.TableStateManager.fixTableStates(TableStateManager.java:296)
> at
> org.apache.hadoop.hbase.master.TableStateManager.start(TableStateManager.java:269)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1004)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2274)
> at
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:583)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> logging roughly 6k times /second.
> failure was caused by a change in behavior for
> {{hbase.rowlock.wait.duration}} in HBASE-17210 (so 1.4.0+, 2.0.0+). Prior to
> that change setting the config <= 0 meant that row locks would succeed only
> if they were immediately available. After the change we fail the lock attempt
> without checking the lock at all.
> workaround: set {{hbase.rowlock.wait.duration}} to a small positive number,
> e.g. 1, if you
[jira] [Commented] (HBASE-20226) Performance Improvement Taking Large Snapshots In Remote Filesystems
[
https://issues.apache.org/jira/browse/HBASE-20226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168631#comment-17168631
]
Hudson commented on HBASE-20226:
Results for branch branch-1
[build #1335 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Performance Improvement Taking Large Snapshots In Remote Filesystems
>
>
> Key: HBASE-20226
> URL: https://issues.apache.org/jira/browse/HBASE-20226
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.7.0
> Environment: HBase 1.4.0 running on an AWS EMR cluster with the
> hbase.rootdir set to point to a folder in S3
>Reporter: Saad Mufti
>Assignee: Bharath Vissapragada
>Priority: Minor
> Labels: perfomance
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
> Attachments: HBASE-20226..01.patch
>
>
> When taking a snapshot of any table, one of the last steps is to delete the
> region manifests, which have already been rolled up into a larger overall
> manifest and thus have redundant information.
> This proposal is to do the deletion in a thread pool bounded by
> hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the
> current single threaded deletion is taking longer than all the rest of the
> snapshot tasks when the Hbase data and the snapshot folder are both in a
> remote filesystem like S3.
> I have a patch for this proposal almost ready and will submit it tomorrow for
> feedback, although I haven't had a chance to write any tests yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24788) Fix the connection leaks on getting hbase admin from unclosed connection
[
https://issues.apache.org/jira/browse/HBASE-24788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168632#comment-17168632
]
Hudson commented on HBASE-24788:
Results for branch branch-1
[build #1335 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1335//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Fix the connection leaks on getting hbase admin from unclosed connection
>
>
> Key: HBASE-24788
> URL: https://issues.apache.org/jira/browse/HBASE-24788
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 1.6.0
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.4.0, 2.2.7
>
>
> Observed the significant increase in ZK connection on performance testing on
> map reduce jobs. Turns out the
> [TableOutputFormat.checkOutputSpecs()|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java#L182]
> is not closing the connection it uses to get the hbase admin. It closes the
> hbase admin but never close the connection to get the admin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24404: --- Release Note: Enhance the flush region command that support to specify a column family. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' was: Enhance the flush region command that support specify a column family. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24404: --- Release Note: Enhance the flush region command that support specify a column family. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' was: Enhance the flush region command that support to specify a column family. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24404: --- Release Note: Enhance the flush region command that support specify a column family. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' was: Enhance the flush region command that support specify a column. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24404: --- Release Note: Enhance the flush region command that support specify a column. Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' was: Enhance the flush region command that support specify a column Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-24404: --- Release Note: Enhance the flush region command that support specify a column Usage: flush 'REGIONNAME','FAMILYNAME' flush 'ENCODED_REGIONNAME','FAMILYNAME' > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168624#comment-17168624 ] Zheng Wang commented on HBASE-24404: bq. Mind put a release note text on the jira, Zheng Wang? No prob. bq. Since it's a new API, I guess it's better backport to branch-2 only, as 2.3.0 was already released. Makes sense. Thanks. > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168610#comment-17168610 ] Wellington Chevreuil commented on HBASE-24404: -- Mind put a release note text on the jira, [~filtertip]? Since it's a new API, I guess it's better backport to branch-2 only, as 2.3.0 was already released. > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] wchevreuil commented on a change in pull request #2179: HBASE-24694 Support flush a single column family of table
wchevreuil commented on a change in pull request #2179:
URL: https://github.com/apache/hbase/pull/2179#discussion_r463507819
##
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
##
@@ -513,6 +513,15 @@ default void modifyColumnFamily(TableName tableName,
ColumnFamilyDescriptor colu
*/
void flush(TableName tableName) throws IOException;
+ /**
+ * Flush a table. Synchronous operation.
Review comment:
We should explain better how this overloaded version differs from
`flush(TableName tableName)`.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/FlushTableSubprocedure.java
##
@@ -88,11 +95,15 @@ private void flushRegions() throws ForeignException {
throw new IllegalStateException("Attempting to flush "
+ table + " but we currently have outstanding tasks");
}
-
+List families = null;
+if (family != null) {
+ LOG.debug("Flush regions with specified family:{}", family);
Review comment:
Nit: It's better to be more specific when debugging: "About to flush
family {} on all regions for table {}"
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/RegionServerFlushTableProcedureManager.java
##
@@ -183,8 +185,17 @@ public Subprocedure buildSubprocedure(String table) {
@Override
public Subprocedure buildSubprocedure(String name, byte[] data) {
+ String family = null;
+ if (data.length > 0) {
Review comment:
Will it always be non null even when we don't specify any family?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
##
@@ -302,6 +302,13 @@
*/
CompletableFuture flush(TableName tableName);
+ /**
+ * Flush a table.
Review comment:
We should explain better how this overloaded version differs from
flush(TableName tableName).
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/FlushTableSubprocedure.java
##
@@ -65,7 +72,7 @@ public Void call() throws Exception {
region.startRegionOperation();
try {
LOG.debug("Flush region " + region.toString() + " started...");
-region.flush(true);
+region.flushcache(families, false, FlushLifeCycleTracker.DUMMY);
Review comment:
Can this affect the other versions that don't specify any families, like
`flush(TableName tableName)`? Because `region.flush(true)`, internally, loads
all families before delegating to `flushcache(List families, boolean
writeFlushRequestWalMarker, FlushLifeCycleTracker tracker)`, but it doesn't
seem we are doing this now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168545#comment-17168545 ] Zheng Wang commented on HBASE-24404: [~wchevreuil]Thanks. > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168545#comment-17168545 ] Zheng Wang edited comment on HBASE-24404 at 7/31/20, 9:09 AM: -- [~wchevreuil] Thanks. was (Author: filtertip): [~wchevreuil]Thanks. > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bsglz commented on pull request #2098: HBASE-24404 Support flush a single column family of region
bsglz commented on pull request #2098: URL: https://github.com/apache/hbase/pull/2098#issuecomment-667019517 @wchevreuil Thanks a lot. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168519#comment-17168519 ] Wellington Chevreuil commented on HBASE-24404: -- Merged branch-2 PR. > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-24404: - Fix Version/s: 2.4.0 3.0.0-alpha-1 > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24404) Support flush a single column family of region
[ https://issues.apache.org/jira/browse/HBASE-24404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-24404: - Affects Version/s: 2.4.0 > Support flush a single column family of region > -- > > Key: HBASE-24404 > URL: https://issues.apache.org/jira/browse/HBASE-24404 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 3.0.0-alpha-1, 2.4.0 >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > > It is useful when we set many column families for a table, could avoid > generate too many small files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] wchevreuil merged pull request #2098: HBASE-24404 Support flush a single column family of region
wchevreuil merged pull request #2098: URL: https://github.com/apache/hbase/pull/2098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
wchevreuil commented on a change in pull request #2180: URL: https://github.com/apache/hbase/pull/2180#discussion_r463474952 ## File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java ## @@ -69,7 +69,7 @@ Compression.Algorithm.NONE, Compression.Algorithm.GZ }; - protected Configuration conf; + protected final Configuration conf; Review comment: > I think we should just release note the break, especially given the on-going issue of "should we be labeling these things IA.Public yet?" Agreed with putting description on the release notes. And feel weird that these are IA Public, should it be changed to IA Private here? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] wchevreuil commented on pull request #72: HBASE-24793 [hbase-operator-tools] hbck2 filesystem fails against hba…
wchevreuil commented on pull request #72: URL: https://github.com/apache/hbase-operator-tools/pull/72#issuecomment-666997744 retest build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on a change in pull request #2094: HBASE-24680 Refactor the checkAndMutate code on the server side
brfrn169 commented on a change in pull request #2094:
URL: https://github.com/apache/hbase/pull/2094#discussion_r463466849
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4272,43 +4275,96 @@ protected Durability getEffectiveDurability(Durability
d) {
}
@Override
+ @Deprecated
public boolean checkAndMutate(byte[] row, byte[] family, byte[] qualifier,
CompareOperator op,
ByteArrayComparable comparator, TimeRange timeRange, Mutation mutation)
throws IOException {
-return doCheckAndRowMutate(row, family, qualifier, op, comparator, null,
timeRange, null,
- mutation);
+CheckAndMutate.Builder builder = CheckAndMutate.newBuilder(row)
+ .ifMatches(family, qualifier, op,
comparator.getValue()).timeRange(timeRange);
+try {
+ if (mutation instanceof Put) {
+return checkAndMutate(builder.build((Put) mutation)).isSuccess();
+ } else if (mutation instanceof Delete) {
+return checkAndMutate(builder.build((Delete) mutation)).isSuccess();
+ } else {
+throw new DoNotRetryIOException(
+ "Unsupported mutate type: " +
mutation.getClass().getSimpleName().toUpperCase());
+ }
+} catch (IllegalArgumentException e) {
Review comment:
Sure, thanks. Will do that as your review.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24805) HBaseTestingUtility.getConnection should be threadsafe
[
https://issues.apache.org/jira/browse/HBASE-24805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-24805:
Status: Patch Available (was: Open)
> HBaseTestingUtility.getConnection should be threadsafe
> --
>
> Key: HBASE-24805
> URL: https://issues.apache.org/jira/browse/HBASE-24805
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> the current javadoc for getConnection carries a thread safety warning:
> {code}
> /**
> * Get a Connection to the cluster. Not thread-safe (This class needs a
> lot of work to make it
> * thread-safe).
> * @return A Connection that can be shared. Don't close. Will be closed on
> shutdown of cluster.
> */
>public Connection getConnection() throws IOException {
> {code}
> We then ignore that warning across our test base. We should make the method
> threadsafe since the intention is to share a single Connection across all
> users of the HTU instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on a change in pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
busbey commented on a change in pull request #2172:
URL: https://github.com/apache/hbase/pull/2172#discussion_r463455659
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionMover2.java
##
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.MiniHBaseCluster;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegionServer;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Tests for Region Mover Load/Unload functionality with and without ack mode
and also to test
+ * exclude functionality useful for rack decommissioning
+ */
+@Category({ MiscTests.class, LargeTests.class})
+public class TestRegionMover2 {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionMover2.class);
+
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionMover2.class);
+
+ private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+
+ private static final TableName TABLE_NAME =
TableName.valueOf("testRegionMover2");
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+TEST_UTIL.startMiniCluster(3);
+TEST_UTIL.getAdmin().balancerSwitch(false, true);
+ }
+
+ @AfterClass
+ public static void tearDownAfterClass() throws Exception {
+TEST_UTIL.shutdownMiniCluster();
+ }
+
+ @Before
+ public void setUp() throws Exception {
+// Create a pre-split table just to populate some regions
+Admin admin = TEST_UTIL.getAdmin();
+if (admin.tableExists(TABLE_NAME)) {
+ TEST_UTIL.deleteTable(TABLE_NAME);
+}
+TableDescriptor tableDesc = TableDescriptorBuilder.newBuilder(TABLE_NAME)
+ .setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam1")).build();
+int startKey = 0;
+int endKey = 8;
+admin.createTable(tableDesc, Bytes.toBytes(startKey),
Bytes.toBytes(endKey), 9);
+ }
+
+ @Test
+ public void testWithMergedRegions() throws Exception {
+MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
+Admin admin = TEST_UTIL.getAdmin();
+Table table = TEST_UTIL.getConnection().getTable(TABLE_NAME);
+List puts = new ArrayList<>();
+for (int i = 0; i < 1; i++) {
+ puts.add(new Put(Bytes.toBytes("rowkey_" + i))
+.addColumn(Bytes.toBytes("fam1"), Bytes.toBytes("q1"),
Bytes.toBytes("val_" + i)));
+}
+table.put(puts);
+admin.flush(TABLE_NAME);
+HRegionServer regionServer = cluster.getRegionServer(0);
+String rsName = regionServer.getServerName().getAddress().toString();
+int numRegions = regionServer.getNumberOfOnlineRegions();
+List hRegions = regionServer.getRegions().stream()
+ .filter(hRegion -> hRegion.getRegionInfo().getTable().equals(TABLE_NAME))
+ .collect(Collectors.toList());
+RegionMover.RegionMoverBuilder rmBuilder =
+ new RegionMover.RegionMoverBuilder(rsName,
TEST_UTIL.getConfiguration()).ack(true)
+.maxthreads(8);
+try (RegionMover rm = rmBuilder.build()) {
+ LOG.debug("Unloading " + regionServer.getServerName());
+ rm.unload();
+ Assert.assertEquals(0,
[GitHub] [hbase] busbey commented on a change in pull request #2172: HBASE-24795 : RegionMover to deal with unknown region while (un)loading
busbey commented on a change in pull request #2172:
URL: https://github.com/apache/hbase/pull/2172#discussion_r463455807
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestRegionMover2.java
##
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.util;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.MiniHBaseCluster;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.regionserver.HRegionServer;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.MiscTests;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Tests for Region Mover Load/Unload functionality with and without ack mode
and also to test
+ * exclude functionality useful for rack decommissioning
+ */
+@Category({ MiscTests.class, LargeTests.class})
+public class TestRegionMover2 {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionMover2.class);
+
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionMover2.class);
+
+ private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+
+ private static final TableName TABLE_NAME =
TableName.valueOf("testRegionMover2");
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+TEST_UTIL.startMiniCluster(3);
+TEST_UTIL.getAdmin().balancerSwitch(false, true);
+ }
+
+ @AfterClass
+ public static void tearDownAfterClass() throws Exception {
+TEST_UTIL.shutdownMiniCluster();
+ }
+
+ @Before
+ public void setUp() throws Exception {
+// Create a pre-split table just to populate some regions
+Admin admin = TEST_UTIL.getAdmin();
+if (admin.tableExists(TABLE_NAME)) {
+ TEST_UTIL.deleteTable(TABLE_NAME);
+}
+TableDescriptor tableDesc = TableDescriptorBuilder.newBuilder(TABLE_NAME)
+ .setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam1")).build();
+int startKey = 0;
+int endKey = 8;
+admin.createTable(tableDesc, Bytes.toBytes(startKey),
Bytes.toBytes(endKey), 9);
+ }
+
+ @Test
+ public void testWithMergedRegions() throws Exception {
+MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
+Admin admin = TEST_UTIL.getAdmin();
+Table table = TEST_UTIL.getConnection().getTable(TABLE_NAME);
Review comment:
okay ignore this now that fixing it is tracked in #2180
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #2094: HBASE-24680 Refactor the checkAndMutate code on the server side
Apache9 commented on a change in pull request #2094:
URL: https://github.com/apache/hbase/pull/2094#discussion_r46398
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4272,43 +4275,96 @@ protected Durability getEffectiveDurability(Durability
d) {
}
@Override
+ @Deprecated
public boolean checkAndMutate(byte[] row, byte[] family, byte[] qualifier,
CompareOperator op,
ByteArrayComparable comparator, TimeRange timeRange, Mutation mutation)
throws IOException {
-return doCheckAndRowMutate(row, family, qualifier, op, comparator, null,
timeRange, null,
- mutation);
+CheckAndMutate.Builder builder = CheckAndMutate.newBuilder(row)
+ .ifMatches(family, qualifier, op,
comparator.getValue()).timeRange(timeRange);
+try {
+ if (mutation instanceof Put) {
+return checkAndMutate(builder.build((Put) mutation)).isSuccess();
+ } else if (mutation instanceof Delete) {
+return checkAndMutate(builder.build((Delete) mutation)).isSuccess();
+ } else {
+throw new DoNotRetryIOException(
+ "Unsupported mutate type: " +
mutation.getClass().getSimpleName().toUpperCase());
+ }
+} catch (IllegalArgumentException e) {
Review comment:
Then we'd better let the try catch for IllegalArgumentException only
wrap the builder.build? And then we call checkAndMutate, without catching the
IllegalArgumentException. I think this will be better to let others know why we
may hit IllegalArgumentException.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] busbey commented on a change in pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
busbey commented on a change in pull request #2180: URL: https://github.com/apache/hbase/pull/2180#discussion_r463454640 ## File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java ## @@ -69,7 +69,7 @@ Compression.Algorithm.NONE, Compression.Algorithm.GZ }; - protected Configuration conf; + protected final Configuration conf; Review comment: Changing this variable to final break binary compatibility, but is needed to make access to this field threadsafe. I could maintain binary compatibility by making a _new_ final variable called "originalConf" or something like that which is used by our own internals. but that would still cause a behavior change because if someone made a subclass of these classes that assigned to the `conf` variable we would not pay any attention to that change. I think we should just release note the break, especially given the on-going issue of "should we be labeling these things IA.Public yet?" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey commented on a change in pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
busbey commented on a change in pull request #2180: URL: https://github.com/apache/hbase/pull/2180#discussion_r463454640 ## File path: hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java ## @@ -69,7 +69,7 @@ Compression.Algorithm.NONE, Compression.Algorithm.GZ }; - protected Configuration conf; + protected final Configuration conf; Review comment: Changing this variable to final breaks binary compatibility, but is needed to make access to this field threadsafe. I could maintain binary compatibility by making a _new_ final variable called "originalConf" or something like that which is used by our own internals. but that would still cause a behavior change because if someone made a subclass of these classes that assigned to the `conf` variable we would not pay any attention to that change. I think we should just release note the break, especially given the on-going issue of "should we be labeling these things IA.Public yet?" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] busbey opened a new pull request #2180: HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
busbey opened a new pull request #2180: URL: https://github.com/apache/hbase/pull/2180 This maintains *most* compatibility while changing the handling of the shared connection to threadsafe. Note about compat as comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-24805) HBaseTestingUtility.getConnection should be threadsafe
[
https://issues.apache.org/jira/browse/HBASE-24805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned HBASE-24805:
---
Assignee: Sean Busbey
> HBaseTestingUtility.getConnection should be threadsafe
> --
>
> Key: HBASE-24805
> URL: https://issues.apache.org/jira/browse/HBASE-24805
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
>
> the current javadoc for getConnection carries a thread safety warning:
> {code}
> /**
> * Get a Connection to the cluster. Not thread-safe (This class needs a
> lot of work to make it
> * thread-safe).
> * @return A Connection that can be shared. Don't close. Will be closed on
> shutdown of cluster.
> */
>public Connection getConnection() throws IOException {
> {code}
> We then ignore that warning across our test base. We should make the method
> threadsafe since the intention is to share a single Connection across all
> users of the HTU instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24805) HBaseTestingUtility.getConnection should be threadsafe
Sean Busbey created HBASE-24805:
---
Summary: HBaseTestingUtility.getConnection should be threadsafe
Key: HBASE-24805
URL: https://issues.apache.org/jira/browse/HBASE-24805
Project: HBase
Issue Type: Bug
Components: test
Reporter: Sean Busbey
the current javadoc for getConnection carries a thread safety warning:
{code}
/**
* Get a Connection to the cluster. Not thread-safe (This class needs a lot
of work to make it
* thread-safe).
* @return A Connection that can be shared. Don't close. Will be closed on
shutdown of cluster.
*/
public Connection getConnection() throws IOException {
{code}
We then ignore that warning across our test base. We should make the method
threadsafe since the intention is to share a single Connection across all users
of the HTU instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bsglz opened a new pull request #2179: HBASE-24694 Support flush a single column family of table
bsglz opened a new pull request #2179: URL: https://github.com/apache/hbase/pull/2179 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 commented on pull request #2094: HBASE-24680 Refactor the checkAndMutate code on the server side
brfrn169 commented on pull request #2094: URL: https://github.com/apache/hbase/pull/2094#issuecomment-666945012 Ping @Apache9 Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
