[GitHub] [hbase] Apache-HBase commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache-HBase commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-623868587 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 23s | master passed | | +1 :green_heart: | compile | 1m 40s | master passed | | +1 :green_heart: | shadedjars | 4m 58s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | compile | 1m 42s | the patch passed | | +1 :green_heart: | javac | 1m 42s | the patch passed | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 12s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 44s | hbase-client in the patch failed. | | +1 :green_heart: | unit | 133m 40s | hbase-server in the patch passed. | | | | 162m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1593 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 78e6f2b62b30 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-client.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/testReport/ | | Max. process+thread count | 4636 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #1649: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
virajjasani commented on pull request #1649: URL: https://github.com/apache/hbase/pull/1649#issuecomment-623865640 > getByte() and two variant of putByte() are missing. Rest looks good. Both are present on JDK7: http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/src/share/classes/sun/misc/Unsafe.java This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #1652: HBASE-24325: TestJMXConnectorServer can fail to start the minicluster…
virajjasani commented on a change in pull request #1652: URL: https://github.com/apache/hbase/pull/1652#discussion_r419867480 ## File path: hbase-server/src/test/java/org/apache/hadoop/hbase/TestJMXConnectorServer.java ## @@ -219,4 +222,18 @@ public void preExecuteProcedures(ObserverContext
[jira] [Commented] (HBASE-24262) Improvement update_config for PressureAwareCompactionThroughputController
[ https://issues.apache.org/jira/browse/HBASE-24262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099534#comment-17099534 ] yaojingyi commented on HBASE-24262: --- [~Joseph295] If we start major compaction on a big table first, then we modified compaction limit configuration and execute 'update_config' command, it will not effective immediately. The compaction which is running is still use the old Controller object. The higher.bound,lower.bound,offPeakHours is still the old value. > Improvement update_config for PressureAwareCompactionThroughputController > - > > Key: HBASE-24262 > URL: https://issues.apache.org/jira/browse/HBASE-24262 > Project: HBase > Issue Type: Improvement > Components: Compaction >Affects Versions: master >Reporter: yaojingyi >Assignee: yaojingyi >Priority: Major > Attachments: HBASE-24262.master.001.patch, > HBASE-24262.master.002.patch > > > If we modified compaction limit configuration,and use 'update_config' to > update it . > It will not effective in this round of major compaction. > I think we can make the modification effective immediately. > [https://github.com/apache/hbase/pull/1588] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1638: HBASE-24315 Remove hadoop-two-compat.xml in hbase-assembly
Apache-HBase commented on pull request #1638: URL: https://github.com/apache/hbase/pull/1638#issuecomment-623857340 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 48s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 33s | master passed | | +1 :green_heart: | compile | 3m 20s | master passed | | +1 :green_heart: | shadedjars | 6m 20s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | root in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 2m 59s | the patch passed | | +1 :green_heart: | javac | 2m 59s | the patch passed | | +1 :green_heart: | shadedjars | 5m 43s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | root in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 238m 2s | root in the patch passed. | | | | 271m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1638 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a0df7010434a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/testReport/ | | Max. process+thread count | 4613 (vs. ulimit of 12500) | | modules | C: hbase-assembly . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1652: HBASE-24325: TestJMXConnectorServer can fail to start the minicluster…
Apache-HBase commented on pull request #1652: URL: https://github.com/apache/hbase/pull/1652#issuecomment-623856109 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | branch-2 passed | | +1 :green_heart: | compile | 0m 56s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 183m 23s | hbase-server in the patch passed. | | | | 205m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1652 | | JIRA Issue | HBASE-24325 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9a910a1ac651 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3a375e668c | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/testReport/ | | Max. process+thread count | 2708 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liuml07 commented on pull request #1647: add IntelliJ project files
liuml07 commented on pull request #1647: URL: https://github.com/apache/hbase/pull/1647#issuecomment-623854938 @ndimiduk I looked at the doc and found it makes sense to include idea files in CVS. It's very useful if `automatically add to Version Control` works for us. However, during testing, I found this I tested this `project-jdk-name` in `misc.xml` would be per-developer. For e.g., in my laptop, it is: ``` diff --git a/.idea/misc.xml b/.idea/misc.xml index 73c342a45e..2e7b92e141 100644 --- a/.idea/misc.xml +++ b/.idea/misc.xml @@ -13,5 +13,5 @@ - + \ No newline at end of file ``` Maybe that is because I named my JDK `1.8` locally. I tested this patch with new clones. - Importing the whole directory works. I can see a lot of `**/*.iml` files for those modules. I can search symbols. - Importing the `pom.xml` also works. I can see a lot of `**/*.iml` files for those modules. I can search symbols. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
Apache-HBase commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-623853790 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 48s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 45s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 23s | master passed | | +1 :green_heart: | compile | 3m 36s | master passed | | +1 :green_heart: | shadedjars | 7m 17s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | root in master failed. | | -0 :warning: | javadoc | 0m 20s | hbase-examples in master failed. | | -0 :warning: | javadoc | 0m 23s | hbase-mapreduce in master failed. | | -0 :warning: | javadoc | 0m 24s | hbase-rest in master failed. | | -0 :warning: | javadoc | 0m 52s | hbase-server in master failed. | | -0 :warning: | javadoc | 0m 56s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 2m 44s | the patch passed | | +1 :green_heart: | javac | 2m 44s | the patch passed | | +1 :green_heart: | shadedjars | 5m 25s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-asyncfs in the patch failed. | | -0 :warning: | javadoc | 0m 17s | root in the patch failed. | | -0 :warning: | javadoc | 0m 17s | hbase-examples in the patch failed. | | -0 :warning: | javadoc | 0m 19s | hbase-mapreduce in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-rest in the patch failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 47s | hbase-thrift in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 160m 45s | root in the patch passed. | | | | 202m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1628 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7860ee3cd534 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-examples.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-asyncfs.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-examples.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-mapreduce.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt | | javadoc |
[GitHub] [hbase] Apache-HBase commented on pull request #1651: HBASE-24323 Avoid two parallel running of GCMulitpleMergedRegionsProc…
Apache-HBase commented on pull request #1651: URL: https://github.com/apache/hbase/pull/1651#issuecomment-623847232 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 46s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 40s | master passed | | +1 :green_heart: | compile | 1m 27s | master passed | | +1 :green_heart: | shadedjars | 7m 18s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 53s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 48s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 205m 4s | hbase-server in the patch passed. | | | | 237m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1651 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a262fe76340e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/testReport/ | | Max. process+thread count | 2904 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1651: HBASE-24323 Avoid two parallel running of GCMulitpleMergedRegionsProc…
Apache-HBase commented on pull request #1651: URL: https://github.com/apache/hbase/pull/1651#issuecomment-623847155 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 5m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 210m 9s | hbase-server in the patch passed. | | | | 237m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1651 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4089331cfffe 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/testReport/ | | Max. process+thread count | 3477 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache-HBase commented on pull request #1593: URL: https://github.com/apache/hbase/pull/1593#issuecomment-623846322 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 52s | master passed | | +1 :green_heart: | checkstyle | 2m 22s | master passed | | +1 :green_heart: | spotbugs | 4m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 41s | the patch passed | | -0 :warning: | checkstyle | 0m 37s | hbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 14m 55s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 52m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1593 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 6f74baa32cc6 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1593/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1652: HBASE-24325: TestJMXConnectorServer can fail to start the minicluster…
Apache-HBase commented on pull request #1652: URL: https://github.com/apache/hbase/pull/1652#issuecomment-623845325 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | branch-2 passed | | +1 :green_heart: | compile | 1m 3s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 16s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 42s | hbase-server in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 5m 10s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 120m 41s | hbase-server in the patch passed. | | | | 146m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1652 | | JIRA Issue | HBASE-24325 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cada94e56caa 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3a375e668c | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/testReport/ | | Max. process+thread count | 3644 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1648: HBASE-8458 Support for batch version of checkAndMutate()
Apache-HBase commented on pull request #1648: URL: https://github.com/apache/hbase/pull/1648#issuecomment-623812105 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 35s | master passed | | +1 :green_heart: | compile | 3m 49s | master passed | | +1 :green_heart: | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 28s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 3m 41s | the patch passed | | +1 :green_heart: | javac | 3m 41s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 55s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. | | -1 :x: | unit | 208m 59s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 5m 25s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 4m 16s | hbase-rest in the patch passed. | | | | 260m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1648 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8b6b1e789d3b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/testReport/ | | Max. process+thread count | 3172 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099502#comment-17099502 ] Michael Stack commented on HBASE-24318: --- Added you as contributor and assigned you this issue [~mattf] sir > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reassigned HBASE-24318: - Assignee: Matthew Foley > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Assignee: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419840454
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
+void call(HBaseRpcController controller, ClientMetaService.Interface stub,
RpcCallback done);
}
- /**
- * Parses the list of master addresses from the provided configuration.
Supported format is
- * comma separated host[:port] values. If no port number if specified,
default master port is
- * assumed.
- * @param conf Configuration to parse from.
- */
- private void parseMasterAddrs(Configuration conf) throws
UnknownHostException {
-String configuredMasters = getMasterAddr(conf);
-for (String masterAddr:
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
- HostAndPort masterHostPort =
-
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
- masterServers.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
-}
-Preconditions.checkArgument(!masterServers.isEmpty(), "At least one master
address is needed");
+ private CompletableFuture
call(ClientMetaService.Interface stub,
+Callable callable) {
+HBaseRpcController controller = rpcControllerFactory.newController();
+CompletableFuture future = new CompletableFuture<>();
+callable.call(controller, stub, resp -> {
+ if (controller.failed()) {
+future.completeExceptionally(controller.getFailed());
+ } else {
+future.complete(resp);
+ }
+});
+return future;
}
- @VisibleForTesting
- public Set getParsedMasterServers() {
-return Collections.unmodifiableSet(masterServers);
+ private IOException badResponse(String debug) {
+return new IOException(String.format("Invalid result for request %s. Will
be retried", debug));
}
- /**
- * Returns a call back that can be passed along to the non-blocking rpc
call. It is invoked once
- * the rpc finishes and the response is propagated to the passed future.
- * @param future Result future to which the rpc response is propagated.
- * @param isValidResp Checks if the rpc response has a valid result.
- * @param transformResult Transforms the result to a different form as
expected by callers.
- * @param hrc RpcController instance for this rpc.
- * @param debug Debug message passed along to the caller in case of
exceptions.
- * @param RPC result type.
- * @param Transformed type of the result.
- * @return A call back that can be embedded in the non-blocking rpc call.
- */
- private RpcCallback getRpcCallBack(CompletableFuture future,
- Predicate isValidResp, Function transformResult,
HBaseRpcController hrc,
- final String debug) {
-return rpcResult -> {
- if (rpcResult == null) {
-future.completeExceptionally(
-new MasterRegistryFetchException(masterServers, hrc.getFailed()));
-return;
- }
- if (!isValidResp.test(rpcResult)) {
-// Rpc returned ok, but result was malformed.
-future.completeExceptionally(new IOException(
-String.format("Invalid result for request %s. Will be retried",
debug)));
-return;
- }
- future.complete(transformResult.apply(rpcResult));
-};
+ // send requests concurrently to hedgedReadsFanout masters
+ private void groupCall(CompletableFuture future, int
startIndexInclusive,
Review comment:
I added more comments for this method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419840354
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
+void call(HBaseRpcController controller, ClientMetaService.Interface stub,
RpcCallback done);
}
- /**
- * Parses the list of master addresses from the provided configuration.
Supported format is
- * comma separated host[:port] values. If no port number if specified,
default master port is
- * assumed.
- * @param conf Configuration to parse from.
- */
- private void parseMasterAddrs(Configuration conf) throws
UnknownHostException {
-String configuredMasters = getMasterAddr(conf);
-for (String masterAddr:
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
- HostAndPort masterHostPort =
-
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
- masterServers.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
-}
-Preconditions.checkArgument(!masterServers.isEmpty(), "At least one master
address is needed");
+ private CompletableFuture
call(ClientMetaService.Interface stub,
+Callable callable) {
+HBaseRpcController controller = rpcControllerFactory.newController();
+CompletableFuture future = new CompletableFuture<>();
+callable.call(controller, stub, resp -> {
+ if (controller.failed()) {
+future.completeExceptionally(controller.getFailed());
+ } else {
+future.complete(resp);
+ }
+});
+return future;
}
- @VisibleForTesting
- public Set getParsedMasterServers() {
-return Collections.unmodifiableSet(masterServers);
+ private IOException badResponse(String debug) {
+return new IOException(String.format("Invalid result for request %s. Will
be retried", debug));
}
- /**
- * Returns a call back that can be passed along to the non-blocking rpc
call. It is invoked once
- * the rpc finishes and the response is propagated to the passed future.
- * @param future Result future to which the rpc response is propagated.
- * @param isValidResp Checks if the rpc response has a valid result.
- * @param transformResult Transforms the result to a different form as
expected by callers.
- * @param hrc RpcController instance for this rpc.
- * @param debug Debug message passed along to the caller in case of
exceptions.
- * @param RPC result type.
- * @param Transformed type of the result.
- * @return A call back that can be embedded in the non-blocking rpc call.
- */
- private RpcCallback getRpcCallBack(CompletableFuture future,
- Predicate isValidResp, Function transformResult,
HBaseRpcController hrc,
- final String debug) {
-return rpcResult -> {
- if (rpcResult == null) {
-future.completeExceptionally(
-new MasterRegistryFetchException(masterServers, hrc.getFailed()));
-return;
- }
- if (!isValidResp.test(rpcResult)) {
-// Rpc returned ok, but result was malformed.
-future.completeExceptionally(new IOException(
-String.format("Invalid result for request %s. Will be retried",
debug)));
-return;
- }
- future.complete(transformResult.apply(rpcResult));
-};
+ // send requests concurrently to hedgedReadsFanout masters
+ private void groupCall(CompletableFuture future, int
startIndexInclusive,
+Callable callable, Predicate isValidResp, String debug,
+ConcurrentLinkedQueue errors) {
+int endIndexExclusive = Math.min(startIndexInclusive + hedgedReadFanOut,
masterStubs.size());
+AtomicInteger remaining = new AtomicInteger(endIndexExclusive -
startIndexInclusive);
+for (int i = startIndexInclusive; i < endIndexExclusive; i++) {
+ addListener(call(masterStubs.get(i), callable), (r, e) -> {
+// a simple check to skip all the later operations earlier
+if (future.isDone()) {
+ return;
+}
+if (e == null && !isValidResp.test(r)) {
+ e = badResponse(debug);
+}
+if (e != null) {
+ // make sure when remaining reaches 0 we have all exceptions in the
errors queue
+ errors.add(e);
+ if (remaining.decrementAndGet() == 0) {
+if (endIndexExclusive == masterStubs.size()) {
+ // we are done, complete the
[jira] [Commented] (HBASE-24321) Add writable MinVersions and read-only Scan to coproc ScanOptions
[ https://issues.apache.org/jira/browse/HBASE-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099497#comment-17099497 ] Geoffrey Jacoby commented on HBASE-24321: - [~zhangduo] - A coproc developer who overrides a compaction coproc and overrides MinVersions on ScanOptions has exactly one thing to worry about: that he or she has set the MinVersions to the wrong value, and hence their cluster may retain too many, or too few versions on one or more tables/CFs. A coproc developer who writes a complicated Scanner wrapper has the above to worry about, as well as the prospect of resource leaks and accidentally crashing a RegionServer or the entire cluster. (As I said, I've been there. :-) ) I don't see how the latter option is ever better, if both options are available. Of course, there will be times when a coproc developer wants to do something really complicated, and needs to do the latter option with all its extra risks -- but a good API should make simple things easy and safe, and limit the reasons you need to do dangerous ones. A good API exists here -- it just needs to be extended. What I'm proposing is pretty much exactly the same as HBASE-19895, which added KeepDeletedCells to ScanOptions -- and which you were the approving committer for, I believe. :-) > Add writable MinVersions and read-only Scan to coproc ScanOptions > - > > Key: HBASE-24321 > URL: https://issues.apache.org/jira/browse/HBASE-24321 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 2.3.0 > > > Between HBase 1.x and 2.0, the RegionObserver pre*ScannerOpen coprocessors > were significantly changed so that the coproc implementer no longer has > access to the actual Scanner, just a ScanOptions object that can be changed > in limited ways. This is safer and prevents resource leaks and other bugs. > While ScanOptions provides support for changing TTL, KeepDeletedCells, and > MaxVersions, a fourth column family config parameter, MinVersions, appears to > have been missed. This prevents coproc implementers from changing MinVersions > dynamically. An example of this is PHOENIX-5645, which in the forthcoming > Phoenix 4.16 (based on HBase 1.x) will allow users to configure a moving > window where all versions are kept, and thus point-in-time queries are safe. > This cannot be put in the forthcoming Phoenix 5.1 (based on HBase 2.1 and > 2.2) because of the coproc changes. > Relatedly, preStoreScannerOpen lacks access to the Scan in HBase 2.0 and up. > This prevents coprocs from reading the Scan parameters to check if, for > example, a Scan has set the max time to a point in the past, and thus needs > to override KeepDeletedCells. This can lead to incorrect behavior when doing > point-in-time queries or using transactional engines that treat physically > committed HBase writes as logically uncommitted parts of a transaction. It's > also a correctness problem for PHOENIX-5645. Please note that only > _read-only_ access to the Scan from the store scanner coproc hook is in scope > for this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1584: HBASE-24256 When fixOverlap hits the max region limit, it is possible…
Apache-HBase commented on pull request #1584: URL: https://github.com/apache/hbase/pull/1584#issuecomment-623808192 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 18s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | compile | 0m 54s | master passed | | +1 :green_heart: | shadedjars | 5m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 5m 23s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 214m 35s | hbase-server in the patch failed. | | | | 239m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1a83adbaabc8 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/testReport/ | | Max. process+thread count | 3421 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24321) Add writable MinVersions and read-only Scan to coproc ScanOptions
[
https://issues.apache.org/jira/browse/HBASE-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099489#comment-17099489
]
Duo Zhang commented on HBASE-24321:
---
{quote}
1. Wrapping an InternalScanner seems much riskier for a coprocessor author to
get right.
{quote}
I would say this is intentional. If you can not get things right through this
way then probably you should not do it. We have lots of assumptions in the
compaction code, so opening new options should be very very careful. We have
this in 1.x is not strong enough to convience me, or even opposite, as the
design here is to fix the broken behavior for 1.x. Please explain more about
the safety for adding these options to ScanOptions.
And for 2, a read only version is fine.
Thanks.
> Add writable MinVersions and read-only Scan to coproc ScanOptions
> -
>
> Key: HBASE-24321
> URL: https://issues.apache.org/jira/browse/HBASE-24321
> Project: HBase
> Issue Type: Improvement
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Major
> Fix For: 2.3.0
>
>
> Between HBase 1.x and 2.0, the RegionObserver pre*ScannerOpen coprocessors
> were significantly changed so that the coproc implementer no longer has
> access to the actual Scanner, just a ScanOptions object that can be changed
> in limited ways. This is safer and prevents resource leaks and other bugs.
> While ScanOptions provides support for changing TTL, KeepDeletedCells, and
> MaxVersions, a fourth column family config parameter, MinVersions, appears to
> have been missed. This prevents coproc implementers from changing MinVersions
> dynamically. An example of this is PHOENIX-5645, which in the forthcoming
> Phoenix 4.16 (based on HBase 1.x) will allow users to configure a moving
> window where all versions are kept, and thus point-in-time queries are safe.
> This cannot be put in the forthcoming Phoenix 5.1 (based on HBase 2.1 and
> 2.2) because of the coproc changes.
> Relatedly, preStoreScannerOpen lacks access to the Scan in HBase 2.0 and up.
> This prevents coprocs from reading the Scan parameters to check if, for
> example, a Scan has set the max time to a point in the past, and thus needs
> to override KeepDeletedCells. This can lead to incorrect behavior when doing
> point-in-time queries or using transactional engines that treat physically
> committed HBase writes as logically uncommitted parts of a transaction. It's
> also a correctness problem for PHOENIX-5645. Please note that only
> _read-only_ access to the Scan from the store scanner coproc hook is in scope
> for this change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1628: HBASE-24304 Separate a hbase-asyncfs module
Apache-HBase commented on pull request #1628: URL: https://github.com/apache/hbase/pull/1628#issuecomment-623803488 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 26s | master passed | | +1 :green_heart: | checkstyle | 2m 9s | master passed | | +1 :green_heart: | spotbugs | 15m 12s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | checkstyle | 2m 2s | root: The patch generated 0 new + 293 unchanged - 2 fixed = 293 total (was 295) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 16s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 13m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 19m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 48s | The patch does not generate ASF License warnings. | | | | 72m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1628 | | Optional Tests | dupname asflicense hadoopcheck xml spotbugs hbaseanti checkstyle | | uname | Linux c104636f107c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Max. process+thread count | 139 (vs. ulimit of 12500) | | modules | C: hbase-asyncfs hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-endpoint hbase-rest hbase-examples hbase-shaded/hbase-shaded-testing-util . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1628/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1652: HBASE-24325: TestJMXConnectorServer can fail to start the minicluster…
Apache-HBase commented on pull request #1652: URL: https://github.com/apache/hbase/pull/1652#issuecomment-623798722 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 22s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 22s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 38m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1652 | | JIRA Issue | HBASE-24325 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 76e65d2ebb0a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 3a375e668c | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1652/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1648: HBASE-8458 Support for batch version of checkAndMutate()
Apache-HBase commented on pull request #1648: URL: https://github.com/apache/hbase/pull/1648#issuecomment-623797593 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 21s | master passed | | +1 :green_heart: | compile | 4m 50s | master passed | | +1 :green_heart: | shadedjars | 5m 45s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 27s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 25s | hbase-rest in master failed. | | -0 :warning: | javadoc | 0m 53s | hbase-server in master failed. | | -0 :warning: | javadoc | 1m 8s | hbase-thrift in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 17s | the patch passed | | +1 :green_heart: | compile | 4m 30s | the patch passed | | +1 :green_heart: | javac | 4m 30s | the patch passed | | +1 :green_heart: | shadedjars | 6m 51s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 29s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 48s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 59s | hbase-thrift in the patch failed. | | -0 :warning: | javadoc | 0m 22s | hbase-rest in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 10s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 15s | hbase-client in the patch passed. | | -1 :x: | unit | 127m 59s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 4m 47s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 32s | hbase-rest in the patch passed. | | | | 182m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1648 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ddcbc38c53ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-thrift.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/testReport/ | | Max. process+thread count | 4630 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) |
[GitHub] [hbase] Apache-HBase commented on pull request #1639: HBASE-24305 Removed deprecations in ServerName
Apache-HBase commented on pull request #1639: URL: https://github.com/apache/hbase/pull/1639#issuecomment-623794366 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 37s | master passed | | +1 :green_heart: | compile | 2m 45s | master passed | | +1 :green_heart: | shadedjars | 5m 49s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 16s | hbase-balancer in master failed. | | -0 :warning: | javadoc | 0m 21s | hbase-backup in master failed. | | -0 :warning: | javadoc | 0m 21s | hbase-rest in master failed. | | -0 :warning: | javadoc | 0m 49s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 52s | the patch passed | | +1 :green_heart: | compile | 2m 46s | the patch passed | | +1 :green_heart: | javac | 2m 46s | the patch passed | | +1 :green_heart: | shadedjars | 5m 53s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 15s | hbase-balancer in the patch failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-backup in the patch failed. | | -0 :warning: | javadoc | 0m 20s | hbase-rest in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 55s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 0m 20s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 197m 22s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 13m 0s | hbase-backup in the patch passed. | | +1 :green_heart: | unit | 4m 1s | hbase-rest in the patch passed. | | | | 252m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1639 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7d5dce9dee8f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-balancer.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-backup.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-rest.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-balancer.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-backup.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-rest.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/testReport/
[GitHub] [hbase] Apache-HBase commented on pull request #1584: HBASE-24256 When fixOverlap hits the max region limit, it is possible…
Apache-HBase commented on pull request #1584: URL: https://github.com/apache/hbase/pull/1584#issuecomment-623794242 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 24s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 7m 15s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 51s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 28s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 6m 49s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 49s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 129m 57s | hbase-server in the patch failed. | | | | 162m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c9ce3f3618b2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/testReport/ | | Max. process+thread count | 3800 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099480#comment-17099480 ] Matthew Foley commented on HBASE-24318: --- Thanks, here's the link. > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24321) Add writable MinVersions and read-only Scan to coproc ScanOptions
[ https://issues.apache.org/jira/browse/HBASE-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099478#comment-17099478 ] Geoffrey Jacoby commented on HBASE-24321: - 1. Wrapping an InternalScanner seems much riskier for a coprocessor author to get right. (Some of my colleagues and I have spent some long debugging sessions in the past trying to find resource leaks that came from coprocs that got new scanner instantiation wrong!) The 2.x pattern where HBase instantiates the scanner and the coproc author just marks which parameters he or she wants to override is really nice, and I want to leverage that. There's nothing about minVersions that makes it any different from KeepDeletedCells, ttl, or max versions which already support this -- it's simply a bit of column family config. 2. What about coprocs that need to access some piece of column-family metadata plus the Scan to make the right choice? (For example, increasing max versions for a CF if a Scan is raw, or checking some Scan attribute that triggers special behavior only for certain Scans and CFs). In that case, you need the Scan in preStoreScannerOpen, don't you? Again, not to _modify_ -- my implementation which I'll post tomorrow makes a copy of the Scan -- but just to read. Nothing I'm proposing here is new, it's just taking former 1.x functionality and adapting it to the 2.x design patterns. I have a draft I'm running tests on now, and the change is quite small. > Add writable MinVersions and read-only Scan to coproc ScanOptions > - > > Key: HBASE-24321 > URL: https://issues.apache.org/jira/browse/HBASE-24321 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 2.3.0 > > > Between HBase 1.x and 2.0, the RegionObserver pre*ScannerOpen coprocessors > were significantly changed so that the coproc implementer no longer has > access to the actual Scanner, just a ScanOptions object that can be changed > in limited ways. This is safer and prevents resource leaks and other bugs. > While ScanOptions provides support for changing TTL, KeepDeletedCells, and > MaxVersions, a fourth column family config parameter, MinVersions, appears to > have been missed. This prevents coproc implementers from changing MinVersions > dynamically. An example of this is PHOENIX-5645, which in the forthcoming > Phoenix 4.16 (based on HBase 1.x) will allow users to configure a moving > window where all versions are kept, and thus point-in-time queries are safe. > This cannot be put in the forthcoming Phoenix 5.1 (based on HBase 2.1 and > 2.2) because of the coproc changes. > Relatedly, preStoreScannerOpen lacks access to the Scan in HBase 2.0 and up. > This prevents coprocs from reading the Scan parameters to check if, for > example, a Scan has set the max time to a point in the past, and thus needs > to override KeepDeletedCells. This can lead to incorrect behavior when doing > point-in-time queries or using transactional engines that treat physically > committed HBase writes as logically uncommitted parts of a transaction. It's > also a correctness problem for PHOENIX-5645. Please note that only > _read-only_ access to the Scan from the store scanner coproc hook is in scope > for this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419820280
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
Review comment:
We have a RpcCallback in the parameters so I think it is enough to show
that this is asynchronous?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419819483
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
+void call(HBaseRpcController controller, ClientMetaService.Interface stub,
RpcCallback done);
}
- /**
- * Parses the list of master addresses from the provided configuration.
Supported format is
- * comma separated host[:port] values. If no port number if specified,
default master port is
- * assumed.
- * @param conf Configuration to parse from.
- */
- private void parseMasterAddrs(Configuration conf) throws
UnknownHostException {
-String configuredMasters = getMasterAddr(conf);
-for (String masterAddr:
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
- HostAndPort masterHostPort =
-
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
- masterServers.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
-}
-Preconditions.checkArgument(!masterServers.isEmpty(), "At least one master
address is needed");
+ private CompletableFuture
call(ClientMetaService.Interface stub,
+Callable callable) {
+HBaseRpcController controller = rpcControllerFactory.newController();
+CompletableFuture future = new CompletableFuture<>();
+callable.call(controller, stub, resp -> {
+ if (controller.failed()) {
+future.completeExceptionally(controller.getFailed());
+ } else {
+future.complete(resp);
+ }
+});
+return future;
}
- @VisibleForTesting
- public Set getParsedMasterServers() {
-return Collections.unmodifiableSet(masterServers);
+ private IOException badResponse(String debug) {
+return new IOException(String.format("Invalid result for request %s. Will
be retried", debug));
}
- /**
- * Returns a call back that can be passed along to the non-blocking rpc
call. It is invoked once
- * the rpc finishes and the response is propagated to the passed future.
- * @param future Result future to which the rpc response is propagated.
- * @param isValidResp Checks if the rpc response has a valid result.
- * @param transformResult Transforms the result to a different form as
expected by callers.
- * @param hrc RpcController instance for this rpc.
- * @param debug Debug message passed along to the caller in case of
exceptions.
- * @param RPC result type.
- * @param Transformed type of the result.
- * @return A call back that can be embedded in the non-blocking rpc call.
- */
- private RpcCallback getRpcCallBack(CompletableFuture future,
- Predicate isValidResp, Function transformResult,
HBaseRpcController hrc,
- final String debug) {
-return rpcResult -> {
- if (rpcResult == null) {
-future.completeExceptionally(
-new MasterRegistryFetchException(masterServers, hrc.getFailed()));
-return;
- }
- if (!isValidResp.test(rpcResult)) {
-// Rpc returned ok, but result was malformed.
-future.completeExceptionally(new IOException(
-String.format("Invalid result for request %s. Will be retried",
debug)));
-return;
- }
- future.complete(transformResult.apply(rpcResult));
-};
+ // send requests concurrently to hedgedReadsFanout masters
+ private void groupCall(CompletableFuture future, int
startIndexInclusive,
+Callable callable, Predicate isValidResp, String debug,
+ConcurrentLinkedQueue errors) {
+int endIndexExclusive = Math.min(startIndexInclusive + hedgedReadFanOut,
masterStubs.size());
+AtomicInteger remaining = new AtomicInteger(endIndexExclusive -
startIndexInclusive);
+for (int i = startIndexInclusive; i < endIndexExclusive; i++) {
+ addListener(call(masterStubs.get(i), callable), (r, e) -> {
+// a simple check to skip all the later operations earlier
+if (future.isDone()) {
+ return;
+}
+if (e == null && !isValidResp.test(r)) {
+ e = badResponse(debug);
Review comment:
The message is already contained in the exception?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For
[GitHub] [hbase] markrmiller opened a new pull request #1652: HBASE-24325: TestJMXConnectorServer can fail to start the minicluster…
markrmiller opened a new pull request #1652: URL: https://github.com/apache/hbase/pull/1652 … due to it's port already having been chosen by another test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24260) Add a ClusterManager that issues commands via coprocessor
[
https://issues.apache.org/jira/browse/HBASE-24260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099473#comment-17099473
]
Hudson commented on HBASE-24260:
Results for branch branch-2
[build #2642 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add a ClusterManager that issues commands via coprocessor
> -
>
> Key: HBASE-24260
> URL: https://issues.apache.org/jira/browse/HBASE-24260
> Project: HBase
> Issue Type: New Feature
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> I have a need to run the chaos monkey in an environment where ssh access is
> restricted. I can get most of what I need via {{RESTApiClusterManager}},
> however {{kill -9}} is a critical and unsupported command. I have a
> {{ClusterManager}} that implements {{kill}} via an executor coprocessor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24295) [Chaos Monkey] abstract logging through the class hierarchy
[
https://issues.apache.org/jira/browse/HBASE-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099474#comment-17099474
]
Hudson commented on HBASE-24295:
Results for branch branch-2
[build #2642 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Chaos Monkey] abstract logging through the class hierarchy
> ---
>
> Key: HBASE-24295
> URL: https://issues.apache.org/jira/browse/HBASE-24295
> Project: HBase
> Issue Type: Task
> Components: integration tests
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> Running chaos monkey and watching the logs, it's very difficult to tell what
> actions are actually running. There's lots of shared methods through the
> class hierarchy that extends from {{abstract class Action}}, and each class
> comes with its own {{Logger}}. As a result, the logs have useless stuff like
> {noformat}
> INFO actions.Action: Started regionserver...
> {noformat}
> Add {{protected abstract Logger getLogger()}} to the class's internal
> interface, and have the concrete implementations provide their logger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24246) Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration
[
https://issues.apache.org/jira/browse/HBASE-24246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099471#comment-17099471
]
Hudson commented on HBASE-24246:
Results for branch branch-2
[build #2642 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around
> merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration
> ---
>
> Key: HBASE-24246
> URL: https://issues.apache.org/jira/browse/HBASE-24246
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: Screen Shot 2020-04-23 at 4.42.41 PM.png
>
>
> Bit of a grab bag of small items.
> Improve log messages when bulk merge runs (particularly out of 'hbck2
> fixMeta').
> Change config so we up the merge maximum since current low max is hit easily.
> When its hit, its merge of merge of merges w/ confusing output while
> happening. LOG when we hit max!
> {code}
> private static final String MAX_MERGE_COUNT_KEY =
> "hbase.master.metafixer.max.merge.count";
> private static final int MAX_MERGE_COUNT_DEFAULT = 10;
> {code}
> Fix the 'HBCK Report' page so it only logs encoded region names because if
> region name is large, the UI becomes unreadable.
> Fixup some logging messages. As is, can confuse operator.
> E.g. explain why having a merge qualifier blocks a region being merged up
> {code}
> org.apache.hadoop.hbase.exceptions.MergeRegionException: Skip merging
> [e1ded5983cb00c1a1a45e17414c5914c, 2634c71a10cf4a216b7c1608b7d24e6a,
> e7d40bb7ab50cdeb20118043aedf2cdc, a9ea228464bd8a40b8908cd263e2f6be,
> 544d32880131443eebb2a01691994b0d, a82d462a6edca634e4f8ecff4e8bc55c,
> 03e30d277a1ab977f93c103cb917c7d7, d0d5fb364838b00942448b89b3d802ff,
> 00ba78a9be4b27cf076622f49c45a43e, 338efcfc48928815423d7a6601b9e477], because
> parent [e1ded5983cb00c1a1a45e17414c5914c] has a merge qualifier
> {code}
> Add note to orphans page that you shouldn't try to address orphans until all
> meta issues are fixed (overlaps, holes).
> Add example on how to fix orphans with bulkload to the orphans section in
> hbck report. See HBASE-23928.
> Why the angle brackets in here:
> {code}
> Why angle brackets here? Usually for lists.
> 2020-04-23 15:21:13,471 INFO org.apache.hadoop.hbase.master.HMaster:
> Client=hbase//10.103.230.91 merge regions [e1ded5983cb00c1a1a45e17414c5914c],
> [2634c71a10cf4a216b7c1608b7d24e6a], [e7d40bb7ab50cdeb20118043aedf2cdc],
> [a9ea228464bd8a40b8908cd263e2f6be], [544d32880131443eebb2a01691994b0d],
> [a82d462a6edca634e4f8ecff4e8bc55c], [03e30d277a1ab977f93c103cb917c7d7],
> [d0d5fb364838b00942448b89b3d802ff], [50cbee16764bb09a9887a07d3a01304f],
> [338efcfc48928815423d7a6601b9e477]
> {code}
> These logs make no sense... need region name at least
> {code}
> What is this? Missing region
> Whats it doing?
> 2020-04-23 15:36:19,896 WARN
> org.apache.hadoop.hbase.regionserver.compactions.CompactionProgress:
> totalCompactingKVs=500851692 less than currentCompactedKVs=562814154
> 2020-04-23 15:36:22,953 WARN
> org.apache.hadoop.hbase.regionserver.compactions.CompactionProgress:
> totalCompactingKVs=500851692 less than currentCompactedKVs=564904980
> 2020-04-23 15:36:26,011 WARN
> org.apache.hadoop.hbase.regionserver.compactions.CompactionProgress:
> totalCompactingKVs=500851692 less than currentCompactedKVs=566948397
> 2020-04-23 15:36:29,071 WARN
> org.apache.hadoop.hbase.regionserver.compactions.CompactionProgress:
> totalCompactingKVs=500851692 less than currentCompactedKVs=568943946
> 2020-04-23 15:36:32,153
[jira] [Commented] (HBASE-24302) Add an "ignoreTimestamps" option (defaulted to false) to HashTable/SyncTable tool
[
https://issues.apache.org/jira/browse/HBASE-24302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099470#comment-17099470
]
Hudson commented on HBASE-24302:
Results for branch branch-2
[build #2642 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add an "ignoreTimestamps" option (defaulted to false) to HashTable/SyncTable
> tool
> -
>
> Key: HBASE-24302
> URL: https://issues.apache.org/jira/browse/HBASE-24302
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.2.5
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.2.5
>
>
> Currently, when hashing and comparing values between a source and a target
> table, HashTable/SyncTable always consider cell timestamp values. However,
> cell timestamp values are not always relevant for client applications, so
> these use cases could benefit of a more flexible comparison logic where
> timestamps could be ignored.
> For such scenarios, HashTable/SyncTable could have better performance, since
> cells with only timestamps diverging would not be copied.
> Another case that would benefit from this option is when bulk deletes are
> wrongly applied at target. At the moment, HashTable/SyncTable on it's own is
> not capable of syncing back the clusters, as the source Puts would have an
> older TS than the delete markers in the target. That would require target to
> complete major compaction on the whole table before HashTable/SyncTable could
> be run.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24303) Undo core of parent TestSecureRESTServer change; use fix over in HBASE-24280 instead
[
https://issues.apache.org/jira/browse/HBASE-24303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099472#comment-17099472
]
Hudson commented on HBASE-24303:
Results for branch branch-2
[build #2642 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2642/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Undo core of parent TestSecureRESTServer change; use fix over in HBASE-24280
> instead
>
>
> Key: HBASE-24303
> URL: https://issues.apache.org/jira/browse/HBASE-24303
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> Parent issue puts in a place that makes TestSecureRESTServer pass up on
> jenkins by shoving into the dependency list the jersey1 ServletContainer.
> Root issue was change in how we specified profiles in nightlies; both hadoop3
> and hadoop2 were mistakenly active (HBASE-24280). This issue is about undoing
> the dependency insertion after fix for HBASE-24280 goes in.
> Don't want to revert the parent. It has cleanups that should stay.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24325) TestJMXConnectorServer can fail to start the minicluster due to it's port already having been chosen by another test.
Mark Robert Miller created HBASE-24325: -- Summary: TestJMXConnectorServer can fail to start the minicluster due to it's port already having been chosen by another test. Key: HBASE-24325 URL: https://issues.apache.org/jira/browse/HBASE-24325 Project: HBase Issue Type: Test Components: test Reporter: Mark Robert Miller -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on pull request #1565: HBASE-24235 Wrap creation of NettyHBaseSaslRpcClient with ugi.doAs
Apache9 commented on pull request #1565: URL: https://github.com/apache/hbase/pull/1565#issuecomment-623789505 > The reason why OpenJDK does not need this for creating SaslClient is because it does not call secCtx.initSecContext() in the constructor of class com.sun.security.sasl.gsskerb.GssKrb5Client which is what gets called when Sasl.createSaslClient gets called with GSSAPI mech. Since OpenJDK uses SunSASL and IBM uses IBMSASL, I would imagine they are both different implementations and IBM SASL is possibly calling secCtx.initSecContext(). The stack trace show it is calling com.ibm.security.jgss.GSSManagerImpl.createContext which I would assume is something similar to secCtx.initSecContext() I think Josh is exactly asking why OpenJDK and IBMJDK use different implementations here. It's 'why', not 'how'... Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1485: HBASE-23969 Meta browser should show all `info` columns
Apache-HBase commented on pull request #1485: URL: https://github.com/apache/hbase/pull/1485#issuecomment-623788962 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 35s | master passed | | +1 :green_heart: | compile | 2m 1s | master passed | | +1 :green_heart: | shadedjars | 5m 41s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 25s | hbase-client in master failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 20s | the patch passed | | +1 :green_heart: | compile | 2m 3s | the patch passed | | +1 :green_heart: | javac | 2m 3s | the patch passed | | +1 :green_heart: | shadedjars | 5m 39s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 24s | hbase-client in the patch failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 41s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 14s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 199m 38s | hbase-server in the patch passed. | | | | 233m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1485 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3e75b08881a2 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-client.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/testReport/ | | Max. process+thread count | 2576 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1637: HBASE-24314 Some classes still use log4j logger directly
Apache9 commented on a change in pull request #1637:
URL: https://github.com/apache/hbase/pull/1637#discussion_r419814730
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestProtoBufRpc.java
##
@@ -81,10 +82,10 @@ public void setUp() throws IOException { // Setup server
for both protocols
this.conf = HBaseConfiguration.create();
this.conf.set(RpcServerFactory.CUSTOM_RPC_SERVER_IMPL_CONF_KEY,
rpcServerImpl);
-Logger log = Logger.getLogger("org.apache.hadoop.ipc.HBaseServer");
-log.setLevel(Level.DEBUG);
-log = Logger.getLogger("org.apache.hadoop.ipc.HBaseServer.trace");
-log.setLevel(Level.TRACE);
+org.apache.log4j.Logger.getLogger("org.apache.hadoop.ipc.HBaseServer")
Review comment:
The slf4j logger uses exactly the same name 'Logger', using a full name
is better to let developers know that this is not the slf4j one.
And I plan to forbid log4j imports in enforcer in later patches, then using
the full name is the only way to reference log4j stuff.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1649: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
Apache-HBase commented on pull request #1649: URL: https://github.com/apache/hbase/pull/1649#issuecomment-623785646 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 12m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 37s | branch-1 passed | | +1 :green_heart: | compile | 0m 18s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | compile | 0m 23s | branch-1 passed with JDK v1.7.0_262 | | +1 :green_heart: | checkstyle | 0m 36s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 23s | branch-1 passed with JDK v1.7.0_262 | | +0 :ok: | spotbugs | 1m 18s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 15s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 13s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 17s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 22s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | findbugs | 1m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 34s | hbase-common in the patch passed. | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 47m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1649 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5fb37e9e1e4c 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1649/out/precommit/personality/provided.sh | | git revision | branch-1 / 9da034e | | Default Java | 1.7.0_262 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/2/testReport/ | | Max. process+thread count | 168 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1638: HBASE-24315 Remove hadoop-two-compat.xml in hbase-assembly
Apache-HBase commented on pull request #1638: URL: https://github.com/apache/hbase/pull/1638#issuecomment-623783777 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 10m 57s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 19m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1638 | | Optional Tests | dupname asflicense xml hadoopcheck | | uname | Linux 3db2764153ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Max. process+thread count | 75 (vs. ulimit of 12500) | | modules | C: hbase-assembly . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1638/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1651: HBASE-24323 Avoid two parallel running of GCMulitpleMergedRegionsProc…
Apache-HBase commented on pull request #1651: URL: https://github.com/apache/hbase/pull/1651#issuecomment-623782192 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotbugs | 2m 2s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 16s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 31m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1651 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b7f7b3efb7a2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1651/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24321) Add writable MinVersions and read-only Scan to coproc ScanOptions
[ https://issues.apache.org/jira/browse/HBASE-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099439#comment-17099439 ] Duo Zhang commented on HBASE-24321: --- And for queries from users, just use preScannerOpen? You are free to modify the Scan object as you want... > Add writable MinVersions and read-only Scan to coproc ScanOptions > - > > Key: HBASE-24321 > URL: https://issues.apache.org/jira/browse/HBASE-24321 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 2.3.0 > > > Between HBase 1.x and 2.0, the RegionObserver pre*ScannerOpen coprocessors > were significantly changed so that the coproc implementer no longer has > access to the actual Scanner, just a ScanOptions object that can be changed > in limited ways. This is safer and prevents resource leaks and other bugs. > While ScanOptions provides support for changing TTL, KeepDeletedCells, and > MaxVersions, a fourth column family config parameter, MinVersions, appears to > have been missed. This prevents coproc implementers from changing MinVersions > dynamically. An example of this is PHOENIX-5645, which in the forthcoming > Phoenix 4.16 (based on HBase 1.x) will allow users to configure a moving > window where all versions are kept, and thus point-in-time queries are safe. > This cannot be put in the forthcoming Phoenix 5.1 (based on HBase 2.1 and > 2.2) because of the coproc changes. > Relatedly, preStoreScannerOpen lacks access to the Scan in HBase 2.0 and up. > This prevents coprocs from reading the Scan parameters to check if, for > example, a Scan has set the max time to a point in the past, and thus needs > to override KeepDeletedCells. This can lead to incorrect behavior when doing > point-in-time queries or using transactional engines that treat physically > committed HBase writes as logically uncommitted parts of a transaction. It's > also a correctness problem for PHOENIX-5645. Please note that only > _read-only_ access to the Scan from the store scanner coproc hook is in scope > for this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24321) Add writable MinVersions and read-only Scan to coproc ScanOptions
[ https://issues.apache.org/jira/browse/HBASE-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099438#comment-17099438 ] Duo Zhang commented on HBASE-24321: --- Reading the comment of ScanOptions, the intenion here is that, for complicated scan logic for compaction/flush, you should wrapper the InternalScanner? You can see the WriteHeavyIncrementObserver in hbase-examples module for more details. > Add writable MinVersions and read-only Scan to coproc ScanOptions > - > > Key: HBASE-24321 > URL: https://issues.apache.org/jira/browse/HBASE-24321 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 2.3.0 > > > Between HBase 1.x and 2.0, the RegionObserver pre*ScannerOpen coprocessors > were significantly changed so that the coproc implementer no longer has > access to the actual Scanner, just a ScanOptions object that can be changed > in limited ways. This is safer and prevents resource leaks and other bugs. > While ScanOptions provides support for changing TTL, KeepDeletedCells, and > MaxVersions, a fourth column family config parameter, MinVersions, appears to > have been missed. This prevents coproc implementers from changing MinVersions > dynamically. An example of this is PHOENIX-5645, which in the forthcoming > Phoenix 4.16 (based on HBase 1.x) will allow users to configure a moving > window where all versions are kept, and thus point-in-time queries are safe. > This cannot be put in the forthcoming Phoenix 5.1 (based on HBase 2.1 and > 2.2) because of the coproc changes. > Relatedly, preStoreScannerOpen lacks access to the Scan in HBase 2.0 and up. > This prevents coprocs from reading the Scan parameters to check if, for > example, a Scan has set the max time to a point in the past, and thus needs > to override KeepDeletedCells. This can lead to incorrect behavior when doing > point-in-time queries or using transactional engines that treat physically > committed HBase writes as logically uncommitted parts of a transaction. It's > also a correctness problem for PHOENIX-5645. Please note that only > _read-only_ access to the Scan from the store scanner coproc hook is in scope > for this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1639: HBASE-24305 Removed deprecations in ServerName
Apache-HBase commented on pull request #1639: URL: https://github.com/apache/hbase/pull/1639#issuecomment-623779537 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | compile | 2m 22s | master passed | | +1 :green_heart: | shadedjars | 5m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 48s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 2m 24s | the patch passed | | +1 :green_heart: | javac | 2m 24s | the patch passed | | +1 :green_heart: | shadedjars | 5m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 21s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 0m 21s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 132m 28s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 9m 58s | hbase-backup in the patch passed. | | +1 :green_heart: | unit | 3m 54s | hbase-rest in the patch passed. | | | | 177m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1639 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 239bfe55165f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/testReport/ | | Max. process+thread count | 5014 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-balancer hbase-server hbase-backup hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24322:
Description:
We had a weird test failure due to accidentally running tests with some Java >
8 (exact version I am unsure of), where Unsafe is available, but the method
signatures were different or some methods were removed, leading to this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
... 7 more
{noformat}
We should also check that all methods that will be invoked on Unsafe in
UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
is available (and usable).
was:
We had a weird test failure due to accidentally running tests with some Java >
8 (exact version I am unsure of), where Unsafe is available, but the method
signatures were different, leading to this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
[jira] [Updated] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24322:
Description:
We had a weird test failure due to accidentally running tests with some Java >
8 (exact version I am unsure of), where Unsafe is available, but the method
signatures were different, leading to this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
... 7 more
{noformat}
We should also check that all methods that will be invoked on Unsafe in
UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
is available (and usable).
was:
We had a weird test failure due to accidentally running tests with some Java >
8 (exact version I am unsure of), where Unsafe is available, but the method
signatures were different, leading to this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
[GitHub] [hbase] Apache-HBase commented on pull request #1650: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
Apache-HBase commented on pull request #1650: URL: https://github.com/apache/hbase/pull/1650#issuecomment-623777405 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 29s | master passed | | +1 :green_heart: | spotbugs | 0m 50s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 0m 24s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 0m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 32m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1650 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 12f185c2ae0a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1650: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
Apache-HBase commented on pull request #1650: URL: https://github.com/apache/hbase/pull/1650#issuecomment-623775110 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 53s | master passed | | +1 :green_heart: | compile | 0m 32s | master passed | | +1 :green_heart: | shadedjars | 6m 49s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-common in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | +1 :green_heart: | compile | 0m 29s | the patch passed | | +1 :green_heart: | javac | 0m 29s | the patch passed | | +1 :green_heart: | shadedjars | 6m 16s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 18s | hbase-common in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 57s | hbase-common in the patch passed. | | | | 30m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1650 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 520671ada551 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/testReport/ | | Max. process+thread count | 221 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419801204
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
##
@@ -367,105 +363,6 @@ public void testAsyncEcho() throws IOException {
}
}
- /**
- * Tests the various request fan out values using a simple RPC hedged across
a mix of running and
- * failing servers.
- */
- @Test
- @Ignore
- public void testHedgedAsyncEcho() throws Exception {
-// Hedging is not supported for blocking connection types.
-Assume.assumeFalse(this instanceof TestBlockingIPC);
-List rpcServers = new ArrayList<>();
-List addresses = new ArrayList<>();
-// Create a mix of running and failing servers.
-final int numRunningServers = 5;
-final int numFailingServers = 3;
-final int numServers = numRunningServers + numFailingServers;
-for (int i = 0; i < numRunningServers; i++) {
- RpcServer rpcServer = createRpcServer(null, "testRpcServer" + i,
- Lists.newArrayList(new RpcServer.BlockingServiceAndInterface(
- SERVICE, null)), new InetSocketAddress("localhost", 0), CONF,
- new FifoRpcScheduler(CONF, 1));
- rpcServer.start();
- addresses.add(rpcServer.getListenerAddress());
- rpcServers.add(rpcServer);
-}
-for (int i = 0; i < numFailingServers; i++) {
- RpcServer rpcServer = createTestFailingRpcServer(null,
"testFailingRpcServer" + i,
- Lists.newArrayList(new RpcServer.BlockingServiceAndInterface(
- SERVICE, null)), new InetSocketAddress("localhost", 0), CONF,
- new FifoRpcScheduler(CONF, 1));
- rpcServer.start();
- addresses.add(rpcServer.getListenerAddress());
- rpcServers.add(rpcServer);
-}
-Configuration conf = HBaseConfiguration.create();
-try (AbstractRpcClient client = createRpcClient(conf)) {
- // Try out various fan out values starting from 1 -> numServers.
- for (int reqFanOut = 1; reqFanOut <= numServers; reqFanOut++) {
-// Update the client's underlying conf, should be ok for the test.
-LOG.debug("Testing with request fan out: " + reqFanOut);
-conf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, reqFanOut);
-Interface stub = newStub(client, addresses);
-BlockingRpcCallback done = new
BlockingRpcCallback<>();
-stub.echo(new HBaseRpcControllerImpl(),
-EchoRequestProto.newBuilder().setMessage("hello").build(), done);
-TestProtos.EchoResponseProto responseProto = done.get();
-assertNotNull(responseProto);
-assertEquals("hello", responseProto.getMessage());
-LOG.debug("Ended test with request fan out: " + reqFanOut);
- }
-} finally {
- for (RpcServer rpcServer: rpcServers) {
-rpcServer.stop();
- }
-}
- }
-
- @Test
- public void testHedgedAsyncTimeouts() throws Exception {
-// Hedging is not supported for blocking connection types.
-Assume.assumeFalse(this instanceof TestBlockingIPC);
-List rpcServers = new ArrayList<>();
-List addresses = new ArrayList<>();
-final int numServers = 3;
-for (int i = 0; i < numServers; i++) {
- RpcServer rpcServer = createRpcServer(null, "testTimeoutRpcServer" + i,
- Lists.newArrayList(new RpcServer.BlockingServiceAndInterface(
- SERVICE, null)), new InetSocketAddress("localhost", 0), CONF,
- new FifoRpcScheduler(CONF, 1));
- rpcServer.start();
- addresses.add(rpcServer.getListenerAddress());
- rpcServers.add(rpcServer);
-}
-Configuration conf = HBaseConfiguration.create();
-int timeout = 100;
-int pauseTime = 1000;
-try (AbstractRpcClient client = createRpcClient(conf)) {
- // Try out various fan out values starting from 1 -> numServers.
- for (int reqFanOut = 1; reqFanOut <= numServers; reqFanOut++) {
-// Update the client's underlying conf, should be ok for the test.
-LOG.debug("Testing with request fan out: " + reqFanOut);
-conf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, reqFanOut);
-Interface stub = newStub(client, addresses);
-HBaseRpcController pcrc = new HBaseRpcControllerImpl();
-pcrc.setCallTimeout(timeout);
-BlockingRpcCallback callback = new
BlockingRpcCallback<>();
-stub.pause(pcrc,
PauseRequestProto.newBuilder().setMs(pauseTime).build(), callback);
-assertNull(callback.get());
-// Make sure the controller has the right exception propagated.
-assertTrue(pcrc.getFailed() instanceof CallTimeoutException);
-LOG.debug("Ended test with request fan out: " + reqFanOut);
- }
-} finally {
- for (RpcServer rpcServer: rpcServers) {
-rpcServer.stop();
- }
-}
- }
-
-
Review comment:
We do not have hedge reads support for rpc so just removed them. Let me
see if we can directly
[GitHub] [hbase] Apache-HBase commented on pull request #1650: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
Apache-HBase commented on pull request #1650: URL: https://github.com/apache/hbase/pull/1650#issuecomment-623773558 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 0m 25s | master passed | | +1 :green_heart: | shadedjars | 5m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 27s | hbase-common in the patch passed. | | | | 24m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1650 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0ad1e8ebb428 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0632e9893f | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/testReport/ | | Max. process+thread count | 228 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1650/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419800630
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -117,105 +147,97 @@ public static String getMasterAddr(Configuration conf)
throws UnknownHostExcepti
return String.format("%s:%d", hostname, port);
}
- /**
- * @return Stub needed to make RPC using a hedged channel to the master end
points.
- */
- private ClientMetaService.Interface getMasterStub() throws IOException {
-return ClientMetaService.newStub(
-rpcClient.createHedgedRpcChannel(masterServers, User.getCurrent(),
rpcTimeoutMs));
+ @FunctionalInterface
+ private interface Callable {
+void call(HBaseRpcController controller, ClientMetaService.Interface stub,
RpcCallback done);
}
- /**
- * Parses the list of master addresses from the provided configuration.
Supported format is
- * comma separated host[:port] values. If no port number if specified,
default master port is
- * assumed.
- * @param conf Configuration to parse from.
- */
- private void parseMasterAddrs(Configuration conf) throws
UnknownHostException {
-String configuredMasters = getMasterAddr(conf);
-for (String masterAddr:
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
- HostAndPort masterHostPort =
-
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
- masterServers.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
-}
-Preconditions.checkArgument(!masterServers.isEmpty(), "At least one master
address is needed");
+ private CompletableFuture
call(ClientMetaService.Interface stub,
+Callable callable) {
+HBaseRpcController controller = rpcControllerFactory.newController();
+CompletableFuture future = new CompletableFuture<>();
+callable.call(controller, stub, resp -> {
+ if (controller.failed()) {
+future.completeExceptionally(controller.getFailed());
+ } else {
+future.complete(resp);
+ }
+});
+return future;
}
- @VisibleForTesting
- public Set getParsedMasterServers() {
-return Collections.unmodifiableSet(masterServers);
+ private IOException badResponse(String debug) {
+return new IOException(String.format("Invalid result for request %s. Will
be retried", debug));
}
- /**
- * Returns a call back that can be passed along to the non-blocking rpc
call. It is invoked once
- * the rpc finishes and the response is propagated to the passed future.
- * @param future Result future to which the rpc response is propagated.
- * @param isValidResp Checks if the rpc response has a valid result.
- * @param transformResult Transforms the result to a different form as
expected by callers.
- * @param hrc RpcController instance for this rpc.
- * @param debug Debug message passed along to the caller in case of
exceptions.
- * @param RPC result type.
- * @param Transformed type of the result.
- * @return A call back that can be embedded in the non-blocking rpc call.
- */
- private RpcCallback getRpcCallBack(CompletableFuture future,
- Predicate isValidResp, Function transformResult,
HBaseRpcController hrc,
- final String debug) {
-return rpcResult -> {
- if (rpcResult == null) {
-future.completeExceptionally(
-new MasterRegistryFetchException(masterServers, hrc.getFailed()));
-return;
- }
- if (!isValidResp.test(rpcResult)) {
-// Rpc returned ok, but result was malformed.
-future.completeExceptionally(new IOException(
-String.format("Invalid result for request %s. Will be retried",
debug)));
-return;
- }
- future.complete(transformResult.apply(rpcResult));
-};
+ // send requests concurrently to hedgedReadsFanout masters
+ private void groupCall(CompletableFuture future, int
startIndexInclusive,
+Callable callable, Predicate isValidResp, String debug,
+ConcurrentLinkedQueue errors) {
+int endIndexExclusive = Math.min(startIndexInclusive + hedgedReadFanOut,
masterStubs.size());
+AtomicInteger remaining = new AtomicInteger(endIndexExclusive -
startIndexInclusive);
+for (int i = startIndexInclusive; i < endIndexExclusive; i++) {
+ addListener(call(masterStubs.get(i), callable), (r, e) -> {
+// a simple check to skip all the later operations earlier
+if (future.isDone()) {
+ return;
+}
+if (e == null && !isValidResp.test(r)) {
+ e = badResponse(debug);
+}
+if (e != null) {
+ // make sure when remaining reaches 0 we have all exceptions in the
errors queue
+ errors.add(e);
+ if (remaining.decrementAndGet() == 0) {
+if (endIndexExclusive == masterStubs.size()) {
+ // we are done, complete the
[GitHub] [hbase] Apache-HBase commented on pull request #1485: HBASE-23969 Meta browser should show all `info` columns
Apache-HBase commented on pull request #1485: URL: https://github.com/apache/hbase/pull/1485#issuecomment-623773036 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 45s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 1m 49s | master passed | | +1 :green_heart: | shadedjars | 5m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 1m 50s | the patch passed | | +1 :green_heart: | javac | 1m 50s | the patch passed | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 19s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 3s | hbase-client in the patch passed. | | -1 :x: | unit | 139m 26s | hbase-server in the patch failed. | | | | 172m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1485 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7ac2ef63b10d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/testReport/ | | Max. process+thread count | 4252 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #1651: HBASE-24323 Avoid two parallel running of GCMulitpleMergedRegionsProc…
huaxiangsun commented on pull request #1651: URL: https://github.com/apache/hbase/pull/1651#issuecomment-623772927 Hi @saintstack a patch for acquire lock on child regions, unittest will come within HBASE-24250. I applied HBASE-24250 and added a unittest, with this patch, it passed. Without this patch, it fails. Please provide your feedback if this is the correct way to acquire locks for a region, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #1593: HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry …
Apache9 commented on a change in pull request #1593:
URL: https://github.com/apache/hbase/pull/1593#discussion_r419799798
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/MasterRegistry.java
##
@@ -61,53 +65,79 @@
/**
* Master based registry implementation. Makes RPCs to the configured master
addresses from config
* {@value org.apache.hadoop.hbase.HConstants#MASTER_ADDRS_KEY}.
- *
+ *
* It supports hedged reads, which can be enabled by setting
* {@value
org.apache.hadoop.hbase.HConstants#MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY} to
True. Fan
* out the requests batch is controlled by
* {@value
org.apache.hadoop.hbase.HConstants#HBASE_RPCS_HEDGED_REQS_FANOUT_KEY}.
- *
+ *
* TODO: Handle changes to the configuration dynamically without having to
restart the client.
*/
@InterfaceAudience.Private
public class MasterRegistry implements ConnectionRegistry {
+
+ /** Configuration key that controls the fan out of requests **/
+ public static final String MASTER_REGISTRY_HEDGED_REQS_FANOUT_KEY =
+"hbase.client.master_registry.hedged.fanout";
+
+ /** Default value for the fan out of hedged requests. **/
+ public static final int MASTER_REGISTRY_HEDGED_REQS_FANOUT_DEFAULT = 2;
+
private static final String MASTER_ADDRS_CONF_SEPARATOR = ",";
+ private final int hedgedReadFanOut;
+
// Configured list of masters to probe the meta information from.
- private final Set masterServers;
+ private final Set masterAddrs;
+
+ private final List masterStubs;
// RPC client used to talk to the masters.
private final RpcClient rpcClient;
private final RpcControllerFactory rpcControllerFactory;
- private final int rpcTimeoutMs;
-
- MasterRegistry(Configuration conf) throws UnknownHostException {
-boolean hedgedReadsEnabled =
conf.getBoolean(MASTER_REGISTRY_ENABLE_HEDGED_READS_KEY,
-MASTER_REGISTRY_ENABLE_HEDGED_READS_DEFAULT);
-Configuration finalConf;
-if (!hedgedReadsEnabled) {
- // If hedged reads are disabled, it is equivalent to setting a fan out
of 1. We make a copy of
- // the configuration so that other places reusing this reference is not
affected.
- finalConf = new Configuration(conf);
- finalConf.setInt(HConstants.HBASE_RPCS_HEDGED_REQS_FANOUT_KEY, 1);
-} else {
- finalConf = conf;
+
+ /**
+ * Parses the list of master addresses from the provided configuration.
Supported format is comma
+ * separated host[:port] values. If no port number if specified, default
master port is assumed.
+ * @param conf Configuration to parse from.
+ */
+ private static Set parseMasterAddrs(Configuration conf) throws
UnknownHostException {
+Set masterAddrs = new HashSet<>();
+String configuredMasters = getMasterAddr(conf);
+for (String masterAddr :
configuredMasters.split(MASTER_ADDRS_CONF_SEPARATOR)) {
+ HostAndPort masterHostPort =
+
HostAndPort.fromString(masterAddr.trim()).withDefaultPort(HConstants.DEFAULT_MASTER_PORT);
+ masterAddrs.add(ServerName.valueOf(masterHostPort.toString(),
ServerName.NON_STARTCODE));
}
-if (conf.get(MASTER_ADDRS_KEY) != null) {
- finalConf.set(MASTER_ADDRS_KEY, conf.get(MASTER_ADDRS_KEY));
+Preconditions.checkArgument(!masterAddrs.isEmpty(), "At least one master
address is needed");
+return masterAddrs;
+ }
+
+ MasterRegistry(Configuration conf) throws IOException {
+this.hedgedReadFanOut = conf.getInt(MASTER_REGISTRY_HEDGED_REQS_FANOUT_KEY,
+ MASTER_REGISTRY_HEDGED_REQS_FANOUT_DEFAULT);
+int rpcTimeoutMs = (int) Math.min(Integer.MAX_VALUE,
+ conf.getLong(HConstants.HBASE_RPC_TIMEOUT_KEY,
HConstants.DEFAULT_HBASE_RPC_TIMEOUT));
+// XXX: we pass cluster id as null here since we do not have a cluster id
yet, we have to fetch
Review comment:
Just because we do not have a test where MasterRegistry is enabled and
we use cluster id to select authentication. Most tests in HBase do not need
authentication.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1649: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
Apache-HBase commented on pull request #1649: URL: https://github.com/apache/hbase/pull/1649#issuecomment-623772491 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 0s | branch-1 passed | | +1 :green_heart: | compile | 0m 21s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | compile | 0m 24s | branch-1 passed with JDK v1.7.0_262 | | +1 :green_heart: | checkstyle | 0m 36s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | branch-1 passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 25s | branch-1 passed with JDK v1.7.0_262 | | +0 :ok: | spotbugs | 1m 20s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 17s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 22s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | checkstyle | 0m 36s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed with JDK v1.8.0_252 | | +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK v1.7.0_262 | | +1 :green_heart: | findbugs | 1m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 2s | hbase-common in the patch passed. | | +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. | | | | 40m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1649 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6bb56b8b7d5e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1649/out/precommit/personality/provided.sh | | git revision | branch-1 / 9da034e | | Default Java | 1.7.0_262 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_252 /usr/lib/jvm/zulu-7-amd64:1.7.0_262 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/1/testReport/ | | Max. process+thread count | 171 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1649/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun opened a new pull request #1651: HBASE-24323 Avoid two parallel running of GCMulitpleMergedRegionsProc…
huaxiangsun opened a new pull request #1651: URL: https://github.com/apache/hbase/pull/1651 …edure over the same region. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24323) Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same region.
[
https://issues.apache.org/jira/browse/HBASE-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099430#comment-17099430
]
Huaxiang Sun commented on HBASE-24323:
--
{code:java}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
index 285891eb2b..364975a51a 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
@@ -63,6 +63,23 @@ public class GCMultipleMergedRegionsProcedure extends
super();
}
+ @Override protected boolean holdLock(MasterProcedureEnv env) {
+ return true;
+ }
+
+ @Override
+ protected LockState acquireLock(final MasterProcedureEnv env) {
+ if (env.getProcedureScheduler().waitRegion(this, mergedChild)) {
+ return LockState.LOCK_EVENT_WAIT;
+ }
+ return LockState.LOCK_ACQUIRED;
+ }
+
+ @Override
+ protected void releaseLock(final MasterProcedureEnv env) {
+ env.getProcedureScheduler().wakeRegion(this, mergedChild);
+ }
+ {code}
> Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same
> region.
>
>
> Key: HBASE-24323
> URL: https://issues.apache.org/jira/browse/HBASE-24323
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0
>Reporter: Huaxiang Sun
>Priority: Major
>
> As [~stack] suggested in one of PRs, "This method is called from
> GCMergedRegionsProcedure and GCMulitpleMergedRegionsProcedure. I was
> wondering if we took a lock on the Region being Merged would it stop two
> overlapping GCs happening?"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1648: HBASE-8458 Support for batch version of checkAndMutate()
Apache-HBase commented on pull request #1648: URL: https://github.com/apache/hbase/pull/1648#issuecomment-623769005 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | checkstyle | 2m 54s | master passed | | +1 :green_heart: | spotbugs | 8m 35s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | -0 :warning: | checkstyle | 0m 31s | hbase-client: The patch generated 1 new + 130 unchanged - 5 fixed = 131 total (was 135) | | -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 2 new + 67 unchanged - 0 fixed = 69 total (was 67) | | -0 :warning: | checkstyle | 0m 46s | hbase-thrift: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 0m 14s | hbase-rest: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 7s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 47s | the patch passed | | +1 :green_heart: | spotbugs | 9m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. | | | | 57m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1648 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle cc hbaseprotoc prototool | | uname | Linux 3713ee748c45 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-thrift.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-rest.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1648/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-24324) NPE from /procedures.jsp on backup master
Nick Dimiduk created HBASE-24324:
Summary: NPE from /procedures.jsp on backup master
Key: HBASE-24324
URL: https://issues.apache.org/jira/browse/HBASE-24324
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 2.3.0
Reporter: Nick Dimiduk
When going to {{/procedures.jsp}} on a backup master (i.e., a user hits refresh
on a window they have open, meanwhile, the active master has flipped over), we
throw an NPE back to the user. Instead, we should do practically anything else.
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.generated.master.procedures_jsp._jspService(procedures_jsp.java:63)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
at
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at
org.apache.hadoop.hbase.http.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:66)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at
org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:52)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1491)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell opened a new pull request #1650: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
apurtell opened a new pull request #1650: URL: https://github.com/apache/hbase/pull/1650 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24322:
Description:
We had a weird test failure due to accidentally running tests with some Java >
8 (exact version I am unsure of), where Unsafe is available, but the method
signatures were different, leading to this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
... 7 more
{noformat}
We should also check that all methods that will be invoked on Unsafe in
UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
is available (and usable).
was:
We had a weird test failure due to accidentally running tests with Java 11,
where Unsafe is available, but the method signatures were different, leading to
this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
[GitHub] [hbase] Apache-HBase commented on pull request #1584: HBASE-24256 When fixOverlap hits the max region limit, it is possible…
Apache-HBase commented on pull request #1584: URL: https://github.com/apache/hbase/pull/1584#issuecomment-623765563 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | checkstyle | 1m 24s | master passed | | +1 :green_heart: | spotbugs | 2m 32s | master passed | ||| _ Patch Compile Tests _ | | -1 :x: | mvninstall | 4m 9s | root in the patch failed. | | -0 :warning: | checkstyle | 1m 25s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 32s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 44m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1584 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 25491c1a0f43 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 85 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1584/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24323) Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same region.
[ https://issues.apache.org/jira/browse/HBASE-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-24323: - Affects Version/s: 2.4.0 2.3.0 3.0.0-alpha-1 > Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same > region. > > > Key: HBASE-24323 > URL: https://issues.apache.org/jira/browse/HBASE-24323 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0 >Reporter: Huaxiang Sun >Priority: Major > > As [~stack] suggested in one of PRs, "This method is called from > GCMergedRegionsProcedure and GCMulitpleMergedRegionsProcedure. I was > wondering if we took a lock on the Region being Merged would it stop two > overlapping GCs happening?" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24323) Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same region.
[ https://issues.apache.org/jira/browse/HBASE-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099420#comment-17099420 ] Huaxiang Sun commented on HBASE-24323: -- I am working on a patch to address it. > Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same > region. > > > Key: HBASE-24323 > URL: https://issues.apache.org/jira/browse/HBASE-24323 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0 >Reporter: Huaxiang Sun >Priority: Major > > As [~stack] suggested in one of PRs, "This method is called from > GCMergedRegionsProcedure and GCMulitpleMergedRegionsProcedure. I was > wondering if we took a lock on the Region being Merged would it stop two > overlapping GCs happening?" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24323) Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same region.
Huaxiang Sun created HBASE-24323: Summary: Avoid two parallel running of GCMulitpleMergedRegionsProcedure over the same region. Key: HBASE-24323 URL: https://issues.apache.org/jira/browse/HBASE-24323 Project: HBase Issue Type: Bug Reporter: Huaxiang Sun As [~stack] suggested in one of PRs, "This method is called from GCMergedRegionsProcedure and GCMulitpleMergedRegionsProcedure. I was wondering if we took a lock on the Region being Merged would it stop two overlapping GCs happening?" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24250) CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
[
https://issues.apache.org/jira/browse/HBASE-24250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099419#comment-17099419
]
Huaxiang Sun commented on HBASE-24250:
--
Hi [~niuyulin], I added a unitest case and applied your patch. It still failed.
The reason is that two procedures are being executed at the same time.
{code:java}
index 86971431dc..a50006a5f4 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
@@ -23,6 +23,7 @@ import static org.junit.Assert.assertTrue;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
+import java.util.Map;
import java.util.function.BooleanSupplier;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.HBaseTestingUtility;
@@ -31,6 +32,10 @@ import org.apache.hadoop.hbase.MetaTableAccessor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.client.RegionInfoBuilder;
+import org.apache.hadoop.hbase.client.Result;
+import
org.apache.hadoop.hbase.master.assignment.GCMultipleMergedRegionsProcedure;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.procedure2.ProcedureExecutor;
import org.apache.hadoop.hbase.testclassification.LargeTests;
import org.apache.hadoop.hbase.testclassification.MasterTests;
import org.apache.hadoop.hbase.util.Threads;
@@ -164,15 +169,34 @@ public class TestMetaFixer {
assertEquals(1, MetaFixer.calculateMerges(10,
report.getOverlaps()).size());
MetaFixer fixer = new MetaFixer(services);
fixer.fixOverlaps(report);
+ CatalogJanitor cj = services.getCatalogJanitor();
await(10, () -> {
try {
- services.getCatalogJanitor().scan();
- final CatalogJanitor.Report postReport =
services.getCatalogJanitor().getLastReport();
- return postReport.isEmpty();
+ if (cj.scan() > 0) {
+ Map mergedRegions =
cj.getLastReport().mergedRegions;
+ for (Map.Entry e : mergedRegions.entrySet()) {
+ List parents =
MetaTableAccessor.getMergeRegions(e.getValue().rawCells());
+ if (parents != null) {
+ ProcedureExecutor pe =
services.getMasterProcedureExecutor();
+ pe.submitProcedure(new
GCMultipleMergedRegionsProcedure(pe.getEnvironment(),
+ e.getKey(), parents));
+ }
+ }
+ return true;
+ }
+ return false;
} catch (Exception e) {
throw new RuntimeException(e);
}
});
+
+ await(10, () -> {
+ return
services.getMasterProcedureExecutor().getActiveProcIds().isEmpty();
+ });
+
+ services.getCatalogJanitor().scan();
+ final CatalogJanitor.Report postReport =
services.getCatalogJanitor().getLastReport();
+ assertTrue(postReport.isEmpty());
}
/**
iMac-Pro:hbase hsun$ {code}
> CatalogJanitor resubmits GCMultipleMergedRegionsProcedure for the same region
> -
>
> Key: HBASE-24250
> URL: https://issues.apache.org/jira/browse/HBASE-24250
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.2.4
> Environment: hdfs 3.1.3 with erasure coding
> hbase 2.2.4
>Reporter: Andrey Elenskiy
>Assignee: niuyulin
>Priority: Major
>
> If a lot of regions were merged (due to change of region sizes, for example),
> there can be a long backlog of procedures to clean up the merged regions. If
> going through this backlog is slower than the CatalogJanitor's scan interval,
> it will end resubmitting GCMultipleMergedRegionsProcedure for the same
> regions over and over again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24322:
Assignee: Andrew Kyle Purtell
Status: Patch Available (was: Open)
> UnsafeAvailChecker should also check that required methods are available
>
>
> Key: HBASE-24322
> URL: https://issues.apache.org/jira/browse/HBASE-24322
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> We had a weird test failure due to accidentally running tests with Java 11,
> where Unsafe is available, but the method signatures were different, leading
> to this:
> {noformat}
> 020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
> Caused by: java.lang.NoSuchMethodError: 'void
> sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
> at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
> at
> org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
> at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
> ... 7 more
> {noformat}
> We should also check that all methods that will be invoked on Unsafe in
> UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
> is available (and usable).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell opened a new pull request #1649: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
apurtell opened a new pull request #1649: URL: https://github.com/apache/hbase/pull/1649 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #1649: HBASE-24322 UnsafeAvailChecker should also check that required methods are available
apurtell commented on pull request #1649:
URL: https://github.com/apache/hbase/pull/1649#issuecomment-623759444
Unit tests pass with change, and the warnings are not tripped on Java 8 (as
expected).
$ find . -name '*output*.txt' -exec fgrep -Hn Unsafe {} \; | grep missing
$
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
[
https://issues.apache.org/jira/browse/HBASE-24322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24322:
Fix Version/s: 2.4.0
1.7.0
3.0.0-alpha-1
> UnsafeAvailChecker should also check that required methods are available
>
>
> Key: HBASE-24322
> URL: https://issues.apache.org/jira/browse/HBASE-24322
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
>
> We had a weird test failure due to accidentally running tests with Java 11,
> where Unsafe is available, but the method signatures were different, leading
> to this:
> {noformat}
> 020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
> Caused by: java.lang.NoSuchMethodError: 'void
> sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
> at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
> at
> org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
> at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
> ... 7 more
> {noformat}
> We should also check that all methods that will be invoked on Unsafe in
> UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
> is available (and usable).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24322) UnsafeAvailChecker should also check that required methods are available
Andrew Kyle Purtell created HBASE-24322:
---
Summary: UnsafeAvailChecker should also check that required
methods are available
Key: HBASE-24322
URL: https://issues.apache.org/jira/browse/HBASE-24322
Project: HBase
Issue Type: Bug
Reporter: Andrew Kyle Purtell
We had a weird test failure due to accidentally running tests with Java 11,
where Unsafe is available, but the method signatures were different, leading to
this:
{noformat}
020-05-02 14:57:15,145 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:143)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:237)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:163)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2911)
Caused by: java.lang.NoSuchMethodError: 'void
sun.misc.Unsafe.putInt(java.lang.Object, int, int)'
at org.apache.hadoop.hbase.util.UnsafeAccess.putInt(UnsafeAccess.java:233)
at
org.apache.hadoop.hbase.util.Bytes$ConverterHolder$UnsafeConverter.putInt(Bytes.java:1499)
at org.apache.hadoop.hbase.util.Bytes.putInt(Bytes.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.appendMetaData(RecoverableZooKeeper.java:850)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:640)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:1027)
at
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.setMasterAddress(MasterAddressTracker.java:211)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2095)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:520)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:315)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:138)
... 7 more
{noformat}
We should also check that all methods that will be invoked on Unsafe in
UnsafeAccess.java are available when deciding in UnsafeAvailChecker if Unsafe
is available (and usable).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099411#comment-17099411 ] Mingliang Liu commented on HBASE-23707: --- Cool, thanks [~ndimiduk] I'll try that and review the IntelliJ article shortly. > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.0, 1.6.0 > > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] busbey commented on pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
busbey commented on pull request #1643: URL: https://github.com/apache/hbase/pull/1643#issuecomment-623755088 I have been working through trying to use the release scripts to roll 3.0.0-alpha-1. unless it is pressing I would like to incorporate this PR into me doing that as a review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #1643: HBASE-24318 Create-release scripts fixes and enhancements
saintstack commented on a change in pull request #1643:
URL: https://github.com/apache/hbase/pull/1643#discussion_r419775446
##
File path: dev-support/create-release/do-release-docker.sh
##
@@ -64,24 +65,26 @@ This script runs the release scripts inside a docker image.
Options:
-d [path]required. working directory. output will be written to "output"
in here.
- -n dry run mode. Checks and local builds, but does not upload
anything.
+ -f "force" -- actually publish this release. Unless you specify
'-f', it will
+ default to dry run mode, which checks and does local builds,
but does not upload anything.
Review comment:
Highlight this? Add to the release note so RMs notice.
##
File path: dev-support/create-release/do-release.sh
##
@@ -66,26 +82,34 @@ function should_build {
fi
}
-if should_build "tag" && [ $SKIP_TAG = 0 ]; then
+if should_build "tag" && [ "$SKIP_TAG" = 0 ]; then
run_silent "Creating release tag $RELEASE_TAG..." "tag.log" \
-"$SELF/release-tag.sh"
- echo "It may take some time for the tag to be synchronized to github."
- echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
- read
+"$SELF/release-build.sh" tag
+ if is_dry_run; then
+export TAG_SAME_DRY_RUN="true";
+ else
+echo "It may take some time for the tag to be synchronized to github."
+echo "Press enter when you've verified that the new tag ($RELEASE_TAG) is
available."
+read -r
+ fi
else
echo "Skipping tag creation for $RELEASE_TAG."
fi
-if should_build "build"; then
- run_silent "Building ${PROJECT}..." "build.log" \
-"$SELF/release-build.sh" build
+if should_build "publish-dist"; then
+ run_silent "Publishing distribution packages (tarballs)" "publish-dist.log" \
+"$SELF/release-build.sh" publish-dist
else
- echo "Skipping build step."
+ echo "Skipping publish-dist step."
Review comment:
publish-dist is new step? If so, put that in release note too... as
change.
##
File path: dev-support/create-release/release-build.sh
##
@@ -20,114 +20,180 @@
trap cleanup EXIT
# Source in utils.
-SELF=$(cd $(dirname $0) && pwd)
+SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=SCRIPTDIR/release-util.sh
. "$SELF/release-util.sh"
# Print usage and exit.
function exit_with_usage {
- cat << EOF
-Usage: release-build.sh
-Creates build deliverables from a tag/commit.
-Arguments:
- build Create binary packages and commit to
dist.apache.org/repos/dist/dev/hbase/
- publish-snapshot Publish snapshot release to Apache snapshots
- publish-release Publish a release to Apache release repo
-
-All other inputs are environment variables:
- GIT_REF - Release tag or commit to build from
- PACKAGE_VERSION - Release identifier in top level package directory (e.g.
2.1.2RC1)
- VERSION - (optional) Version of project being built (e.g. 2.1.2)
- ASF_USERNAME - Username of ASF committer account
- ASF_PASSWORD - Password of ASF committer account
- GPG_KEY - GPG key used to sign release artifacts
- GPG_PASSPHRASE - Passphrase for GPG key
- PROJECT - The project to build. No default.
-
-Set REPO environment to full path to repo to use
-to avoid re-downloading dependencies on each run.
+ cat <<'EOF'
+Usage: release-build.sh
+Creates release deliverables from a tag or commit.
+Argument: one of 'tag', 'publish-dist', 'publish-snapshot', or
'publish-release'
+ tag Prepares for release on specified git branch: Set release
version,
+update CHANGES and RELEASENOTES, create release tag,
+increment version for ongoing dev, and publish to Apache
git repo.
+ publish-dist Build and publish distribution packages (tarballs) to
Apache dist repo
+ publish-snapshot Build and publish maven artifacts snapshot release to
Apache snapshots repo
+ publish-release Build and publish maven artifacts release to Apache
release repo, and
Review comment:
Is this what I do when I want to run a release candidate? Release
candidate goes into staging repo first.
##
File path: dev-support/create-release/do-release-docker.sh
##
@@ -64,24 +65,26 @@ This script runs the release scripts inside a docker image.
Options:
-d [path]required. working directory. output will be written to "output"
in here.
- -n dry run mode. Checks and local builds, but does not upload
anything.
+ -f "force" -- actually publish this release. Unless you specify
'-f', it will
+ default to dry run mode, which checks and does local builds,
but does not upload anything.
-t [tag] tag for the hbase-rm docker image to use for building (default:
"latest").
-j [path]path to local JDK installation to use building. By default the
script will
use openjdk8 installed in the docker image.
- -p [project] project to
[jira] [Comment Edited] (HBASE-8458) Support for batch version of checkAndMutate()
[
https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099405#comment-17099405
]
Toshihiro Suzuki edited comment on HBASE-8458 at 5/4/20, 11:01 PM:
---
Could someone please review the following PR? Actually this feature is
important for our use case.
[https://github.com/apache/hbase/pull/1648]
The highlights of the changes are as follows:
- Introduced CheckAndMutate class that's used to perform CheckAndMutate
operations. The following is the JavaDoc for this class:
{code:java}
* Used to perform CheckAndMutate operations. Currently {@link Put}, {@link
Delete}
* and {@link RowMutations} are supported.
*
* This has a fluent style API to instantiate it, the code is like:
*
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row equals to the specified value
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifEquals(family, qualifier, value)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row doesn't exist
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifNotExists(family, qualifier)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the row
matches the filter
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifMatches(filter)
* .action(delete);
*
*
{code}
- Added new checkAndMutate APIs to the Table and AsyncTable interfaces.
-- The APIs for the Table interface:
{code:java}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return boolean that represents the result for the CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean checkAndMutate(CheckAndMutate checkAndMutate) throws
IOException {
return checkAndMutate(Collections.singletonList(checkAndMutate))[0];
}
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A array of boolean that represents the result for each
CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean[] checkAndMutate(List checkAndMutates) throws
IOException {
throw new NotImplementedException("Add an implementation!");
}
{code}
-- The APIs for the AsyncTable interface:
{code:java}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return A {@link CompletableFuture}s that represent the result for the
CheckAndMutate.
*/
CompletableFuture checkAndMutate(CheckAndMutate checkAndMutate);
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A list of {@link CompletableFuture}s that represent the result for
each
* CheckAndMutate.
*/
List> checkAndMutate(List
checkAndMutates);
/**
* A simple version of batch checkAndMutate. It will fail if there are any
failures.
*
* @param checkAndMutates The list of rows to apply.
* @return A {@link CompletableFuture} that wrapper the result boolean list.
*/
default CompletableFuture> checkAndMutateAll(
List checkAndMutates) {
return allOf(checkAndMutate(checkAndMutates));
}
{code}
- Deprecated the old checkAndMutate APIs of the Table and AsyncTable
interfaces.
- This has Protocol Buffers level changes. I moved MultiRequest#condition to
RegionAction and MultiResponse#processed to RegionActionResult. This is already
discussed in the Jira:
https://issues.apache.org/jira/browse/HBASE-8458?focusedCommentId=17064155=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17064155
Thanks.
was (Author: brfrn169):
Could someone please review the following PR? Actually this feature is
important for our usecase.
https://github.com/apache/hbase/pull/1648
The highlights of the changes are as follows:
- Introduced CheckAndMutate class that's used to perform CheckAndMutate
operations. The following is the JavaDoc for this class:
{code}
* Used to perform CheckAndMutate operations. Currently {@link Put}, {@link
Delete}
* and {@link RowMutations} are supported.
*
* This has a fluent style API to instantiate it, the code is like:
*
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row equals to the specified value
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifEquals(family, qualifier, value)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the column
[GitHub] [hbase] joshelser commented on pull request #746: HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the clas…
joshelser commented on pull request #746: URL: https://github.com/apache/hbase/pull/746#issuecomment-623752040 I think Duo's request https://github.com/apache/hbase/pull/746/files#r341146952 still wasn't addressed (just use `instanceof`). I resolved the other outstanding request to do something different than catch the `UnsupportedOperationException` (extends RuntimeException) because I can't see anything else we can effectively do. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1625: HBASE-24284 [h3/jdk11] REST server won't start
Apache-HBase commented on pull request #1625: URL: https://github.com/apache/hbase/pull/1625#issuecomment-623751968 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 36s | branch-2 passed | | +1 :green_heart: | compile | 6m 20s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 29s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 6m 21s | the patch passed | | +1 :green_heart: | javac | 6m 21s | the patch passed | | +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 27s | hbase-hadoop2-compat in the patch passed. | | +1 :green_heart: | unit | 2m 30s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 1m 4s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 193m 45s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 18m 20s | hbase-mapreduce in the patch passed. | | +1 :green_heart: | unit | 0m 56s | hbase-testing-util in the patch passed. | | +1 :green_heart: | unit | 4m 38s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 3m 20s | hbase-rsgroup in the patch passed. | | +1 :green_heart: | unit | 8m 7s | hbase-shell in the patch passed. | | +1 :green_heart: | unit | 3m 17s | hbase-endpoint in the patch passed. | | +1 :green_heart: | unit | 1m 6s | hbase-it in the patch passed. | | +1 :green_heart: | unit | 4m 17s | hbase-rest in the patch passed. | | +1 :green_heart: | unit | 2m 18s | hbase-examples in the patch passed. | | +1 :green_heart: | unit | 0m 29s | hbase-shaded-mapreduce in the patch passed. | | +1 :green_heart: | unit | 0m 31s | hbase-shaded-testing-util in the patch passed. | | +1 :green_heart: | unit | 1m 2s | hbase-shaded-client-project in the patch passed. | | | | 290m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1625/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1625 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ffe41d3dfe06 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 47dca8eb45 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1625/2/testReport/ | | Max. process+thread count | 3690 (vs. ulimit of 12500) | | modules | C: hbase-hadoop2-compat hbase-client hbase-http hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-rsgroup hbase-shell hbase-endpoint hbase-it hbase-rest hbase-examples hbase-shaded/hbase-shaded-mapreduce hbase-shaded/hbase-shaded-testing-util hbase-archetypes/hbase-shaded-client-project U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1625/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #746: HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the clas…
joshelser commented on a change in pull request #746:
URL: https://github.com/apache/hbase/pull/746#discussion_r419777500
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java
##
@@ -270,39 +269,23 @@ public void unbuffer() {
if (this.instanceOfCanUnbuffer == null) {
// To ensure we compute whether the stream is instance of CanUnbuffer
only once.
this.instanceOfCanUnbuffer = false;
-Class[] streamInterfaces = streamClass.getInterfaces();
-for (Class c : streamInterfaces) {
- if
(c.getCanonicalName().toString().equals("org.apache.hadoop.fs.CanUnbuffer")) {
-try {
- this.unbuffer = streamClass.getDeclaredMethod("unbuffer");
-} catch (NoSuchMethodException | SecurityException e) {
- if (isLogTraceEnabled) {
-LOG.trace("Failed to find 'unbuffer' method in class " +
streamClass
-+ " . So there may be a TCP socket connection "
-+ "left open in CLOSE_WAIT state.", e);
- }
- return;
-}
-this.instanceOfCanUnbuffer = true;
-break;
- }
+if (CanUnbuffer.class.isAssignableFrom(streamClass)) {
+ this.unbuffer = (CanUnbuffer) wrappedStream;
+ this.instanceOfCanUnbuffer = true;
}
}
if (this.instanceOfCanUnbuffer) {
try {
- this.unbuffer.invoke(wrappedStream);
-} catch (IllegalAccessException | IllegalArgumentException |
InvocationTargetException e) {
+ this.unbuffer.unbuffer();
+} catch (UnsupportedOperationException e){
Review comment:
I think this just "sucks". Hadoop doesn't have a proper exception here.
branch-2 and branch-3 are at least consistent here. I don't see us able to do
anything else. Resolving this as such.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1201: HBASE-23639 : Moving classes out of hbase-it /test for direct API use of chaos.
Apache-HBase commented on pull request #1201: URL: https://github.com/apache/hbase/pull/1201#issuecomment-623751215 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 2m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 49s | master passed | | -0 :warning: | patch | 9m 45s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | shadedjars | 5m 0s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 38s | hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -0 :warning: | javadoc | 0m 17s | hbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | javadoc | 1m 59s | root generated 2 new + 87 unchanged - 0 fixed = 89 total (was 87) | ||| _ Other Tests _ | | -1 :x: | unit | 222m 44s | root in the patch failed. | | | | 256m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1201 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d58c850fc79 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e37aafcfc2 | | Default Java | 1.8.0_232 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-it.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/testReport/ | | Max. process+thread count | 3977 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-hadoop-compat hbase-zookeeper hbase-http hbase-server hbase-it . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1201/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-8458) Support for batch version of checkAndMutate()
[
https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099405#comment-17099405
]
Toshihiro Suzuki commented on HBASE-8458:
-
Could someone please review the following PR? Actually this feature is
important for our usecase.
https://github.com/apache/hbase/pull/1648
The highlights of the changes are as follows:
- Introduced CheckAndMutate class that's used to perform CheckAndMutate
operations. The following is the JavaDoc for this class:
{code}
* Used to perform CheckAndMutate operations. Currently {@link Put}, {@link
Delete}
* and {@link RowMutations} are supported.
*
* This has a fluent style API to instantiate it, the code is like:
*
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row equals to the specified value
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifEquals(family, qualifier, value)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row doesn't exist
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifNotExists(family, qualifier)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the row
matches the filter
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifMatches(filter)
* .action(delete);
*
*
{code}
- Added new checkAndMutate APIs to the Table and AsyncTable interfaces.
-- The APIs for the Table interface:
{code}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return boolean that represents the result for the CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean checkAndMutate(CheckAndMutate checkAndMutate) throws
IOException {
return checkAndMutate(Collections.singletonList(checkAndMutate))[0];
}
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A array of boolean that represents the result for each
CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean[] checkAndMutate(List checkAndMutates) throws
IOException {
throw new NotImplementedException("Add an implementation!");
}
{code}
-- The APIs for the AsyncTable interface:
{code}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return A {@link CompletableFuture}s that represent the result for the
CheckAndMutate.
*/
CompletableFuture checkAndMutate(CheckAndMutate checkAndMutate);
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A list of {@link CompletableFuture}s that represent the result for
each
* CheckAndMutate.
*/
List> checkAndMutate(List
checkAndMutates);
/**
* A simple version of batch checkAndMutate. It will fail if there are any
failures.
*
* @param checkAndMutates The list of rows to apply.
* @return A {@link CompletableFuture} that wrapper the result boolean list.
*/
default CompletableFuture> checkAndMutateAll(
List checkAndMutates) {
return allOf(checkAndMutate(checkAndMutates));
}
{code}
- Deprecated the old checkAndMutate APIs of the Table and AsyncTable interfaces.
- This has Protocol Buffers level changes. I moved MultiRequest#condition to
RegionAction and MultiResponse#processed to RegionActionResult. This is already
disscussed in the Jira:
https://issues.apache.org/jira/browse/HBASE-8458?focusedCommentId=17064155=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17064155
Thanks.
> Support for batch version of checkAndMutate()
> -
>
> Key: HBASE-8458
> URL: https://issues.apache.org/jira/browse/HBASE-8458
> Project: HBase
> Issue Type: New Feature
> Components: Client, regionserver
>Reporter: Hari Mankude
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> The use case is that the user has multiple threads loading hundreds of keys
> into a hbase table. Occasionally there are collisions in the keys being
> uploaded by different threads. So for correctness, it is required to do
> checkAndMutate() instead of a put(). However, doing a checkAndMutate() rpc
> for every key update is non optimal. It would be good to have a batch version
> of checkAndMutate() similar to batch put(). The client can partition the keys
> on region boundaries.
> The jira is NOT looking for any type of
[GitHub] [hbase] ndimiduk commented on pull request #1342: HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile
ndimiduk commented on pull request #1342: URL: https://github.com/apache/hbase/pull/1342#issuecomment-623749922 I was hoping master @jojochuang could let me know if this helps his case, given that it was failing reliably for him but not for me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1647: add IntelliJ project files
Apache-HBase commented on pull request #1647: URL: https://github.com/apache/hbase/pull/1647#issuecomment-623749607 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] brfrn169 opened a new pull request #1648: HBASE-8458 Support for batch version of checkAndMutate()
brfrn169 opened a new pull request #1648: URL: https://github.com/apache/hbase/pull/1648 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1647: add IntelliJ project files
Apache-HBase commented on pull request #1647: URL: https://github.com/apache/hbase/pull/1647#issuecomment-623749526 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 2m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1647/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1647 | | Optional Tests | dupname asflicense xml | | uname | Linux fa7bdebea34b 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e2c73b0f49 | | Max. process+thread count | 43 (vs. ulimit of 12500) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1647/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #1584: HBASE-24256 When fixOverlap hits the max region limit, it is possible…
huaxiangsun commented on pull request #1584: URL: https://github.com/apache/hbase/pull/1584#issuecomment-623749412 Removed the code for deleteMergeQualifiers and redid the unittest code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #1192: HBASE-23875 Add JDK11 compilation and unit test support to Jira attached patch precommit
ndimiduk commented on pull request #1192: URL: https://github.com/apache/hbase/pull/1192#issuecomment-623748910 Oops, no. We closed the issue as "Won't Fix". Thanks for checking. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-24261) Redo all of our github notification integrations on new ASF infra feature
[ https://issues.apache.org/jira/browse/HBASE-24261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-24261: Assignee: Bharath Vissapragada > Redo all of our github notification integrations on new ASF infra feature > - > > Key: HBASE-24261 > URL: https://issues.apache.org/jira/browse/HBASE-24261 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Bharath Vissapragada >Priority: Major > > The new [ASF Infra feature for customizing how project gets notifications > from > github|https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories#id-.asf.yamlfeaturesforgitrepositories-Notificationsettingsforrepositories] > appears to have silently thrown away all the integration we already had set > up. > I don't know that full set of things we need. We presumably need to do this > for all of our repos. > * make sure all notifications on PRs is going to issues@ > * make sure we get links on JIRA for related PRs > * make sure we do not get updates on JIRA for every PR comment -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099399#comment-17099399 ] Nick Dimiduk commented on HBASE-23707: -- [~liuml07] I posted up a PR based on the above materials. Mind taking a look? https://github.com/apache/hbase/pull/1647 > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.0, 1.6.0 > > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on pull request #1647: add IntelliJ project files
ndimiduk commented on pull request #1647: URL: https://github.com/apache/hbase/pull/1647#issuecomment-623746342 I tried to add files according to the links from Jira. Mind giving this a try @liuml07 ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #1647: add IntelliJ project files
ndimiduk opened a new pull request #1647: URL: https://github.com/apache/hbase/pull/1647 Attempting to correctly manage Jetbrains/IntelliJ/IDEA project files. WIP. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #1625: HBASE-24284 [h3/jdk11] REST server won't start
saintstack commented on pull request #1625: URL: https://github.com/apache/hbase/pull/1625#issuecomment-623741845 Patch failed because it does the hadoop3 test run with -Phadoop-3.0 instead of -Dhadoop.profile=3.0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1639: HBASE-24305 Removed deprecations in ServerName
Apache-HBase commented on pull request #1639: URL: https://github.com/apache/hbase/pull/1639#issuecomment-623740790 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 37s | master passed | | +1 :green_heart: | checkstyle | 2m 46s | master passed | | +1 :green_heart: | spotbugs | 5m 55s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 17s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | hbase-common: The patch generated 0 new + 0 unchanged - 25 fixed = 0 total (was 25) | | +1 :green_heart: | checkstyle | 0m 11s | hbase-balancer: The patch generated 0 new + 36 unchanged - 2 fixed = 36 total (was 38) | | +1 :green_heart: | checkstyle | 1m 27s | hbase-server: The patch generated 0 new + 188 unchanged - 1 fixed = 188 total (was 189) | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-backup | | +1 :green_heart: | checkstyle | 0m 18s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 7m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 57s | The patch does not generate ASF License warnings. | | | | 54m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1639 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 7bef2bcc0320 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-balancer hbase-server hbase-backup hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1639/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23707) Add IntelliJ check style plugin configuration
[ https://issues.apache.org/jira/browse/HBASE-23707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099390#comment-17099390 ] Nick Dimiduk commented on HBASE-23707: -- Looks like this is the "official" knowledge base article on tracking changes to IntelliJ project files. https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 It also seems the project files can automatically be added by the IDE, https://www.jetbrains.com/help/idea/creating-and-managing-projects.html?_ga=2.28939388.1359844174.1588630841-364026928.1588630841#share-project-through-vcs > Add IntelliJ check style plugin configuration > - > > Key: HBASE-23707 > URL: https://issues.apache.org/jira/browse/HBASE-23707 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.0, 1.6.0 > > > IntelliJ defines project configuration across a number of files, specifically > so that some configurations can be committed with the source repository. The > checkstyle plugin configuration is one such config file; add it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #1257: HBASE 23887 Up to 3x increase BlockCache performance
Apache-HBase commented on pull request #1257: URL: https://github.com/apache/hbase/pull/1257#issuecomment-623739136 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 21s | master passed | | +1 :green_heart: | compile | 1m 8s | master passed | | +1 :green_heart: | shadedjars | 5m 43s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 39s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 1m 10s | the patch passed | | +1 :green_heart: | javac | 1m 10s | the patch passed | | +1 :green_heart: | shadedjars | 5m 39s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 204m 27s | hbase-server in the patch passed. | | | | 231m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/16/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1257 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4090f83a7ad3 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e37aafcfc2 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/16/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/16/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/16/testReport/ | | Max. process+thread count | 3608 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1257/16/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #1597: HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
ndimiduk commented on pull request #1597: URL: https://github.com/apache/hbase/pull/1597#issuecomment-623735172 > I am not willing to veto this change. I agree that it is an improvement to ship configuration settings that get this behavior rather than hard code a change when we see LocalFileSystem. Any further input re: what you would like to see for a +1 @busbey ? It sounds to me from your comment that there's a last mile that I'm missing. I'd rather put this to bed than need revisit it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #1597: HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
ndimiduk commented on a change in pull request #1597: URL: https://github.com/apache/hbase/pull/1597#discussion_r419758688 ## File path: src/main/asciidoc/_chapters/getting_started.adoc ## @@ -426,7 +386,7 @@ You can stop HBase the same way as in the <> procedure, u [[quickstart_fully_distributed]] -=== Advanced - Fully Distributed +=== Fully Distributed for Production In reality, you need a fully-distributed configuration to fully test HBase and to use it in real-world scenarios. In a distributed configuration, the cluster contains multiple nodes, each of which runs one or more HBase daemon. Review comment: Maybe drop my addition of "for Production" then? I thought it odd that our default operating configuration was flagged as "advanced". I agree there's more to a production deploy than just these configs. However, I also don't think it's our job to be prescriptive of what a production deployment looks like, re: monitoring, metrics, log aggregation, I think anything more than a paragraph or two of "recommended" or "strongly encouraged" supporting infrastructure is beyond the scope of our document. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1485: HBASE-23969 Meta browser should show all `info` columns
Apache-HBase commented on pull request #1485: URL: https://github.com/apache/hbase/pull/1485#issuecomment-623733340 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 18s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | checkstyle | 2m 8s | master passed | | +1 :green_heart: | spotbugs | 4m 27s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | -0 :warning: | checkstyle | 0m 31s | hbase-client: The patch generated 3 new + 45 unchanged - 0 fixed = 48 total (was 45) | | -0 :warning: | checkstyle | 1m 16s | hbase-server: The patch generated 4 new + 4 unchanged - 1 fixed = 8 total (was 5) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 45m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1485 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cd554c9e8b10 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 1878db843c | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1485/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24318) Create-release scripts fixes and enhancements
[ https://issues.apache.org/jira/browse/HBASE-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099381#comment-17099381 ] Sean Busbey commented on HBASE-24318: - please link to your PR > Create-release scripts fixes and enhancements > - > > Key: HBASE-24318 > URL: https://issues.apache.org/jira/browse/HBASE-24318 > Project: HBase > Issue Type: Improvement > Components: create-release >Affects Versions: 3.0.0-alpha-1, 2.3.0 > Environment: Linux Docker container, or Mac OS X without Docker >Reporter: Matthew Foley >Priority: Major > > The create-release tools are a set of scripts that promote best practice in > making Apache releases, by automating most of the steps and running them > under Docker for reliability. However, the current state of the scripts has > many bugs and ambiguities. > The proposed PR cleans up the code and clarifies usage. It enables: > - Clear statement of the four steps, which are now called `tag`, > `publish-dist`, `publish-snapshot`, and `publish-release` (the latter two > being mutually exclusive alternatives). > - Ability to do the three tag-dist-release steps with a single command, or do > any of the steps singly. (Running singly had bugs and unfulfilled > dependencies before.) > - Ability to do a reliable and useful "dry run" of all steps or each step, > and chain together the tag step with publish steps in a dry run. > - Ability to run any or all steps correctly in Docker or outside of Docker, > on Linux and Mac. > - Cleaned up all `shellcheck` errors in the scripts, and removed ambiguities > and redundancies in the many environment variables used. > In addition, the changes move the code toward being more general / less > HBase-specific, so it can be run on any Apache project (while still > accommodating HBase-specific features regarding how sub-projects are named > and organized in Jira and release repos). In future I propose to take it > further along that path, and move create-release into Yetus (recognizing that > this create-release code has been passed between a couple other projects > already). > These changes have NO IMPACT on HBase functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24258) [Hadoop3.3] Update license for org.ow2.asm:*
[
https://issues.apache.org/jira/browse/HBASE-24258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24258.
---
Fix Version/s: 2.2.5
2.3.0
3.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
Pushed to branch-2.2+. Thanks for the patch [~weichiu] (And thanks for the
review [~janh])
> [Hadoop3.3] Update license for org.ow2.asm:*
>
>
> Key: HBASE-24258
> URL: https://issues.apache.org/jira/browse/HBASE-24258
> Project: HBase
> Issue Type: Task
> Components: dependencies
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.5
>
>
> Hadoop 3.3 brings a few Jetty dependencies which transitively brings in
> org.ow2.asm:asm-analysis, org.ow2.asm:asm-commons, org.ow2.asm:asm-tree.
> When testing with the latest Jetty (9.4.26.v20200117) I found its
> org.ow2.asm:* updated from 7.1 to 7.2, which changed the declared license
> from "BSD" to "BSD-3-Clause License" (The actual license text did not
> change). The HBase's license checker doesn't accept it.
> File the jira to update it to "BSD 3-Clause License" so that HBase can build.
> {noformat}
> [INFO] | | | +-
> org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.4.26.v20200117:test
> [INFO] | | | | +-
> org.eclipse.jetty:jetty-annotations:jar:9.4.26.v20200117:test
> [INFO] | | | | | +-
> org.eclipse.jetty:jetty-plus:jar:9.4.26.v20200117:test
> [INFO] | | | | | | \-
> org.eclipse.jetty:jetty-jndi:jar:9.4.26.v20200117:test
> [INFO] | | | | | \- org.ow2.asm:asm-commons:jar:7.2:test
> [INFO] | | | | | +- org.ow2.asm:asm-tree:jar:7.2:test
> [INFO] | | | | | \- org.ow2.asm:asm-analysis:jar:7.2:test
> {noformat}
> {noformat}
> This product includes asm-analysis licensed under the BSD-3-Clause.
> ERROR: Please check this License for acceptability here:
> https://www.apache.org/legal/resolved
> If it is okay, then update the list named 'non_aggregate_fine' in the
> LICENSE.vm file.
> If it isn't okay, then revert the change that added the dependency.
> More info on the dependency:
> org.ow2.asm
> asm-analysis
> 7.2
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
