[GitHub] [hbase-native-client] Apache-HBase commented on pull request #14: HBASE-23105: Download lib double conversion, fizz, update folly
Apache-HBase commented on pull request #14: URL: https://github.com/apache/hbase-native-client/pull/14#issuecomment-719187264 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 357 | Docker mode activated. | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | -1 | CMakeLists.txt | 15 | root in master failed. | | -1 | compile | 15 | root in master failed. | ||| _ Patch Compile Tests _ | | -1 | CMakeLists.txt | 15 | root in the patch failed. | | -1 | compile | 16 | root in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 15 | root in the patch failed. | | | | 461 | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-native-client/pull/14 | | Optional Tests | compile unit | | uname | Linux securecluster 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | cmake | | Personality | bin/hbase-native-client-personality.sh | | git revision | master / 926a6f9 | | CMakeLists.txt | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/branch-CMakeLists.txt-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/branch-compile-root.txt | | CMakeLists.txt | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/patch-CMakeLists.txt-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/patch-compile-root.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/artifact/hbase-native-client-precommit/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/testReport/ | | Max. process+thread count | 8 (vs. ulimit of 1000) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/console | | versions | git=2.7.4 cmake=cmake version 3.5.1 make=GNU Make 4.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv commented on pull request #6: HBASE-23105: Download lib double conversion, fizz, update folly
bharathv commented on pull request #6: URL: https://github.com/apache/hbase-native-client/pull/6#issuecomment-719185147 I figured out the issue, submitted a new PR with my patch on top #14. A quick summary of my patch addition - Patched fizz/wangle to find boost libs and pass boost root during compilation - Fixed the linking issue (removed your WHOLE_ARCHIVE workaround), our linking is still a bit messed up but I think the patch should be ok for now - Added cyrussasl dependency so that its compiled correctly. Lets see how the precommit goes. There was an issue with libfmt-dev not available for ubuntu 16.04 but that looks optional, so removed it (not sure though). @phrocker Feel free to apply my patch on top of your PR, I don't wan't to take credit for your work with one small final patch :-). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] Apache-HBase commented on pull request #14: HBASE-23105: Download lib double conversion, fizz, update folly
Apache-HBase commented on pull request #14: URL: https://github.com/apache/hbase-native-client/pull/14#issuecomment-719178082 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/HBase/job/hbase-native-client-github-precommit/job/PR-14/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv opened a new pull request #14: HBASE-23105: Download lib double conversion, fizz, update folly
bharathv opened a new pull request #14: URL: https://github.com/apache/hbase-native-client/pull/14 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
Apache-HBase commented on pull request #2591: URL: https://github.com/apache/hbase/pull/2591#issuecomment-719165758 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 53s | branch-1 passed | | +1 :green_heart: | compile | 0m 40s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 43s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 1s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 55s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 45s | the patch passed | | +1 :green_heart: | checkstyle | 1m 30s | hbase-server: The patch generated 0 new + 42 unchanged - 6 fixed = 42 total (was 48) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 34s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 2m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 121m 51s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 164m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2591 | | JIRA Issue | HBASE-24859 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7b60796fe67d 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2591/out/precommit/personality/provided.sh | | git revision | branch-1 / 8813b3b | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/8/testReport/ | | Max. process+thread count | 4143 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/8/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25221) Backport HBASE-24368 "Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null"
[ https://issues.apache.org/jira/browse/HBASE-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223400#comment-17223400 ] Michael Stack commented on HBASE-25221: --- Thanks for clarification [~psomogyi] > Backport HBASE-24368 "Let HBCKSCP clear 'Unknown Servers', even if > RegionStateNode has RegionLocation == null" > -- > > Key: HBASE-25221 > URL: https://issues.apache.org/jira/browse/HBASE-25221 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ramkrish86 commented on pull request #2582: HBASE-25187 Improve SizeCachedKV variants initialization
ramkrish86 commented on pull request #2582: URL: https://github.com/apache/hbase/pull/2582#issuecomment-719157690 All tests pass now. @anoopsjohn , @saintstack , @openinx - Ping for reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 commented on pull request #2550: HBASE-25187 Improve SizeCachedKV variants initialization
ramkrish86 commented on pull request #2550: URL: https://github.com/apache/hbase/pull/2550#issuecomment-719157579 I will go ahead with the master branch PR and then cherry-pick to other branches as per RM's wish. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ramkrish86 closed pull request #2550: HBASE-25187 Improve SizeCachedKV variants initialization
ramkrish86 closed pull request #2550: URL: https://github.com/apache/hbase/pull/2550 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719148740 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | compile | 1m 22s | master passed | | +1 :green_heart: | shadedjars | 7m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 7m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 6s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 202m 57s | hbase-server in the patch passed. | | | | 240m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4531e1912c87 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eee1cf7ff6 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/testReport/ | | Max. process+thread count | 3465 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2576: URL: https://github.com/apache/hbase/pull/2576#issuecomment-719142381 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 52s | branch-1 passed | | +1 :green_heart: | compile | 0m 40s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 46s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 59s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 3s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 59s | the patch passed | | +1 :green_heart: | compile | 0m 43s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 43s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 45s | the patch passed | | -1 :x: | checkstyle | 1m 48s | hbase-server: The patch generated 2 new + 812 unchanged - 18 fixed = 814 total (was 830) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 32s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK v1.7.0_272 | | -1 :x: | findbugs | 2m 54s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | -1 :x: | unit | 105m 47s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 148m 7s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Useless condition:canAbort == true at this point At HRegion.java:[line 1647] | | Failed junit tests | hadoop.hbase.regionserver.wal.TestDurability | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2576 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 47c840ede344 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2576/out/precommit/personality/provided.sh | | git revision | branch-1 / 8813b3b | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/artifact/out/new-findbugs-hbase-server.html | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/testReport/ | | Max. process+thread count | 4226 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/8/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-71912 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 1m 32s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 5s | hbase-client in the patch passed. | | -1 :x: | unit | 140m 2s | hbase-server in the patch failed. | | | | 171m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c31f51f444f8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eee1cf7ff6 | | Default Java | AdoptOpenJDK-11.0.6+10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/testReport/ | | Max. process+thread count | 4105 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
Apache-HBase commented on pull request #2591: URL: https://github.com/apache/hbase/pull/2591#issuecomment-719121119 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 42s | branch-1 passed | | +1 :green_heart: | compile | 0m 39s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 44s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 58s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 56s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 31s | hbase-server: The patch generated 0 new + 44 unchanged - 4 fixed = 44 total (was 48) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 36s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 40s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 2m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 122m 51s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. | | | | 165m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2591 | | JIRA Issue | HBASE-24859 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d25d6a5431cc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2591/out/precommit/personality/provided.sh | | git revision | branch-1 / 8813b3b | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/7/testReport/ | | Max. process+thread count | 3878 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/7/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-719104837 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 52s | master passed | | +1 :green_heart: | compile | 1m 50s | master passed | | +1 :green_heart: | shadedjars | 8m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 1m 53s | the patch passed | | +1 :green_heart: | javac | 1m 53s | the patch passed | | +1 :green_heart: | shadedjars | 8m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 1s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 144m 3s | hbase-server in the patch passed. | | | | 182m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e229dac2f5b8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/testReport/ | | Max. process+thread count | 3894 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2
Apache-HBase commented on pull request #2596: URL: https://github.com/apache/hbase/pull/2596#issuecomment-719102132 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 39s | branch-2 passed | | +1 :green_heart: | compile | 1m 36s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 28s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 7m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 12s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 192m 32s | hbase-server in the patch passed. | | | | 233m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2596 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux aa61d5922da1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 69f282e21a | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/testReport/ | | Max. process+thread count | 3317 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-719101365 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 23s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 23s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 140m 12s | hbase-server in the patch passed. | | | | 169m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dc74a265b670 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/testReport/ | | Max. process+thread count | 5122 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache9 edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-719100163 > @Apache9 I am having trouble responding to your comment above. I'm not sure what does this mean so I will just ignore it and focus on the tech part below. If this is indeed something important please tell me if I'm doing something wrong. Thanks. > > > After reading this discussion several times, I think the reason we do not want to niterrupte a WAL sync is that it may lead to a region server abort? > > I would say this is not the case here. I checked the code again, the actual sync is done in the disruptor thread, in the rpc thread we just block on a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will just lead to an IOException tp client, the actual sync operation will not be interrupted so we are safe. > > So I do not think we need to disable interrupts here? > > You are correct about the SyncFuture. > > Initially my thinking was the same as yours. > > At some point I became concerned about this case, though: > > * We start the WAL append > * SyncFuture is interrupted, client gets an exception, client thinks the mutation failed > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so we are at risk of the failed from client perspective mutation being applied during WAL replay for some reason > > So we have a case where the client's understanding of what happened is incorrect. What happens in the WAL and the client's understanding of what happend should be in sync. If the WAL commit fails the client should get an exception. If the WAL commit succeeds the client should see success. Whether we interrupt or not is our option. disableInterrupt/enableInterrupt here is consideration for the client. > > I can add the above as a code comment next to the disableInterrupt() call, would that help? > > Here is another case of concern: > > * We start the WAL append > * SyncFuture is interrupted, memstore is not updated, client gets an exception. Unless the WAL is replayed the local cluster A will not have the mutation applied. > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so it is shipped to the remote cluster, and is applied at the remote cluster B. > * Now the data in cluster A and B are out of sync. > > The disableInterrupt/enableInterrupt pair combines the WAL append and memstore update into a single protected operation , so this error case will not happen because of the interrupts I am adding with this patch. I think this is a valid point. We should not return an exception to client unless we can make sure the (update) operation is failed. And you are right about the replication part, if the WAL sync succeeds then we should consider the operation succeeds, that's why we will abort the RS if a WAL sync fails, as we do not know if it succeeds so the only safe way is to abort the RS, close the ongoing WAL file, and reconstruct the memstore by reading the WAL file. But then, I do not think we can interrupt the rpc handler after we issue the WAL sync right? So let's just remove the rpc handler from the Map before we issuing the WAL sync? And think more, for a put operation, usually there are 3 places where we could block, first is acquiring row lock, second is WAL sync, third is completing MVCC. If we consider the argumenta above, it seems that we can interrupt the first acquiring row lock operation? Then is it still worth to add so many interruption checks in our write path? Just my thoughts. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache9 edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-719100163 > @Apache9 I am having trouble responding to your comment above. I'm not sure what does this mean so I will just ignore it and focus on the tech part below. If this is indeed something important please tell me if I'm doing something wrong. Thanks. > > > After reading this discussion several times, I think the reason we do not want to niterrupte a WAL sync is that it may lead to a region server abort? > > I would say this is not the case here. I checked the code again, the actual sync is done in the disruptor thread, in the rpc thread we just block on a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will just lead to an IOException tp client, the actual sync operation will not be interrupted so we are safe. > > So I do not think we need to disable interrupts here? > > You are correct about the SyncFuture. > > Initially my thinking was the same as yours. > > At some point I became concerned about this case, though: > > * We start the WAL append > * SyncFuture is interrupted, client gets an exception, client thinks the mutation failed > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so we are at risk of the failed from client perspective mutation being applied during WAL replay for some reason > > So we have a case where the client's understanding of what happened is incorrect. What happens in the WAL and the client's understanding of what happend should be in sync. If the WAL commit fails the client should get an exception. If the WAL commit succeeds the client should see success. Whether we interrupt or not is our option. disableInterrupt/enableInterrupt here is consideration for the client. > > I can add the above as a code comment next to the disableInterrupt() call, would that help? > > Here is another case of concern: > > * We start the WAL append > * SyncFuture is interrupted, memstore is not updated, client gets an exception. Unless the WAL is replayed the local cluster A will not have the mutation applied. > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so it is shipped to the remote cluster, and is applied at the remote cluster B. > * Now the data in cluster A and B are out of sync. > > The disableInterrupt/enableInterrupt pair combines the WAL append and memstore update into a single protected operation , so this error case will not happen because of the interrupts I am adding with this patch. I think this is a valid point. We should not return an exception to client unless we can make sure the (update) operation is failed. And you are right about the replication part, if the WAL sync succeeds then we should consider the operation succeeds, that's why we will abort the RS if a WAL sync fails, as we do not know if it succeeds so the only safe way is to abort the RS, close the ongoing WAL file, and reconstruct the memstore by reading the WAL file. But then, I do not think we can interrupt the rpc handler after we issue the WAL sync right? So let's just remove the rpc handler from the Map before we issuing the WAL sync? And think more, for a put operation, usually there are 3 places where we could block, first is acquiring row lock, second is WAL sync, third is completing MVCC. If we consider the argumenta above, it seems that we can interrupt the first acquiring row lock operation? Then it is still worth to add so many interruption check in our write path? Just my thoughts. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache9 commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-719100163 > @Apache9 I am having trouble responding to your comment above. I'm not sure what does this mean so I will just ignore it and focus on the tech part below. If this is indeed something important please tell me if I'm doing something wrong. Thanks. > > > After reading this discussion several times, I think the reason we do not want to niterrupte a WAL sync is that it may lead to a region server abort? > > I would say this is not the case here. I checked the code again, the actual sync is done in the disruptor thread, in the rpc thread we just block on a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will just lead to an IOException tp client, the actual sync operation will not be interrupted so we are safe. > > So I do not think we need to disable interrupts here? > > You are correct about the SyncFuture. > > Initially my thinking was the same as yours. > > At some point I became concerned about this case, though: > > * We start the WAL append > * SyncFuture is interrupted, client gets an exception, client thinks the mutation failed > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so we are at risk of the failed from client perspective mutation being applied during WAL replay for some reason > > So we have a case where the client's understanding of what happened is incorrect. What happens in the WAL and the client's understanding of what happend should be in sync. If the WAL commit fails the client should get an exception. If the WAL commit succeeds the client should see success. Whether we interrupt or not is our option. disableInterrupt/enableInterrupt here is consideration for the client. > > I can add the above as a code comment next to the disableInterrupt() call, would that help? > > Here is another case of concern: > > * We start the WAL append > * SyncFuture is interrupted, memstore is not updated, client gets an exception. Unless the WAL is replayed the local cluster A will not have the mutation applied. > * WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so it is shipped to the remote cluster, and is applied at the remote cluster B. > * Now the data in cluster A and B are out of sync. > > The disableInterrupt/enableInterrupt pair combines the WAL append and memstore update into a single protected operation , so this error case will not happen because of the interrupts I am adding with this patch. I think this is a valid point. We should not return an exception to client unless we can make sure the (update) operation is failed. And you are right about the replication part, if the WAL sync succeeds then we should consider the operation succeeds, that's why we will abort the RS if a WAL sync fails, as we do not know if it succeeds so the only safe way is to abort the RS, close the ongoing WAL file, and reconstruct the memstore by reading the WAL file. But then, I do not think we can interrupt the rpc handler after we issue the WAL sync right? So let's just remove the rpc handler from the Map before we issuing the WAL sync? And think more, for a put operation, usually there are 3 places where we could block, first is acquiring row lock, second is WAL sync, third is completing MVCC. If we consider the argumenta above, it seems that we can interrupt the first acquiring row lock operation? Then it is still worth to add so many interruption check in our write path? Just my thoughts. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719094199 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 48s | master passed | | +1 :green_heart: | checkstyle | 1m 38s | master passed | | +1 :green_heart: | spotbugs | 3m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | -0 :warning: | checkstyle | 0m 26s | hbase-client: The patch generated 3 new + 13 unchanged - 0 fixed = 16 total (was 13) | | -0 :warning: | checkstyle | 1m 11s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 44s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | -1 :x: | spotbugs | 1m 11s | hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 46m 9s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-client | | | Inconsistent synchronization of org.apache.hadoop.hbase.client.AsyncConnectionImpl.choreService; locked 66% of time Unsynchronized access at AsyncConnectionImpl.java:66% of time Unsynchronized access at AsyncConnectionImpl.java:[line 200] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 69ab94dbc029 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eee1cf7ff6 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/artifact/yetus-general-check/output/new-spotbugs-hbase-client.html | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/8/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
saintstack commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514632131
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req,
CompletableFuture {
+int numOfReplicas = 1;
+try {
+ RegionLocations metaLocations =
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+ numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+ LOG.error("Failed to get table {}'s region replication, ",
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+ });
+break;
+ case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+ boolean useMetaReplicas =
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+ if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;
+ }
+}
+break;
+ default:
+// Doing nothing
Review comment:
I thought getting here would be an exception... sounds like it is not.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2
Apache-HBase commented on pull request #2596: URL: https://github.com/apache/hbase/pull/2596#issuecomment-719083398 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 33s | branch-2 passed | | +1 :green_heart: | compile | 1m 18s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 15s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 5m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 138m 41s | hbase-server in the patch passed. | | | | 170m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2596 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c11fc3f9c86b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 69f282e21a | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/testReport/ | | Max. process+thread count | 4373 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24977) Meta table shouldn't be modified as read only
[
https://issues.apache.org/jira/browse/HBASE-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24977:
--
Fix Version/s: 2.4.0
3.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Merged to master and branch-2. Thanks for the patch [~pankajkumar]
> Meta table shouldn't be modified as read only
> -
>
> Key: HBASE-24977
> URL: https://issues.apache.org/jira/browse/HBASE-24977
> Project: HBase
> Issue Type: Bug
> Components: meta
>Affects Versions: 2.3.0
>Reporter: Ajeet Rai
>Assignee: Pankaj Kumar
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> After HBASE-23055, Alter of Meta schema is allowed.
> After this change meta table can be made Read only table which is not correct
> as per my view,because it will impact majority of the functionality of HBase.
>
> alter 'hbase:meta',\{METHOD=>'table_att', READONLY=>true}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2537: HBASE-24977 Meta table shouldn't be modified as read only
saintstack commented on pull request #2537: URL: https://github.com/apache/hbase/pull/2537#issuecomment-719082744 Tried it locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #2537: HBASE-24977 Meta table shouldn't be modified as read only
saintstack merged pull request #2537: URL: https://github.com/apache/hbase/pull/2537 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2600: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-1
Apache-HBase commented on pull request #2600: URL: https://github.com/apache/hbase/pull/2600#issuecomment-719080003 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 48s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 24s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 10m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2600/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2600 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux a4cfa20655f2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2600/out/precommit/personality/provided.sh | | git revision | branch-1 / 8813b3b | | Max. process+thread count | 41 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2600/1/console | | versions | git=1.9.1 maven=3.0.5 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719079123 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 1s | master passed | | +1 :green_heart: | compile | 1m 22s | master passed | | +1 :green_heart: | shadedjars | 7m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 7m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 202m 53s | hbase-server in the patch passed. | | | | 234m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 239882b2bb8c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/testReport/ | | Max. process+thread count | 3607 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2601: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-1.4
ndimiduk opened a new pull request #2601: URL: https://github.com/apache/hbase/pull/2601 Signed-off-by: Sean Busbey Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2600: HBASE-24200 Upgrade to Yetus 0.12.0
ndimiduk opened a new pull request #2600: URL: https://github.com/apache/hbase/pull/2600 Signed-off-by: Sean Busbey Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2599: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2.2
Apache-HBase commented on pull request #2599: URL: https://github.com/apache/hbase/pull/2599#issuecomment-719075965 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 3m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2599/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2599 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 87d63820eb8d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2599/out/precommit/personality/provided.sh | | git revision | branch-2.2 / ffd8794af5 | | Max. process+thread count | 50 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2599/1/console | | versions | git=2.11.0 maven=3.5.4 shellcheck=0.7.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2599: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2.2
ndimiduk opened a new pull request #2599: URL: https://github.com/apache/hbase/pull/2599 Signed-off-by: Sean Busbey Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2597: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2
Apache-HBase commented on pull request #2597: URL: https://github.com/apache/hbase/pull/2597#issuecomment-719073907 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 6s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 1s | ASF License check generated no output? | | | | 8m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2597/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2597 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux a73b757cfcd2 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 69f282e21a | | Max. process+thread count | 41 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2597/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2598: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2.3
Apache-HBase commented on pull request #2598: URL: https://github.com/apache/hbase/pull/2598#issuecomment-719073064 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2598: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2.3
Apache-HBase commented on pull request #2598: URL: https://github.com/apache/hbase/pull/2598#issuecomment-719072870 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.3 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2598/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2598 | | Optional Tests | | | uname | Linux a53a896d665c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.3 / 2f9d507362 | | Max. process+thread count | 46 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2598/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2597: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2
Apache-HBase commented on pull request #2597: URL: https://github.com/apache/hbase/pull/2597#issuecomment-719072031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2598: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2.3
ndimiduk opened a new pull request #2598: URL: https://github.com/apache/hbase/pull/2598 Signed-off-by: Sean Busbey Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-24200) Upgrade to Yetus 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-24200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-24200 started by Nick Dimiduk. > Upgrade to Yetus 0.12.0 > --- > > Key: HBASE-24200 > URL: https://issues.apache.org/jira/browse/HBASE-24200 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > A new Yetus release is imminent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24200) Upgrade to Yetus 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-24200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24200: - Affects Version/s: 2.3.4 2.2.7 1.4.14 2.4.0 1.7.0 3.0.0-alpha-1 > Upgrade to Yetus 0.12.0 > --- > > Key: HBASE-24200 > URL: https://issues.apache.org/jira/browse/HBASE-24200 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.4.0, 1.4.14, 2.2.7, 2.3.4 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > A new Yetus release is imminent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24200) Upgrade to Yetus 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-24200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-24200: Assignee: Nick Dimiduk > Upgrade to Yetus 0.12.0 > --- > > Key: HBASE-24200 > URL: https://issues.apache.org/jira/browse/HBASE-24200 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > A new Yetus release is imminent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24200) Upgrade to Yetus 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-24200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24200: - Fix Version/s: 3.0.0-alpha-1 > Upgrade to Yetus 0.12.0 > --- > > Key: HBASE-24200 > URL: https://issues.apache.org/jira/browse/HBASE-24200 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0-alpha-1 > > > A new Yetus release is imminent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24200) Upgrade to Yetus 0.12.0
[ https://issues.apache.org/jira/browse/HBASE-24200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24200: - Status: Patch Available (was: In Progress) > Upgrade to Yetus 0.12.0 > --- > > Key: HBASE-24200 > URL: https://issues.apache.org/jira/browse/HBASE-24200 > Project: HBase > Issue Type: Task > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > > A new Yetus release is imminent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #2597: Backport "HBASE-24200 Upgrade to Yetus 0.12.0" to branch-2
ndimiduk opened a new pull request #2597: URL: https://github.com/apache/hbase/pull/2597 Signed-off-by: Sean Busbey Signed-off-by: Duo Zhang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25228) Delete dev-support/jenkins_precommit_jira_yetus.sh
Nick Dimiduk created HBASE-25228: Summary: Delete dev-support/jenkins_precommit_jira_yetus.sh Key: HBASE-25228 URL: https://issues.apache.org/jira/browse/HBASE-25228 Project: HBase Issue Type: Task Components: build Reporter: Nick Dimiduk We no longer accept contributions via patches attached to Jira. As such, this script in dev-support can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
Apache-HBase commented on pull request #2591: URL: https://github.com/apache/hbase/pull/2591#issuecomment-719067255 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 7s | branch-1 passed | | +1 :green_heart: | compile | 0m 39s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 45s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 44s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 42s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 0s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 53s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 35s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 2m 50s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 123m 31s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 167m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2591 | | JIRA Issue | HBASE-24859 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux fdf2c3c397d5 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2591/out/precommit/personality/provided.sh | | git revision | branch-1 / b30d1d1 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/6/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/6/testReport/ | | Max. process+thread count | 4046 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2591/6/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-native-client] bharathv commented on pull request #6: HBASE-23105: Download lib double conversion, fizz, update folly
bharathv commented on pull request #6:
URL: https://github.com/apache/hbase-native-client/pull/6#issuecomment-719067284
I think your following change also is related to the issue I'm running into,
so perhaps you already know what the problem is
`target_link_libraries(simple-client ${WHOLE_ARCHIVE} ${Boost_context}
${NO_WHOLE_ARCHIVE})`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719066991 Pushed a separate patch which adds metaReplicaId to RegionLocations. It is being used to hold from which meta replica locations come from. It adds 4 bytes (int) to RegionLocations. If it looks good, it can be merged within this patch, otherwise, I can create a followup jira. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
ndimiduk commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-719066566 Landed to master. Will watch this one. Meanwhile, staging PRs for the other active branches. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-719066534 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 42s | master passed | | +1 :green_heart: | compile | 2m 21s | master passed | | +1 :green_heart: | shadedjars | 6m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | compile | 2m 19s | the patch passed | | +1 :green_heart: | javac | 2m 19s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 17s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 290m 0s | root in the patch failed. | | | | 323m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 407a7366a9cc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/testReport/ | | Max. process+thread count | 4627 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
ndimiduk merged pull request #1531: URL: https://github.com/apache/hbase/pull/1531 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-719064543 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 3m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | | | uname | Linux 2519054153ea 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Max. process+thread count | 52 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-719064227 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | hadolint | 0m 2s | There were no new hadolint issues. | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | dupname asflicense shellcheck shelldocs hadolint | | uname | Linux dfbffb697788 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Max. process+thread count | 45 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 hadolint=1.17.5-0-g443423c | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #1531: HBASE-24200 Upgrade to Yetus 0.12.0
Apache-HBase commented on pull request #1531: URL: https://github.com/apache/hbase/pull/1531#issuecomment-719063879 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1531 | | Optional Tests | | | uname | Linux bd4a2e3fc72d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Max. process+thread count | 48 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-1531/10/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-719063108 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 55s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 50s | master passed | | +1 :green_heart: | checkstyle | 1m 53s | master passed | | +1 :green_heart: | spotbugs | 3m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 25s | hbase-common: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | checkstyle | 1m 13s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 23s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 49m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 29e758b2d216 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 12d0397015 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/8/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-24463) Allow operator to limit total normalization work per invocation
[ https://issues.apache.org/jira/browse/HBASE-24463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24463. -- Resolution: Won't Fix The ability to limit the normalizer is now provided by rate limiting. > Allow operator to limit total normalization work per invocation > --- > > Key: HBASE-24463 > URL: https://issues.apache.org/jira/browse/HBASE-24463 > Project: HBase > Issue Type: Improvement > Components: master, Normalizer >Reporter: Nick Dimiduk >Priority: Major > > During review on HBASE-24418, we observed that there's no way for an operator > to limit the total amount of work a normalizer invocation will do. One > suggestion was allowing settings for maximum number of plan executions per > table or maximum number of plan executions per run. I've seen other systems > limit the amount of CPU time permitted for a given run. > At least we have a run lock that prevents multiple invocations from running > concurrently, which should prevent one run from starting before the previous > one finished. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
bharathv commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514590686
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
##
@@ -271,6 +272,11 @@ public void testNumOfSplits(int splitsPerRegion, int
expectedNumOfSplits) throws
TableInputFormat tif = new TableInputFormat();
tif.setConf(job.getConfiguration());
List splits = tif.getSplits(job);
+for (InputSplit split : splits) {
+ TableSplit tableSplit = (TableSplit) split;
+ Assert.assertEquals(tableSplit.getScan().getStartRow(),
HConstants.EMPTY_START_ROW);
Review comment:
Remove the other two asserts? They are implied if the scan is "" ?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -86,7 +86,16 @@ static Version fromCode(int code) {
private byte [] endRow;
private String regionLocation;
private String encodedRegionName = "";
+
+ /** The scan object may be null but the serialized form of scan is never null
Review comment:
nit: new lines missing
```
/**
* .
*/
```
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -214,6 +223,14 @@ public Scan getScan() throws IOException {
return TableMapReduceUtil.convertStringToScan(this.scan);
}
+ /**
+ * Returns a scan object in the serialized form
+ * @return a serialized scan object
+ */
+ public String getScanAsString() {
Review comment:
Mark this InterfaceAudience.Private? This is a test helper method..
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719054811 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 10s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 6m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 5s | hbase-client in the patch passed. | | -1 :x: | unit | 131m 35s | hbase-server in the patch failed. | | | | 166m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6a121e4aa929 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/testReport/ | | Max. process+thread count | 3941 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719052727 Update to address some of comments from Stack. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514588062
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
if (!canUpdateOnError(loc, oldLoc)) {
return;
}
+ // Tell metaReplicaSelector that the location is stale. It will create a
stale entry
+ // with timestamp internally. Next time the client looks up the same
location,
+ // it will pick a different meta replica region. For the current
implementation,
+ // the metaReplicaId is not used, so the primary one is passed in.
+ if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// TODO: pass in -1 as currently fromReplicaId is not being used.
Review comment:
Good catch, missed the comment. I am working on a follow-up patch which
fills in the read fromMetaReplicaId. Going to post it as a new jira or within
this patch depends on Duo's feedback.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514587250
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -433,9 +474,24 @@ private void locateInMeta(TableName tableName,
LocateRequest req) {
Scan scan = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
.addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(locatePrefetchLimit)
.setReadType(ReadType.PREAD);
-if (useMetaReplicas) {
- scan.setConsistency(Consistency.TIMELINE);
+
+switch (this.metaReplicaMode) {
+ case LoadBalance:
+int metaReplicaId = this.metaReplicaSelector.select(tableName,
req.row, req.locateType);
+if (metaReplicaId != RegionInfo.DEFAULT_REPLICA_ID) {
+ // If the selector gives a non-primary meta replica region, then go
with it.
+ // Otherwise, just go to primary in non-hedgedRead mode.
+ scan.setConsistency(Consistency.TIMELINE);
+ scan.setReplicaId(metaReplicaId);
+}
+break;
+ case HedgedRead:
+scan.setConsistency(Consistency.TIMELINE);
+break;
+ default:
+// do nothing
Review comment:
Now I thought about it, it makes sense. Maybe a
CatalogReplicaModeManager class which encaps mode and selector?
Let me create a followup jira after this is merged.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-719042209 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 44s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 6m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 1m 24s | the patch passed | | +1 :green_heart: | javac | 1m 24s | the patch passed | | +1 :green_heart: | shadedjars | 6m 31s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 139m 42s | hbase-server in the patch passed. | | | | 169m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d3193a0f0a64 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/testReport/ | | Max. process+thread count | 4393 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-719041537 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 6m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 136m 6s | hbase-server in the patch passed. | | | | 168m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a2a9b079 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/testReport/ | | Max. process+thread count | 3733 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess
[ https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25227: Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > [branch-1] Fix Java 11 runtime issue in UnsafeAccess > > > Key: HBASE-25227 > URL: https://issues.apache.org/jira/browse/HBASE-25227 > Project: HBase > Issue Type: Bug >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Fix For: 1.7.0 > > > When running tests with Java 11 UnsafeAccess is observed to throw > NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them > as parameters to Unsafe methods which should take 'long'. Not an issue on > branch-2 and up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514584611
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req,
CompletableFuture {
+int numOfReplicas = 1;
+try {
+ RegionLocations metaLocations =
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+ numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+ LOG.error("Failed to get table {}'s region replication, ",
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+ });
+break;
+ case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+ boolean useMetaReplicas =
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+ if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;
+ }
+}
+break;
+ default:
+// Doing nothing
Review comment:
There are three modes:
1. HedgedRead
2. LoadBalance
3. None (no mode is configured).
It will get here when HedgedRead is configured. I assume that there is no
need to log HedgedRead mode, otherwise, it needs to log at other two branches.
wdyt?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514583501
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req,
CompletableFuture {
+int numOfReplicas = 1;
+try {
+ RegionLocations metaLocations =
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+ numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+ LOG.error("Failed to get table {}'s region replication, ",
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+ });
+break;
+ case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+ boolean useMetaReplicas =
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+ if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;
Review comment:
The whole purpose is to set up the metaReplicaMode. In case of
LoadBalance mode, it needs to create CatalogSelector. In case of HedgedRead,
there is nothing to be done. None is not HedgedRead, it means that there is no
metaReplicaMode configured. In this case, it still needs to check if
USE_META_REPLICAS is configured or not, if USE_META_REPLICAS is configured, it
needs to set metaReplicaMode to HedgedRead. HedgedRead is the previous
USE_META_REPLICAS. default branch is added to avoid warning from some java
checking.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223221#comment-17223221 ] Sandeep Guggilam commented on HBASE-25032: -- [~anoop.hbase] [~apurtell] I spent some time looking at the code today. One thing I noticed is that we abort the RS by throwing exception in case of any issues with replication setup with the peer during the [startup|https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L1964] of RS : [https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java#L241] So looks like the current design already treats some aspects of setting up the replication as important and aborts the RS if not setup properly as opposed to our thought of letting RS accept requests even if replication fails in an async thread Should we consider the thought of delaying adding this RS to availableServers on Master till it is actually ready to accept requests ? Thoughts ? > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Sandeep Guggilam >Priority: Major > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess
apurtell commented on pull request #2594: URL: https://github.com/apache/hbase/pull/2594#issuecomment-719038355 I'm not sure what happened here. Applying the commit locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess
apurtell merged pull request #2594: URL: https://github.com/apache/hbase/pull/2594 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514580314
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
##
@@ -182,8 +183,20 @@ public void newDead(ServerName sn) {
}
private void spawnRenewalChore(final UserGroupInformation user) {
-authService = new ChoreService("Relogin service");
-authService.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ChoreService service = getChoreService();
+service.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ }
+
+ /**
+ * If choreService has not been created yet, create the ChoreService.
+ * It is not thread safe.
+ * @return ChoreService
+ */
+ ChoreService getChoreService() {
Review comment:
Yeah, it is only called in the context of constructor. To start the
service in the constructor, it needs to check if meta replica load balancer is
enabled. Duo has a comment to avoid checking multiple configs in
AsyncConnectionImpl and start the service on demand. Tough there is no
multithread requirement for now, I think it makes sense to add synchronized.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2
Apache-HBase commented on pull request #2596: URL: https://github.com/apache/hbase/pull/2596#issuecomment-719035664 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 39s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 0s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | checkstyle | 1m 40s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 56s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 44m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2596 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux b59c48d6f54b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 69f282e21a | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2596/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell merged pull request #2578: HBASE-24768 Clear cached service kerberos ticket in case of SASL fail…
apurtell merged pull request #2578: URL: https://github.com/apache/hbase/pull/2578 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
saintstack commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514543049
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
##
@@ -182,8 +183,20 @@ public void newDead(ServerName sn) {
}
private void spawnRenewalChore(final UserGroupInformation user) {
-authService = new ChoreService("Relogin service");
-authService.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ChoreService service = getChoreService();
+service.scheduleChore(AuthUtil.getAuthRenewalChore(user));
+ }
+
+ /**
+ * If choreService has not been created yet, create the ChoreService.
+ * It is not thread safe.
+ * @return ChoreService
+ */
+ ChoreService getChoreService() {
Review comment:
Do we need this?
It is called from the constructor only. If so, just create choreservice in
the constructor.
If getChoreService can be called from elsewhere, then does this need to be
synchronized so we don't make multiple chore service instances?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -196,8 +202,43 @@ private boolean tryComplete(LocateRequest req,
CompletableFuture {
+int numOfReplicas = 1;
+try {
+ RegionLocations metaLocations =
conn.registry.getMetaRegionLocations().get(
+GET_META_LOCATIONS_TIMEOUT, TimeUnit.MILLISECONDS);
+ numOfReplicas = metaLocations.size();
+} catch (Exception e) {
+ LOG.error("Failed to get table {}'s region replication, ",
META_TABLE_NAME, e);
+}
+return numOfReplicas;
+ });
+break;
+ case None:
+// If user does not configure LOCATOR_META_REPLICAS_MODE, let's check
the legacy config.
+if (this.metaReplicaMode == CatalogReplicaMode.None) {
+ boolean useMetaReplicas =
conn.getConfiguration().getBoolean(USE_META_REPLICAS,
+DEFAULT_USE_META_REPLICAS);
+ if (useMetaReplicas) {
+this.metaReplicaMode = CatalogReplicaMode.HedgedRead;
Review comment:
None is not the same as hedged read, right?
If user configures hedged read as metaReplicaMode, they fall through to
default which 'does nothing'. That don't seem right. Shouldn't it have a case
here?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionLocator.java
##
@@ -37,6 +37,12 @@
*/
@InterfaceAudience.Public
public interface RegionLocator extends Closeable {
+
+ String LOCATOR_META_REPLICAS_MODE = "hbase.locator.meta.replicas.mode";
Review comment:
Needs javadoc to at least point at what these are about.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -433,9 +474,24 @@ private void locateInMeta(TableName tableName,
LocateRequest req) {
Scan scan = new Scan().withStartRow(metaStartKey).withStopRow(metaStopKey,
true)
.addFamily(HConstants.CATALOG_FAMILY).setReversed(true).setCaching(locatePrefetchLimit)
.setReadType(ReadType.PREAD);
-if (useMetaReplicas) {
- scan.setConsistency(Consistency.TIMELINE);
+
+switch (this.metaReplicaMode) {
+ case LoadBalance:
+int metaReplicaId = this.metaReplicaSelector.select(tableName,
req.row, req.locateType);
+if (metaReplicaId != RegionInfo.DEFAULT_REPLICA_ID) {
+ // If the selector gives a non-primary meta replica region, then go
with it.
+ // Otherwise, just go to primary in non-hedgedRead mode.
+ scan.setConsistency(Consistency.TIMELINE);
+ scan.setReplicaId(metaReplicaId);
+}
+break;
+ case HedgedRead:
+scan.setConsistency(Consistency.TIMELINE);
+break;
+ default:
+// do nothing
Review comment:
Yeah, said this before but in follow-on, would be good to shove all this
stuff into a CatalogReplicaMode class. Internally this class would figure which
policy to run. It would have a method that took a Scan that allowed decorating
the Scan w/ whatever the mode needed to implement its policy. Later.
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
if (!canUpdateOnError(loc, oldLoc)) {
return;
}
+ // Tell metaReplicaSelector that the location is stale. It will create a
stale entry
+ // with timestamp internally. Next time the client looks up the same
location,
+ // it will pick a different meta replica region. For the current
implementation,
+ // the metaReplicaId is not used, so the primary one is passed in.
+ if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// TODO: pass
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-719012161 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 2m 47s | master passed | | +1 :green_heart: | shadedjars | 6m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 2m 46s | the patch passed | | +1 :green_heart: | javac | 2m 46s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 167m 33s | root in the patch passed. | | | | 204m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bf321ea0cd9e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/testReport/ | | Max. process+thread count | 7441 (vs. ulimit of 3) | | modules | C: hbase-shell . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request #2596: Backport "HBASE-24419 Normalizer merge plans should consider more than 2 region…" to branch-2
ndimiduk opened a new pull request #2596: URL: https://github.com/apache/hbase/pull/2596 …s when possible The core change here is to the loop in `SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested loop that walks the table's region chain once, looking for contiguous sequences of regions that meet the criteria for merge. The outer loop tracks the starting point of the next sequence, the inner loop looks for the end of that sequence. A single sequence becomes an instance of `MergeNormalizationPlan`. Signed-off-by: Huaxiang Sun This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
ndimiduk merged pull request #2490: URL: https://github.com/apache/hbase/pull/2490 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-719002752 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | checkstyle | 2m 11s | master passed | | +1 :green_heart: | spotbugs | 4m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | -0 :warning: | checkstyle | 0m 33s | hbase-client: The patch generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) | | -0 :warning: | checkstyle | 1m 23s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 60m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 8c8f9a206e7a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25090) CompactionConfiguration logs unrealistic store file sizes
[
https://issues.apache.org/jira/browse/HBASE-25090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-25090.
---
Fix Version/s: 2.4.0
3.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
Merged to master and branch-2. Thanks for the PR [~GeorryHuang]
> CompactionConfiguration logs unrealistic store file sizes
> -
>
> Key: HBASE-25090
> URL: https://issues.apache.org/jira/browse/HBASE-25090
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.3.2
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> I noticed this in the debug logging of a unit test that makes use of a
> MiniCluster. I'm pretty sure {{8.00 EB}} -- exabytes?? -- is not an accurate
> reporting of the size of these HFiles.
> {noformat}
> 2020-09-22 16:20:57,996 INFO
> [StoreOpener-5e0b426a952230abb65623fc54869833-1]
> compactions.CompactionConfiguration(177): size [128 MB, 8.00 EB, 8.00 EB);
> files [3, 10); ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2595: HBASE-25090 CompactionConfiguration logs unrealistic store file sizes
saintstack merged pull request #2595: URL: https://github.com/apache/hbase/pull/2595 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
sandeepvinayak commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514528418
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[]
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+ * @param encodedRegionName The region ID.
+ * @param length Size of region in bytes
*/
public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
- final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName,
length);
Review comment:
@Reidd Scan is not useless for all the input formats of map-reduce,
it is for the `TableInputFormat`.
We do need the scan object for `MultiTableInputFormat`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] sandeepvinayak commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
sandeepvinayak commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514525613
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
##
@@ -271,6 +272,11 @@ public void testNumOfSplits(int splitsPerRegion, int
expectedNumOfSplits) throws
TableInputFormat tif = new TableInputFormat();
tif.setConf(job.getConfiguration());
List splits = tif.getSplits(job);
+for (InputSplit split : splits) {
+ TableSplit tableSplit = (TableSplit) split;
+ Assert.assertEquals(tableSplit.getScan().getStartRow(),
HConstants.EMPTY_START_ROW);
Review comment:
I see your point, thanks !
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-718982958 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 146m 58s | hbase-server in the patch passed. | | | | 177m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1eaa5f30ffd7 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/testReport/ | | Max. process+thread count | 4348 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2523: HBASE-25167 Normalizer support for hot config reloading
Apache-HBase commented on pull request #2523: URL: https://github.com/apache/hbase/pull/2523#issuecomment-718979616 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 46s | master passed | | +1 :green_heart: | spotbugs | 3m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | checkstyle | 0m 24s | hbase-common: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | checkstyle | 1m 15s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 48s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 48m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2523 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux db187a2cc5e7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2523/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-718978987 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 22s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 34s | master passed | | +1 :green_heart: | compile | 1m 47s | master passed | | +1 :green_heart: | shadedjars | 6m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 31s | the patch passed | | +1 :green_heart: | javac | 1m 31s | the patch passed | | +1 :green_heart: | shadedjars | 6m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 33s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 135m 36s | hbase-server in the patch passed. | | | | 169m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f16918938d8d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/testReport/ | | Max. process+thread count | 4807 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on a change in pull request #2584:
URL: https://github.com/apache/hbase/pull/2584#discussion_r514512363
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java
##
@@ -577,6 +635,15 @@ private void removeLocationFromCache(HRegionLocation loc) {
if (!canUpdateOnError(loc, oldLoc)) {
return;
}
+ // Tell metaReplicaSelector that the location is stale. It will create a
stale entry
+ // with timestamp internally. Next time the client looks up the same
location,
+ // it will pick a different meta replica region. For the current
implementation,
+ // the metaReplicaId is not used, so the primary one is passed in.
+ if (this.metaReplicaMode == CatalogReplicaMode.LoadBalance) {
+// metaReplicaId is not used in simpleSelector, default to the primary
one.
+metaReplicaSelector.onError(loc, RegionInfo.DEFAULT_REPLICA_ID);
Review comment:
I pushed a patch with fromReplicaId set to -1. Work on an add-on patch
which will fill in this metaReplicaId.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25223) Use try-with-resources statement in snapshot package
[
https://issues.apache.org/jira/browse/HBASE-25223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223156#comment-17223156
]
Hudson commented on HBASE-25223:
Results for branch branch-2
[build #87 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use try-with-resources statement in snapshot package
>
>
> Key: HBASE-25223
> URL: https://issues.apache.org/jira/browse/HBASE-25223
> Project: HBase
> Issue Type: Improvement
>Reporter: minjikim
>Assignee: minjikim
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> Using try-with-resources statement makes the code simpler.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25224) Maximize sleep for checking meta and namespace regions availability
[
https://issues.apache.org/jira/browse/HBASE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223157#comment-17223157
]
Hudson commented on HBASE-25224:
Results for branch branch-2
[build #87 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/87/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Maximize sleep for checking meta and namespace regions availability
> ---
>
> Key: HBASE-25224
> URL: https://issues.apache.org/jira/browse/HBASE-25224
> Project: HBase
> Issue Type: Improvement
> Components: master
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> The isRegionOnline method in HMaster is used on Master startup to check the
> availability of hbase:meta and hbase:namespace tables.
> I've run into an issue when namespace was not online and Master was just
> waiting there. I've used HBCK2 to fix the cluster but the initialization was
> not completed because RetryCounterFactory was already waiting for 10+ hours.
> Since Master is waiting in an idle state it makes no harm to check the region
> availability more frequently and limit the maximum sleep time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #2523: HBASE-25167 Normalizer support for hot config reloading
ndimiduk commented on a change in pull request #2523:
URL: https://github.com/apache/hbase/pull/2523#discussion_r514493750
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -71,38 +76,78 @@
static final String MERGE_MIN_REGION_SIZE_MB_KEY =
"hbase.normalizer.merge.min_region_size.mb";
static final int DEFAULT_MERGE_MIN_REGION_SIZE_MB = 1;
- private Configuration conf;
private MasterServices masterServices;
- private boolean splitEnabled;
- private boolean mergeEnabled;
- private int minRegionCount;
- private Period mergeMinRegionAge;
- private int mergeMinRegionSizeMb;
+
+ /** Ensure configuration changes are applied atomically. */
+ private final ReadWriteLock configUpdateLock = new ReentrantReadWriteLock();
+ @GuardedBy("configUpdateLock") private Configuration conf;
+ @GuardedBy("configUpdateLock") private boolean splitEnabled;
+ @GuardedBy("configUpdateLock") private boolean mergeEnabled;
+ @GuardedBy("configUpdateLock") private int minRegionCount;
+ @GuardedBy("configUpdateLock") private Period mergeMinRegionAge;
+ @GuardedBy("configUpdateLock") private int mergeMinRegionSizeMb;
public SimpleRegionNormalizer() {
-splitEnabled = DEFAULT_SPLIT_ENABLED;
-mergeEnabled = DEFAULT_MERGE_ENABLED;
-minRegionCount = DEFAULT_MIN_REGION_COUNT;
-mergeMinRegionAge = Period.ofDays(DEFAULT_MERGE_MIN_REGION_AGE_DAYS);
-mergeMinRegionSizeMb = DEFAULT_MERGE_MIN_REGION_SIZE_MB;
+final Lock writeLock = configUpdateLock.writeLock();
+writeLock.lock();
+try {
+ splitEnabled = DEFAULT_SPLIT_ENABLED;
+ mergeEnabled = DEFAULT_MERGE_ENABLED;
+ minRegionCount = DEFAULT_MIN_REGION_COUNT;
+ mergeMinRegionAge = Period.ofDays(DEFAULT_MERGE_MIN_REGION_AGE_DAYS);
+ mergeMinRegionSizeMb = DEFAULT_MERGE_MIN_REGION_SIZE_MB;
+} finally {
+ writeLock.unlock();
+}
}
@Override
public Configuration getConf() {
-return conf;
+final Lock readLock = configUpdateLock.readLock();
+readLock.lock();
+try {
+ return conf;
+} finally {
+ readLock.unlock();
+}
}
@Override
public void setConf(final Configuration conf) {
if (conf == null) {
return;
}
-this.conf = conf;
-splitEnabled = conf.getBoolean(SPLIT_ENABLED_KEY, DEFAULT_SPLIT_ENABLED);
-mergeEnabled = conf.getBoolean(MERGE_ENABLED_KEY, DEFAULT_MERGE_ENABLED);
-minRegionCount = parseMinRegionCount(conf);
-mergeMinRegionAge = parseMergeMinRegionAge(conf);
-mergeMinRegionSizeMb = parseMergeMinRegionSizeMb(conf);
+
+final Lock writeLock = configUpdateLock.writeLock();
+writeLock.lock();
Review comment:
Sure @bharathv , I think that's a nice suggestion. I agree that the
strict ordering guarantees provided by explicit locking are not needed here.
I've pushed a new commit that unwinds the locking and uses atomic instance
assignment as you suggest. Let me know what you think.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2594: HBASE-25227 [branch-1] Fix Java 11 runtime issue in UnsafeAccess
Apache-HBase commented on pull request #2594: URL: https://github.com/apache/hbase/pull/2594#issuecomment-718933092 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 58s | branch-1 passed | | +1 :green_heart: | compile | 0m 19s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 23s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 0m 41s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 23s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 1m 27s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 25s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 56s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 23s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 37s | hbase-common in the patch passed. | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 48m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2594 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c43bf1b1ee01 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2594/out/precommit/personality/provided.sh | | git revision | branch-1 / b30d1d1 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/testReport/ | | Max. process+thread count | 165 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2594/3/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
Apache-HBase commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-718918341 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | -0 :warning: | rubocop | 0m 4s | The patch generated 8 new + 24 unchanged - 1 fixed = 32 total (was 25) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 17m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 26m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2308 | | Optional Tests | dupname asflicense rubocop hadoopcheck xml | | uname | Linux d65df110ab6c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | rubocop | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 76 (vs. ulimit of 3) | | modules | C: . hbase-shell U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2308/18/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) rubocop=0.80.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2385: HBASE-24950 Another solution for Splittable Meta
Apache-HBase commented on pull request #2385: URL: https://github.com/apache/hbase/pull/2385#issuecomment-718914945 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 51s | master passed | | +1 :green_heart: | compile | 4m 29s | master passed | | +1 :green_heart: | shadedjars | 7m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 31s | master passed | | -0 :warning: | patch | 11m 0s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 57s | the patch passed | | +1 :green_heart: | compile | 4m 13s | the patch passed | | +1 :green_heart: | javac | 4m 13s | the patch passed | | +1 :green_heart: | shadedjars | 7m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | javadoc | 0m 24s | hbase-common in the patch passed. | | +1 :green_heart: | javadoc | 0m 25s | hbase-client in the patch passed. | | +1 :green_heart: | javadoc | 0m 16s | hbase-zookeeper in the patch passed. | | +1 :green_heart: | javadoc | 0m 16s | hbase-balancer generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) | | +1 :green_heart: | javadoc | 0m 41s | hbase-server in the patch passed. | | +1 :green_heart: | javadoc | 0m 14s | hbase-it in the patch passed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 2s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 10s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 16s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 44s | hbase-zookeeper in the patch passed. | | +1 :green_heart: | unit | 0m 22s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 195m 30s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 5s | hbase-it in the patch passed. | | | | 245m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2385 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d7f20d92ccca 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bb4a9d335f | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/testReport/ | | Max. process+thread count | 3447 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-balancer hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2385/14/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2490: HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
Apache-HBase commented on pull request #2490: URL: https://github.com/apache/hbase/pull/2490#issuecomment-718913446 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | checkstyle | 1m 42s | master passed | | +1 :green_heart: | spotbugs | 3m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 16s | the patch passed | | +1 :green_heart: | checkstyle | 1m 40s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 48m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2490 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux a7990961c13f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 259fe1984a | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2490/7/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess
[ https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25227: Summary: [branch-1] Fix Java 11 runtime issue in UnsafeAccess (was: [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue) > [branch-1] Fix Java 11 runtime issue in UnsafeAccess > > > Key: HBASE-25227 > URL: https://issues.apache.org/jira/browse/HBASE-25227 > Project: HBase > Issue Type: Bug >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Fix For: 1.7.0 > > > When running tests with Java 11 UnsafeAccess is observed to throw > NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them > as parameters to Unsafe methods which should take 'long'. The Java 8 compiler > does the implicit conversion but the Java 11 compiler does not. Add casts to > fix. Not an issue on branch-2 and up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25227) [branch-1] Fix Java 11 runtime issue in UnsafeAccess
[ https://issues.apache.org/jira/browse/HBASE-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25227: Description: When running tests with Java 11 UnsafeAccess is observed to throw NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them as parameters to Unsafe methods which should take 'long'. Not an issue on branch-2 and up. (was: When running tests with Java 11 UnsafeAccess is observed to throw NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them as parameters to Unsafe methods which should take 'long'. The Java 8 compiler does the implicit conversion but the Java 11 compiler does not. Add casts to fix. Not an issue on branch-2 and up.) > [branch-1] Fix Java 11 runtime issue in UnsafeAccess > > > Key: HBASE-25227 > URL: https://issues.apache.org/jira/browse/HBASE-25227 > Project: HBase > Issue Type: Bug >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Fix For: 1.7.0 > > > When running tests with Java 11 UnsafeAccess is observed to throw > NoSuchMethodErrors. Some of our methods accept 'int' parameters and use them > as parameters to Unsafe methods which should take 'long'. Not an issue on > branch-2 and up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue
apurtell commented on pull request #2594: URL: https://github.com/apache/hbase/pull/2594#issuecomment-718905126 Change uses @Reidd and @anoopsjohn suggestion instead of casts This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar commented on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar commented on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-718901775 Why is this so flaky? Different 5 tests this time, only 1 test last time :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar edited a comment on pull request #2308: HBASE-20598 - Upgrade to JRuby 9.2
nkalmar edited a comment on pull request #2308: URL: https://github.com/apache/hbase/pull/2308#issuecomment-718901775 Why is this so flaky? Different 5 tests this time, only 1 test last time :( All of them seems to be timeout issues. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25224) Maximize sleep for checking meta and namespace regions availability
[
https://issues.apache.org/jira/browse/HBASE-25224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223065#comment-17223065
]
Hudson commented on HBASE-25224:
Results for branch branch-2.3
[build #93 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/93/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Maximize sleep for checking meta and namespace regions availability
> ---
>
> Key: HBASE-25224
> URL: https://issues.apache.org/jira/browse/HBASE-25224
> Project: HBase
> Issue Type: Improvement
> Components: master
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> The isRegionOnline method in HMaster is used on Master startup to check the
> availability of hbase:meta and hbase:namespace tables.
> I've run into an issue when namespace was not online and Master was just
> waiting there. I've used HBCK2 to fix the cluster but the initialization was
> not completed because RetryCounterFactory was already waiting for 10+ hours.
> Since Master is waiting in an idle state it makes no harm to check the region
> availability more frequently and limit the maximum sleep time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #2591: HBASE-24859: Optimize in-memory representation of HBase map reduce table splits
bharathv commented on a change in pull request #2591:
URL: https://github.com/apache/hbase/pull/2591#discussion_r514426736
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
##
@@ -271,6 +272,11 @@ public void testNumOfSplits(int splitsPerRegion, int
expectedNumOfSplits) throws
TableInputFormat tif = new TableInputFormat();
tif.setConf(job.getConfiguration());
List splits = tif.getSplits(job);
+for (InputSplit split : splits) {
+ TableSplit tableSplit = (TableSplit) split;
+ Assert.assertEquals(tableSplit.getScan().getStartRow(),
HConstants.EMPTY_START_ROW);
Review comment:
Ya my point is, that is a weak check. You wouldn't know if that scan is
an empty scan or a default scan. Change the code to something like this and you
can see there is a bug ...
```
for (InputSplit split : splits) {
TableSplit tableSplit = (TableSplit) split;
assertTrue(tableSplit.getScanAsString().isEmpty());
}
In table split code, add this...
public String getScanAsString() {
return scan;
}
```
Run this test
```
@Test
public void testGetSplits() throws IOException, InterruptedException,
ClassNotFoundException {
testNumOfSplits(1, 26);
testNumOfSplits(3, 78); <
}
```
If number of splits per region is > 1, your patch didn't handle the
TableSplit constructor in this call createNInputSplitsUniform(). Your test
still passes because you are asserting on a default Scan object and not the
fact that it shouldn't be set in the first place.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableSplit.java
##
@@ -184,9 +184,24 @@ public TableSplit(final byte [] tableName, byte[]
startRow, byte[] endRow,
* @param startRow The start row of the split.
* @param endRow The end row of the split.
* @param location The location of the region.
+ * @param encodedRegionName The region ID.
+ * @param length Size of region in bytes
*/
public TableSplit(TableName tableName, byte[] startRow, byte[] endRow,
- final String location) {
+final String location, final String encodedRegionName, long length) {
+this(tableName, null, startRow, endRow, location, encodedRegionName,
length);
Review comment:
> I think the separate constructor is more intuitive that it is okay to
not have scan object? What do you think?
IMHO more constructors is less readability. In this case we have 8
constructors for TableSplit. If I were to use it, I'd confused unless I see the
usages of each of these. Either we should have a [fluent
interface](https://en.wikipedia.org/wiki/Fluent_interface) or fewer
constructors is what I think. Subjective of course.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966 @Apache9 I am having trouble responding to your comment above. > After reading this discussion several times, I think the reason we do not want to niterrupte a WAL sync is that it may lead to a region server abort? > I would say this is not the case here. I checked the code again, the actual sync is done in the disruptor thread, in the rpc thread we just block on a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will just lead to an IOException tp client, the actual sync operation will not be interrupted so we are safe. > So I do not think we need to disable interrupts here? You are correct about the SyncFuture. Initially my thinking was the same as yours. At some point I became concerned about this case, though: - We start the WAL append - SyncFuture is interrupted, client gets an exception, client thinks the mutation failed - WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so we are at risk of the failed from client perspective mutation being applied during WAL replay for some reason So we have a case where the client's understanding of what happened is incorrect. What happens in the WAL and the client's understanding of what happend should be in sync. If the WAL commit fails the client should get an exception. If the WAL commit succeeds the client should see success. Whether we interrupt or not is our option. disableInterrupt/enableInterrupt here is consideration for the client. I can add the above as a code comment next to the disableInterrupt() call, would that help? Here is another case of concern: - We start the WAL append - SyncFuture is interrupted, memstore is not updated, client gets an exception. Unless the WAL is replayed the local cluster A will not have the mutation applied. - WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so it is shipped to the remote cluster, and is applied at the remote cluster B. - Now the data in cluster A and B are out of sync. The disableInterrupt/enableInterrupt pair combines the WAL append and memstore update into a single protected operation , so this error case will not happen because of the interrupts I am adding with this patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2594: HBASE-25227 [branch-1] Cast in UnsafeAccess to avoid Java 11 runtime issue
apurtell commented on pull request #2594: URL: https://github.com/apache/hbase/pull/2594#issuecomment-718889835 > directly changing BYTE_ARRAY_BASE_OFFSET's type to long Ok This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-718875966 @Apache9 I am having trouble responding to your comment above. > After reading this discussion several times, I think the reason we do not want to niterrupte a WAL sync is that it may lead to a region server abort? > I would say this is not the case here. I checked the code again, the actual sync is done in the disruptor thread, in the rpc thread we just block on a SyncFuture(as Andrew mentioned above), the interruption on the rpc thread will just lead to an IOException tp client, the actual sync operation will not be interrupted so we are safe. > So I do not think we need to disable interrupts here? You are correct about the SyncFuture. Initially my thinking was the same as yours. At some point I became concerned about this case, though: - We start the WAL append - SyncFuture is interrupted, client gets an exception, client thinks the mutation failed - WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so we are at risk of the failed from client perspective mutation being applied during WAL replay for some reason So we have a case where the client's understanding of what happened is incorrect. What happens in the WAL and the client's understanding of what happend should be in sync. If the WAL commit fails the client should get an exception. If the WAL commit succeeds the client should see success. Whether we interrupt or not is our option. disableInterrupt/enableInterrupt here is consideration for the client. I can add the above as a code comment next to the disableInterrupt() call, would that help? Here is another case of concern: - We start the WAL append - SyncFuture is interrupted, memstore is not updated, client gets an exception. Unless the WAL is replayed the local cluster A will not have the mutation applied. - WALedit is actually applied to the WAL by the disruptor, so the mutation is included in the WAL, and so it is shipped to the remote cluster, and is applied at the remote cluster B. - Now the data in cluster A and B are out of sync. The disableInterrupt/enableInterrupt pair combines the WAL append and memstore update into a single protected operation , so this error case cannot happen. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
