[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777209816 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 39s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 7m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 199m 36s | hbase-server in the patch passed. | | | | 230m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8ba84d171741 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/testReport/ | | Max. process+thread count | 3622 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777194224 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 6m 40s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 35s | hbase-server generated 5 new + 16 unchanged - 0 fixed = 21 total (was 16) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 142m 54s | hbase-server in the patch passed. | | | | 168m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f4d6ac3a2a66 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/testReport/ | | Max. process+thread count | 4638 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777189797 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 7m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 7m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 143m 33s | hbase-server in the patch passed. | | | | 175m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 47f86ad9e457 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/testReport/ | | Max. process+thread count | 4569 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777187779 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 6s | the patch passed | | +1 :green_heart: | compile | 1m 28s | the patch passed | | +1 :green_heart: | javac | 1m 28s | the patch passed | | +1 :green_heart: | shadedjars | 6m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 35s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 135m 17s | hbase-server in the patch passed. | | | | 165m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7d9f8b201dab 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/testReport/ | | Max. process+thread count | 3914 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777166133 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 3m 21s | master passed | | +1 :green_heart: | checkstyle | 1m 16s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | compile | 3m 16s | the patch passed | | +1 :green_heart: | javac | 3m 16s | the patch passed | | -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 2 new + 163 unchanged - 0 fixed = 165 total (was 163) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 58s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 49m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8a53446e2f61 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777158999 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 3m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | compile | 3m 36s | the patch passed | | +1 :green_heart: | javac | 3m 36s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-hadoop-compat | | +1 :green_heart: | checkstyle | 1m 3s | hbase-server: The patch generated 0 new + 5 unchanged - 1 fixed = 5 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 15s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 48m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 11abf6aafd67 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777151546 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 51s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 7m 28s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | shadedjars | 7m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 199m 33s | hbase-server in the patch passed. | | | | 230m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1427a4898c6c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/testReport/ | | Max. process+thread count | 3474 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
shahrs87 commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777144306 @bharathv @gjacoby126 could you guys also help reviewing this change. Thank you ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777132417 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | shadedjars | 6m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 36s | hbase-server generated 5 new + 16 unchanged - 0 fixed = 21 total (was 16) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 142m 48s | hbase-server in the patch passed. | | | | 168m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f8f2135be814 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/testReport/ | | Max. process+thread count | 4715 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777090505 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 6m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 32s | hbase-hadoop-compat in the patch passed. | | -1 :x: | unit | 147m 15s | hbase-server in the patch failed. | | | | 175m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3f97cc747905 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/testReport/ | | Max. process+thread count | 4096 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777086654 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 12s | master passed | | +1 :green_heart: | compile | 1m 27s | master passed | | +1 :green_heart: | shadedjars | 6m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 4s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 35s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 135m 56s | hbase-server in the patch passed. | | | | 166m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9ba5defa2096 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/testReport/ | | Max. process+thread count | 4090 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2922: HBASE-25547: Thread pools should release unused resources
Apache-HBase commented on pull request #2922: URL: https://github.com/apache/hbase/pull/2922#issuecomment-777078974 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 3m 20s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | compile | 3m 17s | the patch passed | | +1 :green_heart: | javac | 3m 17s | the patch passed | | -0 :warning: | checkstyle | 1m 14s | hbase-server: The patch generated 2 new + 163 unchanged - 0 fixed = 165 total (was 163) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 3s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 50m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2922 | | JIRA Issue | HBASE-25547 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux efe24478ff06 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2922/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #2924: HBASE-25549 A new hbase shell command: 'alter_lazy'
bharathv commented on pull request #2924: URL: https://github.com/apache/hbase/pull/2924#issuecomment-777060579 @GeorryHuang I added a comment in the jira. IMO its good to roll "async" behavior into the actual alter command implementation. The original commit that added it is a decade old so I'm guessing very few people around here have any context of it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #2922: HBASE-25547: Thread pools should release unused resources
bharathv commented on a change in pull request #2922:
URL: https://github.com/apache/hbase/pull/2922#discussion_r574098579
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
##
@@ -81,15 +82,16 @@ public ExecutorService(final String servername) {
/**
* Start an executor service with a given name. If there was a service
already
* started with the same name, this throws a RuntimeException.
- * @param name Name of the service to start.
+ * @param config Configuration to use for the executor.
*/
- public void startExecutorService(String name, int maxThreads) {
+ public void startExecutorService(final ExecutorConfig config) {
Review comment:
Updated.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
##
@@ -182,12 +182,60 @@ public void delayedSubmit(EventHandler eh, long delay,
TimeUnit unit) {
return ret;
}
+ /**
+ * Configuration wrapper for {@link Executor}.
+ */
+ public static class ExecutorConfig {
+// Refer to ThreadPoolExecutor javadoc for details of these configuration.
+// Argument validation and bound checks delegated to the underlying
ThreadPoolExecutor
+// implementation.
Review comment:
@wchevreuil @virajjasani I lost the old comments after a force push. I
think I updated the code to get rid of maxPoolSize which is not honored for
unbounded taskQueue (thanks Wellington)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25534) Honor TableDescriptor settings earlier in normalization
[
https://issues.apache.org/jira/browse/HBASE-25534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282735#comment-17282735
]
Hudson commented on HBASE-25534:
Results for branch branch-2.4
[build #49 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/49/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/49/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/49/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/49/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/49/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Honor TableDescriptor settings earlier in normalization
> ---
>
> Key: HBASE-25534
> URL: https://issues.apache.org/jira/browse/HBASE-25534
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 3.0.0-alpha-1, 2.4.2
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.2
>
>
> -Table can be enabled and disabled merge/split in TableDescriptor, we should
> judge before calculating the plan.-
> The normalizer configuration can be set by table level. For example,
> hbase.normalizer.min.region.count can be set by "alter ‘table’,
> CONFIGURATION=>\{'hbase.normalizer.min.region.count' => '5'}". If the table
> is not set, then use the global configuration which is set in hbase-site.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-777025120 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 25s | master passed | | +1 :green_heart: | compile | 3m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 16s | master passed | | +1 :green_heart: | spotbugs | 2m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 32s | the patch passed | | +1 :green_heart: | compile | 3m 38s | the patch passed | | +1 :green_heart: | javac | 3m 38s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-hadoop-compat | | +1 :green_heart: | checkstyle | 1m 5s | hbase-server: The patch generated 0 new + 5 unchanged - 1 fixed = 5 total (was 6) | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 48m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d5e79eb562e4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25364) Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process
[
https://issues.apache.org/jira/browse/HBASE-25364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282703#comment-17282703
]
Hudson commented on HBASE-25364:
Results for branch branch-2
[build #175 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/175//console].
> Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison
> process
> -
>
> Key: HBASE-25364
> URL: https://issues.apache.org/jira/browse/HBASE-25364
> Project: HBase
> Issue Type: Improvement
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> There is a TODO like this "TODO: Redo so only a single pass over the arrays
> rather than one to compare and then a second composing midpoint." in
> getMidpoint() of class HFileWriteImpl
>
> The old logic compares the left byte array and the right byte array twice：
> # A comparison is performed before composing MinimumMidpointArray
> # During composing of MinimumMidpointArray, bytes were comparing again
>
> My optimization combines them into one
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-site] apurtell commented on pull request #7: HBASE-25563 Add "2.4 Documentation" to the website
apurtell commented on pull request #7: URL: https://github.com/apache/hbase-site/pull/7#issuecomment-776991616 Generated by: hbase $ git checkout rel/2.4.0 hbase $ mvn clean install -DskipTests hbase $ mvn site site:stage -DskipTests hbase-site $ git checkout asf-site hbase-site $ cp -r ../hbase/target/staging ./2.4 hbase-site $ git add 2.4 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25563) Add "2.4 Documentation" to the website
[ https://issues.apache.org/jira/browse/HBASE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25563: Status: Patch Available (was: Open) > Add "2.4 Documentation" to the website > -- > > Key: HBASE-25563 > URL: https://issues.apache.org/jira/browse/HBASE-25563 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Andrew Kyle Purtell >Priority: Major > > Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25563) Add "2.4 Documentation" to the website
[ https://issues.apache.org/jira/browse/HBASE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282682#comment-17282682 ] Andrew Kyle Purtell commented on HBASE-25563: - Opened PR here: https://github.com/apache/hbase-site/pull/7 Not sure if it's the right place. > Add "2.4 Documentation" to the website > -- > > Key: HBASE-25563 > URL: https://issues.apache.org/jira/browse/HBASE-25563 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Andrew Kyle Purtell >Priority: Major > > Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-site] apurtell opened a new pull request #7: HBASE-25563 Add "2.4 Documentation" to the website
apurtell opened a new pull request #7: URL: https://github.com/apache/hbase-site/pull/7 Signed-off-by: Andrew Purtell This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
shahrs87 commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-776977524 @saintstack This is ready for review now. Please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-776975930 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 44s | master passed | | +1 :green_heart: | compile | 1m 17s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 30s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 33s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 142m 14s | hbase-server in the patch passed. | | | | 171m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 10615d9ff343 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/testReport/ | | Max. process+thread count | 4092 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-776973697 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | shadedjars | 6m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | shadedjars | 6m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 36s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 136m 24s | hbase-server in the patch passed. | | | | 167m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 39e2cf8c3506 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-11.0.6+10 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/testReport/ | | Max. process+thread count | 4047 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-25563) Add "2.4 Documentation" to the website
[ https://issues.apache.org/jira/browse/HBASE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reassigned HBASE-25563: --- Assignee: Andrew Kyle Purtell > Add "2.4 Documentation" to the website > -- > > Key: HBASE-25563 > URL: https://issues.apache.org/jira/browse/HBASE-25563 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Assignee: Andrew Kyle Purtell >Priority: Major > > Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25519) BLOCKSIZE needs to support pretty print
[
https://issues.apache.org/jira/browse/HBASE-25519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282622#comment-17282622
]
Hudson commented on HBASE-25519:
Results for branch branch-2
[build #174 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/174/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/174/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/174/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> BLOCKSIZE needs to support pretty print
> ---
>
> Key: HBASE-25519
> URL: https://issues.apache.org/jira/browse/HBASE-25519
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> HBASE-25439 added a new unit in PrettyPrint.Unit, the BLOCKSIZE in CF should
> also support this feature.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2945: [HBASE-25539] Add age of oldest wal metric
Apache-HBase commented on pull request #2945: URL: https://github.com/apache/hbase/pull/2945#issuecomment-776897415 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 3m 39s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | +1 :green_heart: | compile | 3m 35s | the patch passed | | +1 :green_heart: | javac | 3m 35s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 6 new + 5 unchanged - 1 fixed = 11 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 8s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 48m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2945 | | JIRA Issue | HBASE-25539 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3e4f706001a6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3e743df6d0 | | Default Java | AdoptOpenJDK-1.8.0_232-b09 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2945/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=3.1.12 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25559) Terminate threads of oldsources while RS is closing
[
https://issues.apache.org/jira/browse/HBASE-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282600#comment-17282600
]
Hudson commented on HBASE-25559:
Results for branch branch-2.3
[build #167 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/167/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/167/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/167/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/167/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/167/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Terminate threads of oldsources while RS is closing
> ---
>
> Key: HBASE-25559
> URL: https://issues.apache.org/jira/browse/HBASE-25559
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.2.6, 2.3.4, 2.4.1
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.5, 2.4.2
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] joshelser commented on a change in pull request #2931: HBASE-25395 Introduce PersistedStoreEngine and PersistedStoreFileManager
joshelser commented on a change in pull request #2931:
URL: https://github.com/apache/hbase/pull/2931#discussion_r573100029
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StorefileTrackingUtils.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.MetaTableAccessor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility class to support persistent storefile tracking
+ */
+@InterfaceAudience.Private
+public final class StorefileTrackingUtils {
+
+ private static Logger LOG =
LoggerFactory.getLogger(StorefileTrackingUtils.class);
+ public static final long SLEEP_DELTA_MS =
TimeUnit.MILLISECONDS.toMillis(100);
+
+ private StorefileTrackingUtils() {
+// private for utility class
+ }
+
+ public static boolean isStorefileTrackingPersistEnabled(Configuration conf) {
+boolean isStoreTrackingPersistEnabled =
+ conf.getBoolean(HConstants.STOREFILE_TRACKING_PERSIST_ENABLED,
+HConstants.DEFAULT_STOREFILE_TRACKING_PERSIST_ENABLED);
+boolean isPersistedStoreEngineSet =
+ conf.get(StoreEngine.STORE_ENGINE_CLASS_KEY,
DefaultStoreEngine.class.getName())
+.equals(PersistedStoreEngine.class.getName());
+boolean isFeatureEnabled = isStoreTrackingPersistEnabled &&
isPersistedStoreEngineSet;
+if (isStoreTrackingPersistEnabled ^ isPersistedStoreEngineSet) {
+ // check if both configuration are correct.
+ String errorMessage = String.format("please set %s to true and set store
engine key %s to %s "
+ + "to enable persist storefile tracking",
+HConstants.STOREFILE_TRACKING_PERSIST_ENABLED,
+StoreEngine.STORE_ENGINE_CLASS_KEY,
PersistedStoreEngine.class.getName());
+ throw new IllegalArgumentException(errorMessage);
+}
+return isFeatureEnabled;
+ }
+
+ /**
+ * if storefile tracking feature is configured, Initialize hbase:storefile
table and wait for it
+ * to be online. Otherwise, look for hbase:storefile table and remove it
+ *
+ * @param masterServices masterServices
+ * @throws IOException if hbase:storefile table cannot be initialized and be
online
+ */
+ public static void init(MasterServices masterServices) throws IOException {
+createStorefileTable(masterServices);
+waitForStoreFileTableOnline(masterServices);
+ }
+
+ /**
+ * Cleans up all storefile related state on the cluster. disable and delete
hbase:storefile
+ * if found
+ * @param masterServices {@link MasterServices}
+ * @throws IOException if failures
+ */
+ private static void cleanup(MasterServices masterServices) throws
IOException {
+try {
+
masterServices.getConnection().getAdmin().disableTable(TableName.STOREFILE_TABLE_NAME);
+
masterServices.getConnection().getAdmin().deleteTable(TableName.STOREFILE_TABLE_NAME);
+} catch (IOException ex) {
+ final String message = "Failed disable and deleting table " +
+TableName.STOREFILE_TABLE_NAME.getNameAsString();
+ LOG.error(message);
+ throw new IOException(message, ex);
+}
+ }
+
+ public static StoreFilePathAccessor
createStoreFilePathAccessor(Configuration conf,
+Connection connection) {
+return new HTableStoreFilePathAccessor(conf, connection);
+ }
+
+ public static List convertStoreFilesToPaths(Collection
storeFiles) {
+return
storeFiles.stream().map(HStoreFile::getPath).collect(Collectors.toList());
+ }
+
+ private static void createStorefileTable(MasterServices masterServices)
+throws IOException {
+if
[jira] [Commented] (HBASE-25563) Add "2.4 Documentation" to the website
[ https://issues.apache.org/jira/browse/HBASE-25563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282574#comment-17282574 ] Nick Dimiduk commented on HBASE-25563: -- FYI [~apurtell]. Looks like I accomplished this for 2.3.0 via instructions on this [PR|https://github.com/apache/hbase-site/pull/5]. > Add "2.4 Documentation" to the website > -- > > Key: HBASE-25563 > URL: https://issues.apache.org/jira/browse/HBASE-25563 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Priority: Major > > Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25564) Drop "2.2 Documentation" from the website
[ https://issues.apache.org/jira/browse/HBASE-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282576#comment-17282576 ] Nick Dimiduk commented on HBASE-25564: -- Oh pardon me. Maybe not EOL, but is no longer the stable branch? > Drop "2.2 Documentation" from the website > - > > Key: HBASE-25564 > URL: https://issues.apache.org/jira/browse/HBASE-25564 > Project: HBase > Issue Type: Task > Components: community, documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Nick Dimiduk >Priority: Major > > 2.2 is EOL. Let's remove the docs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25564) Drop "2.2 Documentation" from the website
Nick Dimiduk created HBASE-25564: Summary: Drop "2.2 Documentation" from the website Key: HBASE-25564 URL: https://issues.apache.org/jira/browse/HBASE-25564 Project: HBase Issue Type: Task Components: community, documentation Affects Versions: 3.0.0-alpha-1 Reporter: Nick Dimiduk 2.2 is EOL. Let's remove the docs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25563) Add "2.4 Documentation" to the website
Nick Dimiduk created HBASE-25563: Summary: Add "2.4 Documentation" to the website Key: HBASE-25563 URL: https://issues.apache.org/jira/browse/HBASE-25563 Project: HBase Issue Type: Task Components: community, documentation Affects Versions: 3.0.0-alpha-1 Reporter: Nick Dimiduk Docs from the 2.4.0 build should be slotted into the website. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on pull request #2747: HBASE-24850 CellComparator perf improvement
saintstack commented on pull request #2747: URL: https://github.com/apache/hbase/pull/2747#issuecomment-776839762 @ramkrish86 Did you get a chance to read through the above (and then close this)? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25364) Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process
[ https://issues.apache.org/jira/browse/HBASE-25364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25364. --- Fix Version/s: 2.5.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Thank you for the nice improvement [~GeorryHuang] (thanks for thorough review [~niuyulin]). Merged to branch-2 and master. > Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison > process > - > > Key: HBASE-25364 > URL: https://issues.apache.org/jira/browse/HBASE-25364 > Project: HBase > Issue Type: Improvement >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0 > > > There is a TODO like this "TODO: Redo so only a single pass over the arrays > rather than one to compare and then a second composing midpoint." in > getMidpoint() of class HFileWriteImpl > > The old logic compares the left byte array and the right byte array twice： > # A comparison is performed before composing MinimumMidpointArray > # During composing of MinimumMidpointArray, bytes were comparing again > > My optimization combines them into one -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack merged pull request #2741: HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of t…
saintstack merged pull request #2741: URL: https://github.com/apache/hbase/pull/2741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #2945: [HBASE-25539] Adding replication log queue
shahrs87 commented on a change in pull request #2945:
URL: https://github.com/apache/hbase/pull/2945#discussion_r573862560
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationLogQueue.java
##
@@ -0,0 +1,93 @@
+package org.apache.hadoop.hbase.replication.regionserver;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.PriorityBlockingQueue;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.hadoop.hbase.wal.AbstractFSWALProvider;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class ReplicationLogQueue {
Review comment:
@saintstack This is still in draft mode. Wanted to run tests first
before I submit for review. Thank you !
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #2945: [HBASE-25539] Adding replication log queue
shahrs87 commented on a change in pull request #2945: URL: https://github.com/apache/hbase/pull/2945#discussion_r573862445 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationLogQueue.java ## @@ -0,0 +1,93 @@ +package org.apache.hadoop.hbase.replication.regionserver; Review comment: @saintstack This is still in draft mode. Wanted to run tests first before I submit for review. Thank you ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2945: [HBASE-25539] Adding replication log queue
saintstack commented on a change in pull request #2945:
URL: https://github.com/apache/hbase/pull/2945#discussion_r573860953
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
##
@@ -87,9 +86,10 @@
private static final Logger LOG =
LoggerFactory.getLogger(ReplicationSource.class);
// Queues of logs to process, entry in format of walGroupId->queue,
// each presents a queue for one wal group
- private Map> queues = new HashMap<>();
+ //private Map> queues = new HashMap<>();
Review comment:
Remove rather than comment out (This is a draft so maybe you are just
trying stuff -- if so, ignore this comment)
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationLogQueue.java
##
@@ -0,0 +1,93 @@
+package org.apache.hadoop.hbase.replication.regionserver;
Review comment:
License
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationLogQueue.java
##
@@ -0,0 +1,93 @@
+package org.apache.hadoop.hbase.replication.regionserver;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.PriorityBlockingQueue;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.hadoop.hbase.wal.AbstractFSWALProvider;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class ReplicationLogQueue {
Review comment:
Class comment on what this thing does
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25507) Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException: Invalid HFile block magic"
[
https://issues.apache.org/jira/browse/HBASE-25507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282470#comment-17282470
]
Hudson commented on HBASE-25507:
Results for branch master
[build #207 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException:
> Invalid HFile block magic"
>
>
> Key: HBASE-25507
> URL: https://issues.apache.org/jira/browse/HBASE-25507
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
>Affects Versions: 3.0.0-alpha-1, 2.4.1
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.5, 2.4.2
>
> Attachments: errorlogs.png,
> increasing-of-established-sockets-image.png, problem-region-move-logs.png
>
>
> Recently, we found socket leaks on our production cluster. The leaked sockets
> are in ESTABLISHED state. We found this happened on RS who owned a particular
> region from our analysis of metrics monitor and logs. RS without this region
> works normally.
> On the RS who owns the particular region, we found Exceptions as follows,
> {code:java}
> java.io.IOException: java.io.IOException: Could not seek
> StoreFileScanner[org.apache.hadoop.hbase.ioo
> .HalfStoreFileReader$1@5388be2f, cur=null] to key
> org.apache.hadoop.hbase.CellUtil$FirstOnRowDeleteFF
> amilyCell@25aa56fd
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.parallelSeek(StoreScanner.java:1128)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:437)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:329)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:302)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:88
> 06)
> at
> org.apache.hadoop.hbase.regionserver.compactions.StripeCompactor$StripeInternalScannerFacc
> tory.createScanner(StripeCompactor.java:82)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:316)
> at
> org.apache.hadoop.hbase.regionserver.compactions.StripeCompactor.compact(StripeCompactor..
> java:120)
> at
> org.apache.hadoop.hbase.regionserver.compactions.StripeCompactionPolicy$SplitStripeCompacc
> tionRequest.execute(StripeCompactionPolicy.java:662)
> at
> org.apache.hadoop.hbase.regionserver.StripeStoreEngine$StripeCompaction.compact(StripeStoo
> reEngine.java:114)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1461)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2121)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(Comm
> pactSplitThread.java:519)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitt
> Thread.java:555)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Could not seek
> StoreFileScanner[org.apache.hadoop.hbase.io.HalfStoree
> FileReader$1@5388be2f, cur=null] to key
> org.apache.hadoop.hbase.CellUtil$FirstOnRowDeleteFamilyCell@@
> 25aa56fd
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:229)
> at
> org.apache.hadoop.hbase.regionserver.handler.ParallelSeekHandler.process(ParallelSeekHandd
> ler.java:56)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
> ... 3 more
> Caused by: java.io.IOException: Invalid HFile block
[jira] [Commented] (HBASE-25534) Honor TableDescriptor settings earlier in normalization
[
https://issues.apache.org/jira/browse/HBASE-25534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282472#comment-17282472
]
Hudson commented on HBASE-25534:
Results for branch master
[build #207 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Honor TableDescriptor settings earlier in normalization
> ---
>
> Key: HBASE-25534
> URL: https://issues.apache.org/jira/browse/HBASE-25534
> Project: HBase
> Issue Type: Improvement
> Components: Normalizer
>Affects Versions: 3.0.0-alpha-1, 2.4.2
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.2
>
>
> -Table can be enabled and disabled merge/split in TableDescriptor, we should
> judge before calculating the plan.-
> The normalizer configuration can be set by table level. For example,
> hbase.normalizer.min.region.count can be set by "alter ‘table’,
> CONFIGURATION=>\{'hbase.normalizer.min.region.count' => '5'}". If the table
> is not set, then use the global configuration which is set in hbase-site.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25559) Terminate threads of oldsources while RS is closing
[
https://issues.apache.org/jira/browse/HBASE-25559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282471#comment-17282471
]
Hudson commented on HBASE-25559:
Results for branch master
[build #207 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Terminate threads of oldsources while RS is closing
> ---
>
> Key: HBASE-25559
> URL: https://issues.apache.org/jira/browse/HBASE-25559
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.2.6, 2.3.4, 2.4.1
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.5, 2.4.2
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24772) Use GetoptLong or OptionParser in hbase-shell
[
https://issues.apache.org/jira/browse/HBASE-24772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282469#comment-17282469
]
Hudson commented on HBASE-24772:
Results for branch master
[build #207 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/207/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use GetoptLong or OptionParser in hbase-shell
> -
>
> Key: HBASE-24772
> URL: https://issues.apache.org/jira/browse/HBASE-24772
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Elliot Miller
>Assignee: Laxman Goswami
>Priority: Minor
> Labels: beginner-friendly
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> Currently, our hbase-shell command line argument parser is custom-rolled. It
> would be awesome to instead use Ruby's GetoptLong or OptionParser.
> * [https://ruby-doc.org/stdlib-2.3.0/libdoc/getoptlong/rdoc/GetoptLong.html]
> * [https://ruby-doc.org/stdlib-2.3.0/libdoc/optparse/rdoc/OptionParser.html]
> There is a long-standing {{FIXME}} comment in {{bin/hirb.rb}} to address
> this:
> [https://github.com/apache/hbase/blob/975cdf7b88f001aa51d83395c663892f3b5ffbdb/bin/hirb.rb#L50]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282362#comment-17282362
]
Viraj Jasani commented on HBASE-23887:
--
Thanks [~pustota] for the patch and all the benchmark with various workloads.
+1 to the updated PR. I am positive on committing this given that we have new
pluggable cache implementation in place.
[~reidchan] [~bharathv] [~busbey] [~elserj] would you like to take a look at
[https://github.com/apache/hbase/pull/2934] ?
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, PR#1257.diff,
> cmp.png, evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.eviction.bytes.size.limit - set how many bytes have to
> evicted each time that start to avoid of putting data to BlockCache
> By default: if 10 times (100 secunds) evicted more than 10 MB (each time)
> then we start to skip 50% of data blocks.
> When heavy evitions process end then new logic off and will put into
> BlockCache all blocks again.
>
> Descriptions of the test:
> 4 nodes E5-2698 v4 @ 2.20GHz, 700 Gb Mem.
> 4 RegionServers
> 4 tables by 64 regions by 1.88 Gb data in each = 600 Gb total (only FAST_DIFF)
> Total BlockCache Size = 48 Gb (8 % of data in HFiles)
> Random read in 20 threads
>
> I am going to make Pull Request, hope it is right
