[jira] [Commented] (HBASE-27000) Block cache stats (Misses Caching) display error in RS web UI
[
https://issues.apache.org/jira/browse/HBASE-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533193#comment-17533193
]
Hudson commented on HBASE-27000:
Results for branch branch-2
[build #535 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/535//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Block cache stats (Misses Caching) display error in RS web UI
> -
>
> Key: HBASE-27000
> URL: https://issues.apache.org/jira/browse/HBASE-27000
> Project: HBase
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
> Attachments: image-2022-05-05-20-11-47-884.png
>
>
> Block cache stats (Misses Caching) display error in RS web UI.
> !image-2022-05-05-20-11-47-884.png|width=547,height=303!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27000) Block cache stats (Misses Caching) display error in RS web UI
[
https://issues.apache.org/jira/browse/HBASE-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533189#comment-17533189
]
Hudson commented on HBASE-27000:
Results for branch branch-2.4
[build #347 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/347//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Block cache stats (Misses Caching) display error in RS web UI
> -
>
> Key: HBASE-27000
> URL: https://issues.apache.org/jira/browse/HBASE-27000
> Project: HBase
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
> Attachments: image-2022-05-05-20-11-47-884.png
>
>
> Block cache stats (Misses Caching) display error in RS web UI.
> !image-2022-05-05-20-11-47-884.png|width=547,height=303!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4414: HBASE-27013 Introduce read all bytes when using pread for prefetch
Apache-HBase commented on PR #4414: URL: https://github.com/apache/hbase/pull/4414#issuecomment-1120127554 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 39s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 30s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 44s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 183m 59s | hbase-server in the patch passed. | | | | 204m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4414 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e25bbff0a000 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-common.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/testReport/ | | Max. process+thread count | 2501 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4414: HBASE-27013 Introduce read all bytes when using pread for prefetch
Apache-HBase commented on PR #4414: URL: https://github.com/apache/hbase/pull/4414#issuecomment-1120127240 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 46s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 15s | master passed | | +1 :green_heart: | compile | 0m 45s | master passed | | +1 :green_heart: | shadedjars | 3m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 3s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 3m 47s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 37s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 182m 17s | hbase-server in the patch passed. | | | | 201m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4414 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3626c6a53b7e 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/testReport/ | | Max. process+thread count | 2365 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27013) Introduce read all bytes when using pread for prefetch
[
https://issues.apache.org/jira/browse/HBASE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533170#comment-17533170
]
Duo Zhang commented on HBASE-27013:
---
So the problem here is, the implementation of S3A is not HDFS, we can not reuse
the stream to send multiple pread requests with random offset. Seems not like a
good enough pread implementation...
In general, in pread mode, a FSDataInputStream may be used by different read
requests so even if you fixed this problem, it could still introduce a lot of
aborts as different read request may read from different offsets...
Thanks...
> Introduce read all bytes when using pread for prefetch
> --
>
> Key: HBASE-27013
> URL: https://issues.apache.org/jira/browse/HBASE-27013
> Project: HBase
> Issue Type: Improvement
> Components: HFile, Performance
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
>
> h2. Problem statement
> When prefetching HFiles from blob storage like S3 and use it with the storage
> implementation like S3A, we found there is a logical issue in HBase pread
> that causes the reading of the remote HFile aborts the input stream multiple
> times. This aborted stream and reopen slow down the reads and trigger many
> aborted bytes and waste time in recreating the connection especially when SSL
> is enabled.
> h2. ROOT CAUSE
> The root cause of above issue was due to
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> is reading an input stream that does not guarrentee to return the data block
> and the next block header as an option data to be cached.
> In the case of the input stream read short and when the input stream read
> passed the length of the necessary data block with few more bytes within the
> size of next block header, the
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> returns to the caller without a cached the next block header. As a result,
> before HBase tries to read the next block,
> [HFileBlock#readBlockDataInternal|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664]
> in hbase tries to re-read the next block header from the input stream. Here,
> the reusable input stream has move the current position pointer ahead from
> the offset of the last read data block, when using with the [S3A
> implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361],
> the input stream is then closed, aborted all the remaining bytes and reopen
> a new input stream at the offset of the last read data block .
> h2. How do we fix it?
> S3A is doing the right job that HBase is telling to move the offset from
> position A back to A - N, so there is not much thing we can do on how S3A
> handle the inputstream. meanwhile in the case of HDFS, this operation is fast.
> Such that, we should fix in HBase level, and try always to read datablock +
> next block header when we're using blob storage to avoid expensive draining
> the bytes in a stream and reopen the socket with the remote storage.
> h2. Draw back and discussion
> * A known drawback is, when we're at the last block, we will read extra
> length that should not be a header, and we still read that into the byte
> buffer array. the size should be always 33 bytes, and it should not a big
> issue in data correctness because the trailer will tell when the last
> datablock should end. And we just waste a 33 byte read and that data is not
> being used.
> * I don't know if we can use HFileStreamReader but that will change the
> Prefetch logic a lot, such that this minimum change should be the best.
> h2. initial result
> We use YCSB 1 billion records data, and we enable prefetch for the userable.
> the collected the S3A metrics of {{stream_read_bytes_discarded_in_abort}} to
> compare the solution, each region server have abort ~290 GB data to be
> prefetch to bucketcache.
> * before the change, we have a total of 4235973338472 bytes (~4235GB) has
> been aborted on a sample region server for about 290GB data.
> ** the overall time was about 45 ~ 60 mins
>
> {code}
> % grep "stream_read_bytes_discarded_in_abort"
> ~/prefetch-result/prefetch-s3a-jmx-metrics.json | grep -wv
> "stream_read_bytes_discarded_in_abort\":0,"
>
[GitHub] [hbase] Apache-HBase commented on pull request #4414: HBASE-27013 Introduce read all bytes when using pread for prefetch
Apache-HBase commented on PR #4414: URL: https://github.com/apache/hbase/pull/4414#issuecomment-1120098502 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 45s | master passed | | +1 :green_heart: | compile | 2m 46s | master passed | | +1 :green_heart: | checkstyle | 0m 39s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 38s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 10s | the patch passed | | +1 :green_heart: | compile | 2m 40s | the patch passed | | +1 :green_heart: | javac | 2m 40s | the patch passed | | -0 :warning: | checkstyle | 0m 27s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 6s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 33s | patch has 63 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 35m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4414 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 685d3068b9be 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 65 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4414/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27013) Introduce read all bytes when using pread for prefetch
[
https://issues.apache.org/jira/browse/HBASE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak-Lon (Stephen) Wu updated HBASE-27013:
-
Summary: Introduce read all bytes when using pread for prefetch (was:
Introduce read all bytes when using pread for Prefetch)
> Introduce read all bytes when using pread for prefetch
> --
>
> Key: HBASE-27013
> URL: https://issues.apache.org/jira/browse/HBASE-27013
> Project: HBase
> Issue Type: Improvement
> Components: HFile, Performance
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
>
> h2. Problem statement
> When prefetching HFiles from blob storage like S3 and use it with the storage
> implementation like S3A, we found there is a logical issue in HBase pread
> that causes the reading of the remote HFile aborts the input stream multiple
> times. This aborted stream and reopen slow down the reads and trigger many
> aborted bytes and waste time in recreating the connection especially when SSL
> is enabled.
> h2. ROOT CAUSE
> The root cause of above issue was due to
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> is reading an input stream that does not guarrentee to return the data block
> and the next block header as an option data to be cached.
> In the case of the input stream read short and when the input stream read
> passed the length of the necessary data block with few more bytes within the
> size of next block header, the
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> returns to the caller without a cached the next block header. As a result,
> before HBase tries to read the next block,
> [HFileBlock#readBlockDataInternal|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664]
> in hbase tries to re-read the next block header from the input stream. Here,
> the reusable input stream has move the current position pointer ahead from
> the offset of the last read data block, when using with the [S3A
> implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361],
> the input stream is then closed, aborted all the remaining bytes and reopen
> a new input stream at the offset of the last read data block .
> h2. How do we fix it?
> S3A is doing the right job that HBase is telling to move the offset from
> position A back to A - N, so there is not much thing we can do on how S3A
> handle the inputstream. meanwhile in the case of HDFS, this operation is fast.
> Such that, we should fix in HBase level, and try always to read datablock +
> next block header when we're using blob storage to avoid expensive draining
> the bytes in a stream and reopen the socket with the remote storage.
> h2. Draw back and discussion
> * A known drawback is, when we're at the last block, we will read extra
> length that should not be a header, and we still read that into the byte
> buffer array. the size should be always 33 bytes, and it should not a big
> issue in data correctness because the trailer will tell when the last
> datablock should end. And we just waste a 33 byte read and that data is not
> being used.
> * I don't know if we can use HFileStreamReader but that will change the
> Prefetch logic a lot, such that this minimum change should be the best.
> h2. initial result
> We use YCSB 1 billion records data, and we enable prefetch for the userable.
> the collected the S3A metrics of {{stream_read_bytes_discarded_in_abort}} to
> compare the solution, each region server have abort ~290 GB data to be
> prefetch to bucketcache.
> * before the change, we have a total of 4235973338472 bytes (~4235GB) has
> been aborted on a sample region server for about 290GB data.
> ** the overall time was about 45 ~ 60 mins
>
> {code}
> % grep "stream_read_bytes_discarded_in_abort"
> ~/prefetch-result/prefetch-s3a-jmx-metrics.json | grep -wv
> "stream_read_bytes_discarded_in_abort\":0,"
> "stream_read_bytes_discarded_in_abort":3136854553,
> "stream_read_bytes_discarded_in_abort":19119241,
> "stream_read_bytes_discarded_in_abort":2131591701471,
> "stream_read_bytes_discarded_in_abort":150484654298,
> "stream_read_bytes_discarded_in_abort":106536641550,
>
[jira] [Commented] (HBASE-27013) Introduce read all bytes when using pread for Prefetch
[
https://issues.apache.org/jira/browse/HBASE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533162#comment-17533162
]
Tak-Lon (Stephen) Wu commented on HBASE-27013:
--
we may have better solution, and we can discuss here to see if we can do
anything to abort input stream is being aborted or closed before reading
> Introduce read all bytes when using pread for Prefetch
> --
>
> Key: HBASE-27013
> URL: https://issues.apache.org/jira/browse/HBASE-27013
> Project: HBase
> Issue Type: Improvement
> Components: HFile, Performance
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
>
> h2. Problem statement
> When prefetching HFiles from blob storage like S3 and use it with the storage
> implementation like S3A, we found there is a logical issue in HBase pread
> that causes the reading of the remote HFile aborts the input stream multiple
> times. This aborted stream and reopen slow down the reads and trigger many
> aborted bytes and waste time in recreating the connection especially when SSL
> is enabled.
> h2. ROOT CAUSE
> The root cause of above issue was due to
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> is reading an input stream that does not guarrentee to return the data block
> and the next block header as an option data to be cached.
> In the case of the input stream read short and when the input stream read
> passed the length of the necessary data block with few more bytes within the
> size of next block header, the
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> returns to the caller without a cached the next block header. As a result,
> before HBase tries to read the next block,
> [HFileBlock#readBlockDataInternal|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664]
> in hbase tries to re-read the next block header from the input stream. Here,
> the reusable input stream has move the current position pointer ahead from
> the offset of the last read data block, when using with the [S3A
> implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361],
> the input stream is then closed, aborted all the remaining bytes and reopen
> a new input stream at the offset of the last read data block .
> h2. How do we fix it?
> S3A is doing the right job that HBase is telling to move the offset from
> position A back to A - N, so there is not much thing we can do on how S3A
> handle the inputstream. meanwhile in the case of HDFS, this operation is fast.
> Such that, we should fix in HBase level, and try always to read datablock +
> next block header when we're using blob storage to avoid expensive draining
> the bytes in a stream and reopen the socket with the remote storage.
> h2. Draw back and discussion
> * A known drawback is, when we're at the last block, we will read extra
> length that should not be a header, and we still read that into the byte
> buffer array. the size should be always 33 bytes, and it should not a big
> issue in data correctness because the trailer will tell when the last
> datablock should end. And we just waste a 33 byte read and that data is not
> being used.
> * I don't know if we can use HFileStreamReader but that will change the
> Prefetch logic a lot, such that this minimum change should be the best.
> h2. initial result
> We use YCSB 1 billion records data, and we enable prefetch for the userable.
> the collected the S3A metrics of {{stream_read_bytes_discarded_in_abort}} to
> compare the solution, each region server have abort ~290 GB data to be
> prefetch to bucketcache.
> * before the change, we have a total of 4235973338472 bytes (~4235GB) has
> been aborted on a sample region server for about 290GB data.
> ** the overall time was about 45 ~ 60 mins
>
> {code}
> % grep "stream_read_bytes_discarded_in_abort"
> ~/prefetch-result/prefetch-s3a-jmx-metrics.json | grep -wv
> "stream_read_bytes_discarded_in_abort\":0,"
> "stream_read_bytes_discarded_in_abort":3136854553,
> "stream_read_bytes_discarded_in_abort":19119241,
> "stream_read_bytes_discarded_in_abort":2131591701471,
> "stream_read_bytes_discarded_in_abort":150484654298,
>
[jira] [Assigned] (HBASE-27013) Introduce read all bytes when using pread for Prefetch
[
https://issues.apache.org/jira/browse/HBASE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak-Lon (Stephen) Wu reassigned HBASE-27013:
Assignee: Tak-Lon (Stephen) Wu
> Introduce read all bytes when using pread for Prefetch
> --
>
> Key: HBASE-27013
> URL: https://issues.apache.org/jira/browse/HBASE-27013
> Project: HBase
> Issue Type: Improvement
> Components: HFile, Performance
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
>
> h2. Problem statement
> When prefetching HFiles from blob storage like S3 and use it with the storage
> implementation like S3A, we found there is a logical issue in HBase pread
> that causes the reading of the remote HFile aborts the input stream multiple
> times. This aborted stream and reopen slow down the reads and trigger many
> aborted bytes and waste time in recreating the connection especially when SSL
> is enabled.
> h2. ROOT CAUSE
> The root cause of above issue was due to
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> is reading an input stream that does not guarrentee to return the data block
> and the next block header as an option data to be cached.
> In the case of the input stream read short and when the input stream read
> passed the length of the necessary data block with few more bytes within the
> size of next block header, the
> [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
> returns to the caller without a cached the next block header. As a result,
> before HBase tries to read the next block,
> [HFileBlock#readBlockDataInternal|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664]
> in hbase tries to re-read the next block header from the input stream. Here,
> the reusable input stream has move the current position pointer ahead from
> the offset of the last read data block, when using with the [S3A
> implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361],
> the input stream is then closed, aborted all the remaining bytes and reopen
> a new input stream at the offset of the last read data block .
> h2. How do we fix it?
> S3A is doing the right job that HBase is telling to move the offset from
> position A back to A - N, so there is not much thing we can do on how S3A
> handle the inputstream. meanwhile in the case of HDFS, this operation is fast.
> Such that, we should fix in HBase level, and try always to read datablock +
> next block header when we're using blob storage to avoid expensive draining
> the bytes in a stream and reopen the socket with the remote storage.
> h2. Draw back and discussion
> * A known drawback is, when we're at the last block, we will read extra
> length that should not be a header, and we still read that into the byte
> buffer array. the size should be always 33 bytes, and it should not a big
> issue in data correctness because the trailer will tell when the last
> datablock should end. And we just waste a 33 byte read and that data is not
> being used.
> * I don't know if we can use HFileStreamReader but that will change the
> Prefetch logic a lot, such that this minimum change should be the best.
> h2. initial result
> We use YCSB 1 billion records data, and we enable prefetch for the userable.
> the collected the S3A metrics of {{stream_read_bytes_discarded_in_abort}} to
> compare the solution, each region server have abort ~290 GB data to be
> prefetch to bucketcache.
> * before the change, we have a total of 4235973338472 bytes (~4235GB) has
> been aborted on a sample region server for about 290GB data.
> ** the overall time was about 45 ~ 60 mins
>
> {code}
> % grep "stream_read_bytes_discarded_in_abort"
> ~/prefetch-result/prefetch-s3a-jmx-metrics.json | grep -wv
> "stream_read_bytes_discarded_in_abort\":0,"
> "stream_read_bytes_discarded_in_abort":3136854553,
> "stream_read_bytes_discarded_in_abort":19119241,
> "stream_read_bytes_discarded_in_abort":2131591701471,
> "stream_read_bytes_discarded_in_abort":150484654298,
> "stream_read_bytes_discarded_in_abort":106536641550,
> "stream_read_bytes_discarded_in_abort":1785264521717,
>
[jira] [Updated] (HBASE-27013) Introduce read all bytes when using pread for Prefetch
[
https://issues.apache.org/jira/browse/HBASE-27013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak-Lon (Stephen) Wu updated HBASE-27013:
-
Description:
h2. Problem statement
When prefetching HFiles from blob storage like S3 and use it with the storage
implementation like S3A, we found there is a logical issue in HBase pread that
causes the reading of the remote HFile aborts the input stream multiple times.
This aborted stream and reopen slow down the reads and trigger many aborted
bytes and waste time in recreating the connection especially when SSL is
enabled.
h2. ROOT CAUSE
The root cause of above issue was due to
[BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
is reading an input stream that does not guarrentee to return the data block
and the next block header as an option data to be cached.
In the case of the input stream read short and when the input stream read
passed the length of the necessary data block with few more bytes within the
size of next block header, the
[BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257]
returns to the caller without a cached the next block header. As a result,
before HBase tries to read the next block,
[HFileBlock#readBlockDataInternal|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664]
in hbase tries to re-read the next block header from the input stream. Here,
the reusable input stream has move the current position pointer ahead from the
offset of the last read data block, when using with the [S3A
implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361],
the input stream is then closed, aborted all the remaining bytes and reopen a
new input stream at the offset of the last read data block .
h2. How do we fix it?
S3A is doing the right job that HBase is telling to move the offset from
position A back to A - N, so there is not much thing we can do on how S3A
handle the inputstream. meanwhile in the case of HDFS, this operation is fast.
Such that, we should fix in HBase level, and try always to read datablock +
next block header when we're using blob storage to avoid expensive draining the
bytes in a stream and reopen the socket with the remote storage.
h2. Draw back and discussion
* A known drawback is, when we're at the last block, we will read extra length
that should not be a header, and we still read that into the byte buffer array.
the size should be always 33 bytes, and it should not a big issue in data
correctness because the trailer will tell when the last datablock should end.
And we just waste a 33 byte read and that data is not being used.
* I don't know if we can use HFileStreamReader but that will change the
Prefetch logic a lot, such that this minimum change should be the best.
h2. initial result
We use YCSB 1 billion records data, and we enable prefetch for the userable.
the collected the S3A metrics of {{stream_read_bytes_discarded_in_abort}} to
compare the solution, each region server have abort ~290 GB data to be prefetch
to bucketcache.
* before the change, we have a total of 4235973338472 bytes (~4235GB) has been
aborted on a sample region server for about 290GB data.
** the overall time was about 45 ~ 60 mins
{code}
% grep "stream_read_bytes_discarded_in_abort"
~/prefetch-result/prefetch-s3a-jmx-metrics.json | grep -wv
"stream_read_bytes_discarded_in_abort\":0,"
"stream_read_bytes_discarded_in_abort":3136854553,
"stream_read_bytes_discarded_in_abort":19119241,
"stream_read_bytes_discarded_in_abort":2131591701471,
"stream_read_bytes_discarded_in_abort":150484654298,
"stream_read_bytes_discarded_in_abort":106536641550,
"stream_read_bytes_discarded_in_abort":1785264521717,
"stream_read_bytes_discarded_in_abort":58939845642,
{code}
* After the change, we only have 87100225454 bytes (~87GB) data to be aborted.
** the reason is about the position is way behind the asked target position,
then S3A reopen the stream and move the position to the current offset. This is
a different problem we will need to look into later.
** the overall time is then cut to 30~38 mins, about 30% faster.
{code}
% grep "stream_read_bytes_discarded_in_abort" ~/fixed-formatted-jmx2.json
"stream_read_bytes_discarded_in_abort": 0,
"stream_read_bytes_discarded_in_abort": 87100225454,
"stream_read_bytes_discarded_in_abort": 67043088,
{code}
[GitHub] [hbase] 2005hithlj commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
2005hithlj commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1120084218 Have re-triggered the pre-commit check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1120081545 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | branch-2 passed | | +1 :green_heart: | compile | 0m 48s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 34s | the patch passed | | +1 :green_heart: | compile | 0m 49s | the patch passed | | +1 :green_heart: | javac | 0m 49s | the patch passed | | +1 :green_heart: | shadedjars | 3m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 197m 37s | hbase-server in the patch passed. | | | | 216m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bb1db8c4f040 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/testReport/ | | Max. process+thread count | 2543 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27013) Introduce read all bytes when using pread for Prefetch
Tak-Lon (Stephen) Wu created HBASE-27013: Summary: Introduce read all bytes when using pread for Prefetch Key: HBASE-27013 URL: https://issues.apache.org/jira/browse/HBASE-27013 Project: HBase Issue Type: Improvement Components: HFile, Performance Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13 Reporter: Tak-Lon (Stephen) Wu h2. Problem statement When prefetching HFiles from blob storage like S3 and use it with the storage implementation like S3A, we found there is a logical issue in HBase pread that causes the reading of the remote HFile aborts the input stream multiple times. This aborted stream and reopen slow down the reads and trigger many aborted bytes and waste time in recreating the connection especially when SSL is enabled. h2. ROOT CAUSE The root cause of above issue was due to [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257] is reading an input stream that does not guarrentee to return the data block and the next block header as an option data to be cached. In the case of the input stream read short and we passed the length of the necessary data block plus few bytes within the size of next block header, [BlockIOUtils#preadWithExtra|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-common/src/main/java/org/apache/hadoop/hbase/io/util/BlockIOUtils.java#L214-L257] returns to the caller. As a result, when we're trying to read the next block, [HFileBlock#|https://github.com/apache/hbase/blob/9c8c9e7fbf8005ea89fa9b13d6d063b9f0240443/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1648-L1664] in hbase will try to re-read the header from the input stream, but because the reusable input stream has move the current position pointer ahead from the offset of the last read data block, with the [S3A implementation|https://github.com/apache/hadoop/blob/29401c820377d02a992eecde51083cf87f8e57af/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L339-L361], the input stream is then closed, aborted all the remaining bytes and reopen a new input stream at the offset of the last read data block . h2. How do we fix it? S3A is doing the right job that HBase is telling to move the offset from position A back to A - N, so there is not much thing we can do on how S3A handle the inputstream. meanwhile in the case of HDFS, this operation is fast. Such that, we should fix in HBase level, and try always to read datablock + next block header when we're using blob storage to avoid expensive draining the bytes in a stream and reopen the socket with the remote storage. h2. Draw back and discussion * A known drawback is, when we're at the last block, we will read extra length that should not be a header, and we still read that into the byte buffer array. the size should be always 33 bytes, and it should not a big issue in data correctness because the trailer will tell when the last datablock should end. And we just waste a 33 byte read and that data is not being used. * I don't know if we can use HFileStreamReader but that will change the Prefetch logic a lot, such that this minimum change should be the best. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1120075839 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | branch-2 passed | | +1 :green_heart: | compile | 0m 34s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 3s | the patch passed | | +1 :green_heart: | compile | 0m 35s | the patch passed | | +1 :green_heart: | javac | 0m 35s | the patch passed | | +1 :green_heart: | shadedjars | 3m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 178m 15s | hbase-server in the patch passed. | | | | 194m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e65448cdda35 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/testReport/ | | Max. process+thread count | 2363 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
Apache-HBase commented on PR #4125: URL: https://github.com/apache/hbase/pull/4125#issuecomment-1120045443 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 51s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 19s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 4m 31s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 1m 46s | root generated 9 new + 75 unchanged - 7 fixed = 84 total (was 82) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 330m 16s | root in the patch passed. | | | | 355m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4125 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ffded5673e6c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/testReport/ | | Max. process+thread count | 4963 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
shahrs87 commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1120007986 Looks like there are 2 spotless checks happening, one for the branch and other for the patch. The branch one is failing but the patch one succeeded. So there is nothing for me to do ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
Apache-HBase commented on PR #4125: URL: https://github.com/apache/hbase/pull/4125#issuecomment-112019 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 47s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 31s | master passed | | +1 :green_heart: | compile | 2m 24s | master passed | | +1 :green_heart: | shadedjars | 4m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 18s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | compile | 2m 17s | the patch passed | | +1 :green_heart: | javac | 2m 18s | the patch passed | | +1 :green_heart: | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 245m 18s | root in the patch failed. | | | | 278m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4125 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6f5b862280dd 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/testReport/ | | Max. process+thread count | 2599 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1119997827 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | branch-2 passed | | +1 :green_heart: | compile | 2m 17s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 31s | branch-2 passed | | -1 :x: | spotless | 0m 39s | branch has 24 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 19s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 6s | the patch passed | | +1 :green_heart: | compile | 2m 14s | the patch passed | | +1 :green_heart: | javac | 2m 14s | the patch passed | | +1 :green_heart: | checkstyle | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 4s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 29m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 4b692fb6110c 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/artifact/yetus-general-check/output/branch-spotless.txt | | Max. process+thread count | 65 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27000) Block cache stats (Misses Caching) display error in RS web UI
[
https://issues.apache.org/jira/browse/HBASE-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533084#comment-17533084
]
Hudson commented on HBASE-27000:
Results for branch branch-2.5
[build #112 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Block cache stats (Misses Caching) display error in RS web UI
> -
>
> Key: HBASE-27000
> URL: https://issues.apache.org/jira/browse/HBASE-27000
> Project: HBase
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
> Attachments: image-2022-05-05-20-11-47-884.png
>
>
> Block cache stats (Misses Caching) display error in RS web UI.
> !image-2022-05-05-20-11-47-884.png|width=547,height=303!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26984) Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction
[
https://issues.apache.org/jira/browse/HBASE-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533083#comment-17533083
]
Hudson commented on HBASE-26984:
Results for branch branch-2.5
[build #112 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction
> ---
>
> Key: HBASE-26984
> URL: https://issues.apache.org/jira/browse/HBASE-26984
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Affects Versions: 2.4.11
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> Run itbll chaos monkey in k8s cluster, found chaos monkey thread died in
> GracefulRollingRestartRsAction.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26995) Remove ref guide check in pre commit and nightly for branches other than master
[
https://issues.apache.org/jira/browse/HBASE-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533082#comment-17533082
]
Hudson commented on HBASE-26995:
Results for branch branch-2.5
[build #112 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/112/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove ref guide check in pre commit and nightly for branches other than
> master
> ---
>
> Key: HBASE-26995
> URL: https://issues.apache.org/jira/browse/HBASE-26995
> Project: HBase
> Issue Type: Sub-task
> Components: build, scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.13
>
>
> Our nightly builds are failing after HBASE-26932. Let's fix it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] shahrs87 commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
shahrs87 commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1119990920 branch-2 PR is also ready for review and merge. @Apache9 @apurtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1119979136 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 48s | branch-2 passed | | +1 :green_heart: | compile | 0m 46s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | shadedjars | 3m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 193m 26s | hbase-server in the patch passed. | | | | 211m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7267dcc78f98 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/testReport/ | | Max. process+thread count | 2376 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4412: HBASE-26977 Fix branch-2 spotless check error
Apache-HBase commented on PR #4412: URL: https://github.com/apache/hbase/pull/4412#issuecomment-1119969082 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | branch-2 passed | | +1 :green_heart: | compile | 1m 0s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 37s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | shadedjars | 3m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 190m 8s | hbase-server in the patch passed. | | | | 211m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4412 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 25341e0eda51 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/testReport/ | | Max. process+thread count | 2155 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1119968778 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | branch-2 passed | | +1 :green_heart: | compile | 0m 35s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 0m 34s | the patch passed | | +1 :green_heart: | javac | 0m 34s | the patch passed | | +1 :green_heart: | shadedjars | 3m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 178m 16s | hbase-server in the patch passed. | | | | 195m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 96110dc7da99 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/testReport/ | | Max. process+thread count | 2694 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4412: HBASE-26977 Fix branch-2 spotless check error
Apache-HBase commented on PR #4412: URL: https://github.com/apache/hbase/pull/4412#issuecomment-1119966240 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | branch-2 passed | | +1 :green_heart: | compile | 0m 40s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 189m 32s | hbase-server in the patch passed. | | | | 206m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4412 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bb6c2550fdf5 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/testReport/ | | Max. process+thread count | 2318 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
Apache-HBase commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1119945749 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 58s | branch-1 passed | | +1 :green_heart: | compile | 0m 26s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 30s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 0m 57s | branch-1 passed | | +1 :green_heart: | shadedjars | 1m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 25s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 1m 42s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 41s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 15s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 1m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 2m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 1m 47s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 151m 8s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 174m 44s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL | | | hadoop.hbase.replication.TestReplicationEndpoint | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4377 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8937db4df215 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4377/out/precommit/personality/provided.sh | | git revision | branch-1 / 1599a48687 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/8/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/8/testReport/ | | Max. process+thread count | 3873 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/8/console | | versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 | | Powered by | Apache Yetus
[jira] [Commented] (HBASE-27000) Block cache stats (Misses Caching) display error in RS web UI
[
https://issues.apache.org/jira/browse/HBASE-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533052#comment-17533052
]
Hudson commented on HBASE-27000:
Results for branch master
[build #580 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Block cache stats (Misses Caching) display error in RS web UI
> -
>
> Key: HBASE-27000
> URL: https://issues.apache.org/jira/browse/HBASE-27000
> Project: HBase
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
> Attachments: image-2022-05-05-20-11-47-884.png
>
>
> Block cache stats (Misses Caching) display error in RS web UI.
> !image-2022-05-05-20-11-47-884.png|width=547,height=303!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26984) Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction
[
https://issues.apache.org/jira/browse/HBASE-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533051#comment-17533051
]
Hudson commented on HBASE-26984:
Results for branch master
[build #580 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Chaos Monkey thread dies in ITBLL Chaos GracefulRollingRestartRsAction
> ---
>
> Key: HBASE-26984
> URL: https://issues.apache.org/jira/browse/HBASE-26984
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Affects Versions: 2.4.11
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> Run itbll chaos monkey in k8s cluster, found chaos monkey thread died in
> GracefulRollingRestartRsAction.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26931) Add the information of EOL releases to our ref guide on master branch
[
https://issues.apache.org/jira/browse/HBASE-26931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533049#comment-17533049
]
Hudson commented on HBASE-26931:
Results for branch master
[build #580 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add the information of EOL releases to our ref guide on master branch
> -
>
> Key: HBASE-26931
> URL: https://issues.apache.org/jira/browse/HBASE-26931
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> This is because we decide to only maintain the documentation for master
> branch, so we should have all the information in this doc.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26995) Remove ref guide check in pre commit and nightly for branches other than master
[
https://issues.apache.org/jira/browse/HBASE-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533050#comment-17533050
]
Hudson commented on HBASE-26995:
Results for branch master
[build #580 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/580/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove ref guide check in pre commit and nightly for branches other than
> master
> ---
>
> Key: HBASE-26995
> URL: https://issues.apache.org/jira/browse/HBASE-26995
> Project: HBase
> Issue Type: Sub-task
> Components: build, scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.13
>
>
> Our nightly builds are failing after HBASE-26932. Let's fix it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4413: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
Apache-HBase commented on PR #4413: URL: https://github.com/apache/hbase/pull/4413#issuecomment-1119826762 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 57s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 20s | branch-2 passed | | +1 :green_heart: | compile | 2m 17s | branch-2 passed | | -0 :warning: | checkstyle | 0m 6s | The patch fails to run checkstyle in hbase-server | | -1 :x: | spotless | 0m 39s | branch has 24 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 15s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 1s | the patch passed | | +1 :green_heart: | compile | 2m 12s | the patch passed | | +1 :green_heart: | javac | 2m 12s | the patch passed | | +1 :green_heart: | checkstyle | 0m 29s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 2s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | -1 :x: | spotless | 0m 33s | patch has 65 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 28m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4413 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 65c7c27bad7c 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-server.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-general-check/output/branch-spotless.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4413/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
Apache-HBase commented on PR #4125: URL: https://github.com/apache/hbase/pull/4125#issuecomment-1119826106 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 17s | master passed | | +1 :green_heart: | compile | 6m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 1s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 9m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 9s | the patch passed | | +1 :green_heart: | compile | 6m 16s | the patch passed | | +1 :green_heart: | javac | 6m 16s | the patch passed | | +1 :green_heart: | checkstyle | 1m 0s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 11m 32s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 14s | patch has 71 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 10m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 58m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4125 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml | | uname | Linux 343eb60c6ee6 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 138 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4125/8/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4412: HBASE-26977 Fix branch-2 spotless check error
Apache-HBase commented on PR #4412: URL: https://github.com/apache/hbase/pull/4412#issuecomment-1119823796 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 43s | branch-2 passed | | +1 :green_heart: | compile | 3m 8s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 52s | branch-2 passed | | -1 :x: | spotless | 0m 53s | branch has 24 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 2m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 26s | the patch passed | | +1 :green_heart: | compile | 3m 0s | the patch passed | | +1 :green_heart: | javac | 3m 0s | the patch passed | | +1 :green_heart: | checkstyle | 0m 37s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotless | 1m 5s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 39m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4412 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux cbf382731c8e 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91eab0b657 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/artifact/yetus-general-check/output/branch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4412/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4407: HBASE-26999 HStore should try write WAL compaction marker before repl…
Apache-HBase commented on PR #4407: URL: https://github.com/apache/hbase/pull/4407#issuecomment-1119800925 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | master passed | | +1 :green_heart: | compile | 0m 49s | master passed | | +1 :green_heart: | shadedjars | 4m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 6s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 187m 9s | hbase-server in the patch passed. | | | | 208m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4407 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bea60886bc6d 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/testReport/ | | Max. process+thread count | 2837 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4407: HBASE-26999 HStore should try write WAL compaction marker before repl…
Apache-HBase commented on PR #4407: URL: https://github.com/apache/hbase/pull/4407#issuecomment-1119800610 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 0m 47s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 190m 38s | hbase-server in the patch passed. | | | | 208m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4407 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux dcff55d32530 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/testReport/ | | Max. process+thread count | 2593 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #4361: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
shahrs87 commented on PR #4361: URL: https://github.com/apache/hbase/pull/4361#issuecomment-1119797134 Thank you @apurtell @Apache9 @wchevreuil @virajjasani for your review/approval. There is one flaky test `TestSizeFailures#testScannerSeesAllRecords` and is unrelated to the patch. I don't know why is `yetus/patch` subsystem failing. Any hints ? Created PR for branch-2. There is a minor conflict in HBaseTestingUtility class name. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj opened a new pull request, #4412: HBASE-26977 Fix branch-2 spotless check error
2005hithlj opened a new pull request, #4412: URL: https://github.com/apache/hbase/pull/4412 https://issues.apache.org/jira/browse/HBASE-26977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26998) TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in branch-1
[
https://issues.apache.org/jira/browse/HBASE-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li resolved HBASE-26998.
---
Fix Version/s: 1.7.2
Hadoop Flags: Reviewed
Resolution: Fixed
> TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in
> branch-1
>
>
> Key: HBASE-26998
> URL: https://issues.apache.org/jira/browse/HBASE-26998
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.1
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Major
> Fix For: 1.7.2
>
>
> As titled, I found that
> {{TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack}} is broken
> while reviewing the PR for HBASE-26977 ([pre-commit test log
> here|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/7/artifact/out/patch-unit-hbase-server.txt])
> with below exceptions:
> {noformat}
> [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 10.166 s <<< FAILURE! - in
> org.apache.hadoop.hbase.regionserver.TestRegionProcessRowsWithLocks
> [ERROR]
> testProcessExceptionAndRollBack(org.apache.hadoop.hbase.regionserver.TestRegionProcessRowsWithLocks)
> Time elapsed: 5.849 s <<< ERROR!
> com.google.protobuf.ServiceException: Error calling method
> hbase.pb.RowProcessorService.Process
> at
> org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callBlockingMethod(CoprocessorRpcChannel.java:75)
> at
> org.apache.hadoop.hbase.protobuf.generated.RowProcessorProtos$RowProcessorService$BlockingStub.process(RowProcessorProtos.java:1631)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionProcessRowsWithLocks.incrementCounter(TestRegionProcessRowsWithLocks.java:193)
> at
> org.apache.hadoop.hbase.regionserver.TestRegionProcessRowsWithLocks.testProcessExceptionAndRollBack(TestRegionProcessRowsWithLocks.java:173)
> {noformat}
> Manually test the single case in my local environment also failed. Using
> {{git bisect}}, I found the issue was introduced by HBASE-26393 and have been
> existing for several months (since Oct 23, 2021).
> The fix is straight forward and I will prepare a PR soon.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (HBASE-27012) nightly ITBLL
Sean Busbey created HBASE-27012: --- Summary: nightly ITBLL Key: HBASE-27012 URL: https://issues.apache.org/jira/browse/HBASE-27012 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27011) migrate existing nightly test
Sean Busbey created HBASE-27011: --- Summary: migrate existing nightly test Key: HBASE-27011 URL: https://issues.apache.org/jira/browse/HBASE-27011 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey once we can run a cluster on k8s in the nightly worker node, migrate our existing test to use it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] carp84 commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
carp84 commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1119751918 #4044 is merged, @2005hithlj please rebase and push the commit to trigger the pre-commit check again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27010) container images for HBase
Sean Busbey created HBASE-27010: --- Summary: container images for HBase Key: HBASE-27010 URL: https://issues.apache.org/jira/browse/HBASE-27010 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need container images of the build that got made during the nightly run -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27008) container images for Hadoop
Sean Busbey created HBASE-27008: --- Summary: container images for Hadoop Key: HBASE-27008 URL: https://issues.apache.org/jira/browse/HBASE-27008 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need image(s) for the hadoop cluster we run the nightly tests on top of -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27009) container images for ZooKeeper
Sean Busbey created HBASE-27009: --- Summary: container images for ZooKeeper Key: HBASE-27009 URL: https://issues.apache.org/jira/browse/HBASE-27009 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey we need image(s) for a zk cluster for our nightly cluster run -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] carp84 merged pull request #4404: HBASE-26998 TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in branch-1
carp84 merged PR #4404: URL: https://github.com/apache/hbase/pull/4404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27007) nightly support scripts for install/setup of single-node k8s env
Sean Busbey created HBASE-27007: --- Summary: nightly support scripts for install/setup of single-node k8s env Key: HBASE-27007 URL: https://issues.apache.org/jira/browse/HBASE-27007 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey minikube , kind, something like that. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] carp84 commented on pull request #4404: HBASE-26998 TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in branch-1
carp84 commented on PR #4404: URL: https://github.com/apache/hbase/pull/4404#issuecomment-1119749215 Thanks @Apache9 for review. I will merge this one since all failed test cases are irrelative to changes here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27006) cordon off large ci worker nodes
Sean Busbey created HBASE-27006: --- Summary: cordon off large ci worker nodes Key: HBASE-27006 URL: https://issues.apache.org/jira/browse/HBASE-27006 Project: HBase Issue Type: Sub-task Reporter: Sean Busbey Make sure our node labeling keeps test workloads that off of the new large spec nodes so that they're only used for the nightly k8s based tests -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (HBASE-27005) move nightly cluster tests to run on k8s
Sean Busbey created HBASE-27005: --- Summary: move nightly cluster tests to run on k8s Key: HBASE-27005 URL: https://issues.apache.org/jira/browse/HBASE-27005 Project: HBase Issue Type: Umbrella Components: integration tests Reporter: Sean Busbey -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4258: HBASE-25768 Support an overall coarse and fast balance strategy for StochasticLoadBalancer
Apache-HBase commented on PR #4258: URL: https://github.com/apache/hbase/pull/4258#issuecomment-1119744466 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 48s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 1m 45s | root in master failed. | | -1 :x: | compile | 0m 11s | hbase-balancer in master failed. | | -1 :x: | compile | 0m 27s | hbase-server in master failed. | | +1 :green_heart: | shadedjars | 4m 24s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 7s | hbase-balancer in master failed. | | -0 :warning: | javadoc | 0m 7s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 5s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 34s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 24s | hbase-server generated 23 new + 0 unchanged - 0 fixed = 23 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 59s | hbase-balancer in the patch passed. | | -1 :x: | unit | 257m 20s | hbase-server in the patch failed. | | | | 287m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4258 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ebaf295f9649 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/branch-mvninstall-root.txt | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/branch-compile-hbase-balancer.txt | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/branch-compile-hbase-server.txt | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/branch-javadoc-hbase-balancer.txt | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/testReport/ | | Max. process+thread count | 2676 (vs. ulimit of 3) | | modules | C: hbase-balancer hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26977) HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
[ https://issues.apache.org/jira/browse/HBASE-26977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532927#comment-17532927 ] LiangJun He commented on HBASE-26977: - I will do it, Thanks. [~zhangduo] > HMaster's ShutdownHook does not take effect, if tablesOnMaster is false > --- > > Key: HBASE-26977 > URL: https://issues.apache.org/jira/browse/HBASE-26977 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.7.1, 2.4.9 >Reporter: LiangJun He >Assignee: LiangJun He >Priority: Major > Attachments: image-2022-04-26-21-12-51-843.png, > image-2022-04-26-21-18-24-884.png, image-2022-04-26-21-22-38-572.png > > > Because we are going to upgrade the HBase kernel version, during the test, I > found that the HMaster ShutdownHook of version 1.7.1 and 2.4.9 did not take > effect if tablesOnMaster is false. After further confirmation, I found that > all versions of HBase 1.x and 2.x have this problem. > I analyzed the cause of the problem： > Before HBase 3.x, HMaster registered ShutdownHook by calling the run() method > of HRegionServer, but the run() method will be blocked on the > waitForMasterActive() method of HMaster(before registering the shutdownhook) > if tablesOnMaster is false. Eventually, the ShutdownHook of HMaster was not > registered successfully. > 1. The code path of HRegionServer's run() method： > !image-2022-04-26-21-18-24-884.png! > > 2. The preRegistrationInitialization() method will be called first: > !image-2022-04-26-21-12-51-843.png! > 3. The HMaster's waitForMasterActive() method will be blocked if > tablesOnMaster is false: > !image-2022-04-26-21-22-38-572.png! > Finally, because the default value of tablesOnMaster is false, the > ShutdownHook of HMaster was not registered successfully. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27004) Client integration test is failing on branch-2.x
[
https://issues.apache.org/jira/browse/HBASE-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532921#comment-17532921
]
Duo Zhang commented on HBASE-27004:
---
OK, I think this an environment problem.
In /etc/hosts, we have this
{noformat}
127.0.0.1 localhost
# 172.31.12.81 jenkins-hbase3
44.239.2.152 jenkins-hbase3.apache.org jenkins-hbase3§
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
{noformat}
And run ifconfig, we have this
{noformat}
docker0: flags=4099 mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
inet6 fe80::42:19ff:fe1f:d548 prefixlen 64 scopeid 0x20
ether 02:42:19:1f:d5:48 txqueuelen 0 (Ethernet)
RX packets 1218934 bytes 104357808 (104.3 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1584095 bytes 12223746408 (12.2 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens5: flags=4163 mtu 9001
inet 172.31.12.81 netmask 255.255.240.0 broadcast 172.31.15.255
inet6 fe80::819:d0ff:fef7:4d27 prefixlen 64 scopeid 0x20
ether 0a:19:d0:f7:4d:27 txqueuelen 1000 (Ethernet)
RX packets 77029263 bytes 97160203631 (97.1 GB)
RX errors 0 dropped 318 overruns 0 frame 0
TX packets 68782755 bytes 68879173234 (68.8 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1000 (Local Loopback)
RX packets 29587206 bytes 4643512773 (4.6 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 29587206 bytes 4643512773 (4.6 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
{noformat}
So here, 44.239.2.152 should be EIP of this node, and we resolve
jenkins-hbase3.apache.org to this EIP and try to bind it, then obviously, we
will get this error, because there is no NIC has this IP address...
Let me file an INFRA ticket about this.
> Client integration test is failing on branch-2.x
>
>
> Key: HBASE-27004
> URL: https://issues.apache.org/jira/browse/HBASE-27004
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
>
> It is because we can not start hadoop cluster.
> The error is
> {noformat}
> 22/05/05 20:04:18 INFO service.AbstractService: Service HistoryClientService
> failed in state STARTED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [jenkins-hbase3.apache.org:0]
> java.net.BindException: Cannot assign requested address; For more details
> see: http://wiki.apache.org/hadoop/BindException
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.net.BindException: Problem binding to [jenkins-hbase3.apache.org:0]
> java.net.BindException: Cannot assign requested address; For more details
> see: http://wiki.apache.org/hadoop/BindException
> at
> org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:138)
> at
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
> at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.serviceStart(HistoryClientService.java:128)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceStart(JobHistoryServer.java:202)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper$1.run(MiniMRYarnCluster.java:236)
> Caused by: java.net.BindException: Problem binding to
> [jenkins-hbase3.apache.org:0] java.net.BindException: Cannot assign requested
> address; For more details see: http://wiki.apache.org/hadoop/BindException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:735)
> at
[GitHub] [hbase] Apache-HBase commented on pull request #4258: HBASE-25768 Support an overall coarse and fast balance strategy for StochasticLoadBalancer
Apache-HBase commented on PR #4258: URL: https://github.com/apache/hbase/pull/4258#issuecomment-1119730748 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 35s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 4m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 2s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 4m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 10m 23s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 238m 59s | hbase-server in the patch passed. | | | | 273m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4258 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3241c1f19eb5 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/testReport/ | | Max. process+thread count | 2761 (vs. ulimit of 3) | | modules | C: hbase-balancer hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27004) Client integration test is failing on branch-2.x
Duo Zhang created HBASE-27004:
-
Summary: Client integration test is failing on branch-2.x
Key: HBASE-27004
URL: https://issues.apache.org/jira/browse/HBASE-27004
Project: HBase
Issue Type: Bug
Reporter: Duo Zhang
It is because we can not start hadoop cluster.
The error is
{noformat}
22/05/05 20:04:18 INFO service.AbstractService: Service HistoryClientService
failed in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException:
Problem binding to [jenkins-hbase3.apache.org:0] java.net.BindException: Cannot
assign requested address; For more details see:
http://wiki.apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException:
Problem binding to [jenkins-hbase3.apache.org:0] java.net.BindException: Cannot
assign requested address; For more details see:
http://wiki.apache.org/hadoop/BindException
at
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:138)
at
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65)
at org.apache.hadoop.yarn.ipc.YarnRPC.getServer(YarnRPC.java:54)
at
org.apache.hadoop.mapreduce.v2.hs.HistoryClientService.serviceStart(HistoryClientService.java:128)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceStart(JobHistoryServer.java:202)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at
org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper$1.run(MiniMRYarnCluster.java:236)
Caused by: java.net.BindException: Problem binding to
[jenkins-hbase3.apache.org:0] java.net.BindException: Cannot assign requested
address; For more details see: http://wiki.apache.org/hadoop/BindException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:735)
at org.apache.hadoop.ipc.Server.bind(Server.java:615)
at org.apache.hadoop.ipc.Server$Listener.(Server.java:1179)
at org.apache.hadoop.ipc.Server.(Server.java:2983)
at org.apache.hadoop.ipc.RPC$Server.(RPC.java:1003)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:425)
at
org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:346)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:844)
at
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.createServer(RpcServerFactoryPBImpl.java:168)
at
org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:131)
... 8 more
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:438)
at sun.nio.ch.Net.bind(Net.java:430)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:225)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.hadoop.ipc.Server.bind(Server.java:598)
... 16 more
{noformat}
https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/534/artifact/output-integration/hadoop-2/hadoop_cluster_command.err/*view*/
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] tomscut commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
tomscut commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119689033 Thanks @wchevreuil for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4407: HBASE-26999 HStore should try write WAL compaction marker before repl…
Apache-HBase commented on PR #4407: URL: https://github.com/apache/hbase/pull/4407#issuecomment-1119648054 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 33s | master passed | | +1 :green_heart: | compile | 2m 44s | master passed | | +1 :green_heart: | checkstyle | 0m 32s | master passed | | +1 :green_heart: | spotless | 0m 47s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 31s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 4s | the patch passed | | +1 :green_heart: | compile | 2m 38s | the patch passed | | +1 :green_heart: | javac | 2m 38s | the patch passed | | -0 :warning: | checkstyle | 0m 30s | hbase-server: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 57s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 46s | patch has 63 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 43m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4407 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 1f18946135ce 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4407/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26977) HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
[ https://issues.apache.org/jira/browse/HBASE-26977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532852#comment-17532852 ] Duo Zhang commented on HBASE-26977: --- Seems the branch-2 commit breaks the spotless check... Please run a spotless:apply to append an addendum [~liyu] [~heliangjun]. Thanks. > HMaster's ShutdownHook does not take effect, if tablesOnMaster is false > --- > > Key: HBASE-26977 > URL: https://issues.apache.org/jira/browse/HBASE-26977 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 1.7.1, 2.4.9 >Reporter: LiangJun He >Assignee: LiangJun He >Priority: Major > Attachments: image-2022-04-26-21-12-51-843.png, > image-2022-04-26-21-18-24-884.png, image-2022-04-26-21-22-38-572.png > > > Because we are going to upgrade the HBase kernel version, during the test, I > found that the HMaster ShutdownHook of version 1.7.1 and 2.4.9 did not take > effect if tablesOnMaster is false. After further confirmation, I found that > all versions of HBase 1.x and 2.x have this problem. > I analyzed the cause of the problem： > Before HBase 3.x, HMaster registered ShutdownHook by calling the run() method > of HRegionServer, but the run() method will be blocked on the > waitForMasterActive() method of HMaster(before registering the shutdownhook) > if tablesOnMaster is false. Eventually, the ShutdownHook of HMaster was not > registered successfully. > 1. The code path of HRegionServer's run() method： > !image-2022-04-26-21-18-24-884.png! > > 2. The preRegistrationInitialization() method will be called first: > !image-2022-04-26-21-12-51-843.png! > 3. The HMaster's waitForMasterActive() method will be blocked if > tablesOnMaster is false: > !image-2022-04-26-21-22-38-572.png! > Finally, because the default value of tablesOnMaster is false, the > ShutdownHook of HMaster was not registered successfully. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119575452 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 19s | master passed | | +1 :green_heart: | compile | 0m 16s | master passed | | +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 3m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 12m 40s | hbase-mapreduce in the patch passed. | | | | 27m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 524a66f9dcfc 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/testReport/ | | Max. process+thread count | 3214 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119575262 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 3s | master passed | | +1 :green_heart: | compile | 0m 14s | master passed | | +1 :green_heart: | shadedjars | 3m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 6s | the patch passed | | +1 :green_heart: | compile | 0m 15s | the patch passed | | +1 :green_heart: | javac | 0m 15s | the patch passed | | +1 :green_heart: | shadedjars | 3m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 12m 30s | hbase-mapreduce in the patch passed. | | | | 26m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fdc1afee7a89 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/testReport/ | | Max. process+thread count | 2822 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119574254 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | master passed | | +1 :green_heart: | compile | 0m 32s | master passed | | +1 :green_heart: | checkstyle | 0m 12s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 37s | patch has 26 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 0m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 25m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 07408bd20ab7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4258: HBASE-25768 Support an overall coarse and fast balance strategy for StochasticLoadBalancer
Apache-HBase commented on PR #4258: URL: https://github.com/apache/hbase/pull/4258#issuecomment-1119510874 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 27s | master passed | | +1 :green_heart: | compile | 2m 36s | master passed | | +1 :green_heart: | checkstyle | 0m 41s | master passed | | +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 17s | the patch passed | | +1 :green_heart: | compile | 2m 35s | the patch passed | | -0 :warning: | javac | 0m 22s | hbase-balancer generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) | | +1 :green_heart: | checkstyle | 0m 40s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 35m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4258 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux e6d798d4c737 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-balancer hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4258/11/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119493864 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 4m 7s | root in master failed. | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 12m 37s | hbase-mapreduce in the patch passed. | | | | 29m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 64871a4e14d1 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/artifact/yetus-jdk11-hadoop3-check/output/branch-mvninstall-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/testReport/ | | Max. process+thread count | 3246 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119493665 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | compile | 0m 16s | master passed | | +1 :green_heart: | shadedjars | 3m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 15s | the patch passed | | +1 :green_heart: | javac | 0m 15s | the patch passed | | +1 :green_heart: | shadedjars | 3m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 12m 31s | hbase-mapreduce in the patch passed. | | | | 28m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 63010c82691a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/testReport/ | | Max. process+thread count | 2776 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4411: HBASE-27003 Optimize log format for PerformanceEvaluation
Apache-HBase commented on PR #4411: URL: https://github.com/apache/hbase/pull/4411#issuecomment-1119492122 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 59s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | master passed | | +1 :green_heart: | compile | 0m 32s | master passed | | +1 :green_heart: | checkstyle | 0m 12s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 30s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 35s | patch has 26 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 26m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4411 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 67bf5891f5ff 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-mapreduce U: hbase-mapreduce | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4411/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25768) Support an overall coarse and fast balance strategy for StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532775#comment-17532775 ] Xiaolin Ha commented on HBASE-25768: Hi, [~kingWang] , the patch is for branch-2.4+. If you want to apply this patch for 2.1.0, most improvements since 2.1.0 is not required here. The main changes here are the chosen cost functions. You can try to back port it if you have interest. Thanks. > Support an overall coarse and fast balance strategy for StochasticLoadBalancer > -- > > Key: HBASE-25768 > URL: https://issues.apache.org/jira/browse/HBASE-25768 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 3.0.0-alpha-1, 2.0.0, 1.4.13 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > When we use StochasticLoadBalancer + balanceByTable, we could face two > difficulties. > # For each table, their regions are distributed uniformly, but for the > overall cluster, still exiting imbalance between RSes; > # When there are large-scaled restart of RSes, or expansion for groups or > cluster, we hope the balancer can execute as soon as possible, but the > StochasticLoadBalancer may need a lot of time to compute costs. > We can detect these circumstances in StochasticLoadBalancer(such as using the > percentage of skew tables), and before the normal balance steps trying, we > can add a strategy to let it just balance like the SimpleLoadBalancer or use > few light cost functions here. > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4410: HBASE-27002 Config BucketCache as victim handler of LRUCache
Apache-HBase commented on PR #4410: URL: https://github.com/apache/hbase/pull/4410#issuecomment-1119485678 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | master passed | | +1 :green_heart: | compile | 0m 37s | master passed | | +1 :green_heart: | shadedjars | 3m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 33s | the patch passed | | +1 :green_heart: | javac | 0m 33s | the patch passed | | +1 :green_heart: | shadedjars | 3m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 202m 45s | hbase-server in the patch failed. | | | | 219m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4410 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux eca0c5b87a7a 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/testReport/ | | Max. process+thread count | 3001 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #4361: HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer.
wchevreuil commented on PR #4361: URL: https://github.com/apache/hbase/pull/4361#issuecomment-1119484240 > Looks like Wellington is busy/unavailable. @Apache9 Do you have any suggestions on this patch ? @apurtell @virajjasani Can you please help review this patch ? Thank you. Sorry @shahrs87 , missed your request, previously. Adding this extra check for the source thread state makes sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] tomscut opened a new pull request, #4411: HBASE-27003. Optimize log format for PerformanceEvaluation
tomscut opened a new pull request, #4411: URL: https://github.com/apache/hbase/pull/4411 JIRA: HBASE-27003. The logs in PerformanceEvaluation look a little confusing to new users, we should optimize the format. Before:  After:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4410: HBASE-27002 Config BucketCache as victim handler of LRUCache
Apache-HBase commented on PR #4410: URL: https://github.com/apache/hbase/pull/4410#issuecomment-1119466920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 48s | master passed | | +1 :green_heart: | compile | 0m 43s | master passed | | +1 :green_heart: | shadedjars | 3m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 29s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | shadedjars | 3m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 175m 22s | hbase-server in the patch passed. | | | | 193m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4410 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cf8c6a1e8f69 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/testReport/ | | Max. process+thread count | 2933 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27003) Optimize log format for PerformanceEvaluation
Tao Li created HBASE-27003: -- Summary: Optimize log format for PerformanceEvaluation Key: HBASE-27003 URL: https://issues.apache.org/jira/browse/HBASE-27003 Project: HBase Issue Type: Improvement Reporter: Tao Li Assignee: Tao Li Attachments: image-2022-05-06-18-13-50-763.png, image-2022-05-06-18-14-28-578.png, image-2022-05-06-18-15-13-913.png The logs in PerformanceEvaluation look a little confusing to new users, we should optimize the format. Before: !image-2022-05-06-18-13-50-763.png|width=787,height=156! After: !image-2022-05-06-18-15-13-913.png|width=674,height=147! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26995) Remove ref guide check in pre commit and nightly for branches other than master
[
https://issues.apache.org/jira/browse/HBASE-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532756#comment-17532756
]
Hudson commented on HBASE-26995:
Results for branch branch-1
[build #223 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-1/223/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-1/223//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-1/223//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-1/223//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Remove ref guide check in pre commit and nightly for branches other than
> master
> ---
>
> Key: HBASE-26995
> URL: https://issues.apache.org/jira/browse/HBASE-26995
> Project: HBase
> Issue Type: Sub-task
> Components: build, scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 1.7.2, 3.0.0-alpha-3, 2.4.13
>
>
> Our nightly builds are failing after HBASE-26932. Let's fix it.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26999) HStore should try write WAL compaction marker before replacing compacted files in StoreEngine
[
https://issues.apache.org/jira/browse/HBASE-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532752#comment-17532752
]
Wellington Chevreuil commented on HBASE-26999:
--
I was thinking we needed locking here to avoid race conditions between flushes
and compactions within the same RS. But I realised now File SFT keeps it's own
in-memory list of store files that gets updated within a lock context, so I
guess we are good to do as you said. Let me update the PR.
> HStore should try write WAL compaction marker before replacing compacted
> files in StoreEngine
> -
>
> Key: HBASE-26999
> URL: https://issues.apache.org/jira/browse/HBASE-26999
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> On HBASE-26064, it seems we altered the order we update different places with
> the results of a compaction:
> {noformat}
> @@ -1510,14 +1149,13 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> List newFiles) throws IOException {
> // Do the steps necessary to complete the compaction.
> setStoragePolicyFromFileName(newFiles);
> - List sfs = commitStoreFiles(newFiles, true);
> + List sfs = storeEngine.commitStoreFiles(newFiles, true);
> if (this.getCoprocessorHost() != null) {
> for (HStoreFile sf : sfs) {
> getCoprocessorHost().postCompact(this, sf, cr.getTracker(), cr,
> user);
> }
> }
> - writeCompactionWalRecord(filesToCompact, sfs);
> - replaceStoreFiles(filesToCompact, sfs);
> + replaceStoreFiles(filesToCompact, sfs, true);
> ...
> @@ -1581,25 +1219,24 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> this.region.getRegionInfo(), compactionDescriptor,
> this.region.getMVCC());
> }
>
> - void replaceStoreFiles(Collection compactedFiles,
> Collection result)
> - throws IOException {
> - this.lock.writeLock().lock();
> - try {
> -
> this.storeEngine.getStoreFileManager().addCompactionResults(compactedFiles,
> result);
> - synchronized (filesCompacting) {
> - filesCompacting.removeAll(compactedFiles);
> - }
> -
> - // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> - // sizes later so it's not super-critical if we miss these.
> - RegionServerServices rsServices = region.getRegionServerServices();
> - if (rsServices != null &&
> rsServices.getRegionServerSpaceQuotaManager() != null) {
> - updateSpaceQuotaAfterFileReplacement(
> -
> rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> - compactedFiles, result);
> - }
> - } finally {
> - this.lock.writeLock().unlock();
> + @RestrictedApi(explanation = "Should only be called in TestHStore", link =
> "",
> + allowedOnPath = ".*/(HStore|TestHStore).java")
> + void replaceStoreFiles(Collection compactedFiles,
> Collection result,
> + boolean writeCompactionMarker) throws IOException {
> + storeEngine.replaceStoreFiles(compactedFiles, result);
> + if (writeCompactionMarker) {
> + writeCompactionWalRecord(compactedFiles, result);
> + }
> + synchronized (filesCompacting) {
> + filesCompacting.removeAll(compactedFiles);
> + }
> + // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> + // sizes later so it's not super-critical if we miss these.
> + RegionServerServices rsServices = region.getRegionServerServices();
> + if (rsServices != null && rsServices.getRegionServerSpaceQuotaManager()
> != null) {
> + updateSpaceQuotaAfterFileReplacement(
> + rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> + compactedFiles, result); {noformat}
> While running some large scale load test, we run into File SFT metafiles
> inconsistency that we believe could have been avoided if the original order
> was in place. Here the scenario we had:
> 1) Region R with one CF f was open on RS1. At this time, the given store had
> some files, let's say these were file1, file2 and file3;
> 2) Compaction started on RS1;
> 3) RS1 entered a long GC pause, lost ZK lock. Compaction is still running,
> though.
> 4) RS2 opens R. The related File SFT instance for this store then creates a
> new meta file with file1, file2 and file3.
> 5) Compaction on RS1 successfully completes the
> *storeEngine.replaceStoreFiles* call. This updates the in memory cache of
> valid files (StoreFileManager) and the SFT metafile for the store engine on
> RS1 with the compaction resulting file, say file4, removing file1,
[GitHub] [hbase] skysiders commented on pull request #4379: HBASE-26985 check permission for SecureBulkLoadManager
skysiders commented on PR #4379: URL: https://github.com/apache/hbase/pull/4379#issuecomment-1119417035 Hi @virajjasani ,could you please have a look this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26999) HStore should try write WAL compaction marker before replacing compacted files in StoreEngine
[
https://issues.apache.org/jira/browse/HBASE-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532723#comment-17532723
]
Duo Zhang commented on HBASE-26999:
---
{quote}
Not if we put both the update and rollback in the same lock context
{quote}
This is a no no. You need to hold the write lock all the time when writing SFT
files and also writing WAL, this will block everything...
Thanks...
> HStore should try write WAL compaction marker before replacing compacted
> files in StoreEngine
> -
>
> Key: HBASE-26999
> URL: https://issues.apache.org/jira/browse/HBASE-26999
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> On HBASE-26064, it seems we altered the order we update different places with
> the results of a compaction:
> {noformat}
> @@ -1510,14 +1149,13 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> List newFiles) throws IOException {
> // Do the steps necessary to complete the compaction.
> setStoragePolicyFromFileName(newFiles);
> - List sfs = commitStoreFiles(newFiles, true);
> + List sfs = storeEngine.commitStoreFiles(newFiles, true);
> if (this.getCoprocessorHost() != null) {
> for (HStoreFile sf : sfs) {
> getCoprocessorHost().postCompact(this, sf, cr.getTracker(), cr,
> user);
> }
> }
> - writeCompactionWalRecord(filesToCompact, sfs);
> - replaceStoreFiles(filesToCompact, sfs);
> + replaceStoreFiles(filesToCompact, sfs, true);
> ...
> @@ -1581,25 +1219,24 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> this.region.getRegionInfo(), compactionDescriptor,
> this.region.getMVCC());
> }
>
> - void replaceStoreFiles(Collection compactedFiles,
> Collection result)
> - throws IOException {
> - this.lock.writeLock().lock();
> - try {
> -
> this.storeEngine.getStoreFileManager().addCompactionResults(compactedFiles,
> result);
> - synchronized (filesCompacting) {
> - filesCompacting.removeAll(compactedFiles);
> - }
> -
> - // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> - // sizes later so it's not super-critical if we miss these.
> - RegionServerServices rsServices = region.getRegionServerServices();
> - if (rsServices != null &&
> rsServices.getRegionServerSpaceQuotaManager() != null) {
> - updateSpaceQuotaAfterFileReplacement(
> -
> rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> - compactedFiles, result);
> - }
> - } finally {
> - this.lock.writeLock().unlock();
> + @RestrictedApi(explanation = "Should only be called in TestHStore", link =
> "",
> + allowedOnPath = ".*/(HStore|TestHStore).java")
> + void replaceStoreFiles(Collection compactedFiles,
> Collection result,
> + boolean writeCompactionMarker) throws IOException {
> + storeEngine.replaceStoreFiles(compactedFiles, result);
> + if (writeCompactionMarker) {
> + writeCompactionWalRecord(compactedFiles, result);
> + }
> + synchronized (filesCompacting) {
> + filesCompacting.removeAll(compactedFiles);
> + }
> + // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> + // sizes later so it's not super-critical if we miss these.
> + RegionServerServices rsServices = region.getRegionServerServices();
> + if (rsServices != null && rsServices.getRegionServerSpaceQuotaManager()
> != null) {
> + updateSpaceQuotaAfterFileReplacement(
> + rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> + compactedFiles, result); {noformat}
> While running some large scale load test, we run into File SFT metafiles
> inconsistency that we believe could have been avoided if the original order
> was in place. Here the scenario we had:
> 1) Region R with one CF f was open on RS1. At this time, the given store had
> some files, let's say these were file1, file2 and file3;
> 2) Compaction started on RS1;
> 3) RS1 entered a long GC pause, lost ZK lock. Compaction is still running,
> though.
> 4) RS2 opens R. The related File SFT instance for this store then creates a
> new meta file with file1, file2 and file3.
> 5) Compaction on RS1 successfully completes the
> *storeEngine.replaceStoreFiles* call. This updates the in memory cache of
> valid files (StoreFileManager) and the SFT metafile for the store engine on
> RS1 with the compaction resulting file, say file4, removing file1, file2 and
> file3. Note that the SFT meta file used by RS1 here is different (older)
[jira] [Commented] (HBASE-26999) HStore should try write WAL compaction marker before replacing compacted files in StoreEngine
[
https://issues.apache.org/jira/browse/HBASE-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532716#comment-17532716
]
Wellington Chevreuil commented on HBASE-26999:
--
{quote}
but for #2 and #3, there could be race, so even rollback is not safe here.
{quote}
Not if we put both the update and rollback in the same lock context, and make
sure #2 and #3 have a read lock before asking for the list of compacted files
from SFM (check the latest commits on the PR).
> HStore should try write WAL compaction marker before replacing compacted
> files in StoreEngine
> -
>
> Key: HBASE-26999
> URL: https://issues.apache.org/jira/browse/HBASE-26999
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> On HBASE-26064, it seems we altered the order we update different places with
> the results of a compaction:
> {noformat}
> @@ -1510,14 +1149,13 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> List newFiles) throws IOException {
> // Do the steps necessary to complete the compaction.
> setStoragePolicyFromFileName(newFiles);
> - List sfs = commitStoreFiles(newFiles, true);
> + List sfs = storeEngine.commitStoreFiles(newFiles, true);
> if (this.getCoprocessorHost() != null) {
> for (HStoreFile sf : sfs) {
> getCoprocessorHost().postCompact(this, sf, cr.getTracker(), cr,
> user);
> }
> }
> - writeCompactionWalRecord(filesToCompact, sfs);
> - replaceStoreFiles(filesToCompact, sfs);
> + replaceStoreFiles(filesToCompact, sfs, true);
> ...
> @@ -1581,25 +1219,24 @@ public class HStore implements Store, HeapSize,
> StoreConfigInformation,
> this.region.getRegionInfo(), compactionDescriptor,
> this.region.getMVCC());
> }
>
> - void replaceStoreFiles(Collection compactedFiles,
> Collection result)
> - throws IOException {
> - this.lock.writeLock().lock();
> - try {
> -
> this.storeEngine.getStoreFileManager().addCompactionResults(compactedFiles,
> result);
> - synchronized (filesCompacting) {
> - filesCompacting.removeAll(compactedFiles);
> - }
> -
> - // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> - // sizes later so it's not super-critical if we miss these.
> - RegionServerServices rsServices = region.getRegionServerServices();
> - if (rsServices != null &&
> rsServices.getRegionServerSpaceQuotaManager() != null) {
> - updateSpaceQuotaAfterFileReplacement(
> -
> rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> - compactedFiles, result);
> - }
> - } finally {
> - this.lock.writeLock().unlock();
> + @RestrictedApi(explanation = "Should only be called in TestHStore", link =
> "",
> + allowedOnPath = ".*/(HStore|TestHStore).java")
> + void replaceStoreFiles(Collection compactedFiles,
> Collection result,
> + boolean writeCompactionMarker) throws IOException {
> + storeEngine.replaceStoreFiles(compactedFiles, result);
> + if (writeCompactionMarker) {
> + writeCompactionWalRecord(compactedFiles, result);
> + }
> + synchronized (filesCompacting) {
> + filesCompacting.removeAll(compactedFiles);
> + }
> + // These may be null when the RS is shutting down. The space quota
> Chores will fix the Region
> + // sizes later so it's not super-critical if we miss these.
> + RegionServerServices rsServices = region.getRegionServerServices();
> + if (rsServices != null && rsServices.getRegionServerSpaceQuotaManager()
> != null) {
> + updateSpaceQuotaAfterFileReplacement(
> + rsServices.getRegionServerSpaceQuotaManager().getRegionSizeStore(),
> getRegionInfo(),
> + compactedFiles, result); {noformat}
> While running some large scale load test, we run into File SFT metafiles
> inconsistency that we believe could have been avoided if the original order
> was in place. Here the scenario we had:
> 1) Region R with one CF f was open on RS1. At this time, the given store had
> some files, let's say these were file1, file2 and file3;
> 2) Compaction started on RS1;
> 3) RS1 entered a long GC pause, lost ZK lock. Compaction is still running,
> though.
> 4) RS2 opens R. The related File SFT instance for this store then creates a
> new meta file with file1, file2 and file3.
> 5) Compaction on RS1 successfully completes the
> *storeEngine.replaceStoreFiles* call. This updates the in memory cache of
> valid files (StoreFileManager) and the SFT metafile for the store engine on
> RS1 with the compaction resulting file, say file4, removing file1,
[GitHub] [hbase] Apache-HBase commented on pull request #4404: HBASE-26998 TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in branch-1
Apache-HBase commented on PR #4404: URL: https://github.com/apache/hbase/pull/4404#issuecomment-1119399926 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 3s | branch-1 passed | | +1 :green_heart: | compile | 0m 27s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 29s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 0m 54s | branch-1 passed | | +1 :green_heart: | shadedjars | 1m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 25s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 1m 42s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 39s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 16s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 54s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 1m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 2m 58s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 20s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 1m 47s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 153m 29s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 177m 0s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestMasterRegistry | | | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles | | | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL | | | hadoop.hbase.replication.TestReplicationEndpoint | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4404/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4404 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux efec4955bf67 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4404/out/precommit/personality/provided.sh | | git revision | branch-1 / c6b240bd91 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4404/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4404/4/testReport/ | | Max. process+thread count | 3652 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4404/4/console | | versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 | | Powered by | Apache Yetus
[GitHub] [hbase] skysiders commented on pull request #4391: HBASE-26994 MasterFileSystem create directory without permission check
skysiders commented on PR #4391: URL: https://github.com/apache/hbase/pull/4391#issuecomment-1119341689 Hi @virajjasani , thanks for your review.I have another PR [HBASE-26895](https://github.com/apache/hbase/pull/4379/files).Could you please have a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4410: HBASE-27002 Config BucketCache as victim handler of LRUCache
Apache-HBase commented on PR #4410: URL: https://github.com/apache/hbase/pull/4410#issuecomment-1119341457 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | master passed | | +1 :green_heart: | compile | 2m 17s | master passed | | +1 :green_heart: | checkstyle | 0m 31s | master passed | | +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 2s | the patch passed | | +1 :green_heart: | compile | 2m 13s | the patch passed | | +1 :green_heart: | javac | 2m 13s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 32m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4410 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 89e9d5686303 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e9046a64f4 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 65 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4410/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27002) Config BucketCache as victim handler of LRUCache
[
https://issues.apache.org/jira/browse/HBASE-27002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuesen Liang updated HBASE-27002:
-
Summary: Config BucketCache as victim handler of LRUCache (was: Config
BucketCache as victim of LRUCache)
> Config BucketCache as victim handler of LRUCache
>
>
> Key: HBASE-27002
> URL: https://issues.apache.org/jira/browse/HBASE-27002
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, BucketCache
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
>
> Bucket cache is no longer as a victim handler for LRU by HBASE-19357.
> {noformat}
> When BC is used, data blocks will be strictly on BC only where as index/bloom
> blocks are on LRU L1 cache.
> {noformat}
> In this situation, the LRU cache's memory is totally on heap.
> If the index and bloom filters on a region server are big, then a big LRU
> cache will introduce more GC cost.
> So we should add a *configuration for user* to choose victim handler.
> A small LRU cache with a big victim offheap bucket cache can reduce GC cost.
>
> For example: a region server has 5GB index, 80 GB bloom filter, and 256GB
> DRAM, its configurations can be as follow:
> {code:xml}
> export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"
> {code}
> {code:xml}
>
> hbase.blockcache.victim.handler
> true
>
>
> hfile.block.cache.size
> 0.3
>
>
> hbase.bucketcache.ioengine
> offheap
>
>
> hbase.bucketcache.size
> 16
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] liangxs opened a new pull request, #4410: HBASE-27002 Config BucketCache as victim of LRUCache
liangxs opened a new pull request, #4410: URL: https://github.com/apache/hbase/pull/4410 HBase JIRA: [HBASE-27002](https://issues.apache.org/jira/browse/HBASE-27002) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27002) Config BucketCache as victim of LRUCache
[
https://issues.apache.org/jira/browse/HBASE-27002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuesen Liang updated HBASE-27002:
-
Description:
Bucket cache is no longer as a victim handler for LRU by HBASE-19357.
{noformat}
When BC is used, data blocks will be strictly on BC only where as index/bloom
blocks are on LRU L1 cache.
{noformat}
In this situation, the LRU cache's memory is totally on heap.
If the index and bloom filters on a region server are big, then a big LRU cache
will introduce more GC cost.
So we should add a *configuration for user* to choose victim handler.
A small LRU cache with a big victim offheap bucket cache can reduce GC cost.
For example: a region server has 5GB index, 80 GB bloom filter, and 256GB DRAM,
its configurations can be as follow:
{code:xml}
export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"
{code}
{code:xml}
hbase.blockcache.victim.handler
true
hfile.block.cache.size
0.3
hbase.bucketcache.ioengine
offheap
hbase.bucketcache.size
16
{code}
was:
Bucket cache is no longer as a victim handler for LRU by HBASE-19357.
{noformat}
When BC is used, data blocks will be strictly on BC only where as index/bloom
blocks are on LRU L1 cache.
{noformat}
In this situation, the LRU cache's memory is totally on heap.
If the index and bloom filters on a region server are big, then a big LRU cache
will introduce more GC cost.
So we should add a *configuration for user* to choose victim handler.
A small LRU cache with a big victim offheap bucket cache can reduce GC cost.
For example: a region server has 5GB index, 80 GB bloom filter, and 256GB DRAM,
its configurations can be as follow:
{code:xml}
export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"
{code}
{code:xml}
hbase.bucketcache.as.lrucache.victim
true
hfile.block.cache.size
0.3
hbase.bucketcache.ioengine
offheap
hbase.bucketcache.size
16
{code}
> Config BucketCache as victim of LRUCache
>
>
> Key: HBASE-27002
> URL: https://issues.apache.org/jira/browse/HBASE-27002
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, BucketCache
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
>
> Bucket cache is no longer as a victim handler for LRU by HBASE-19357.
> {noformat}
> When BC is used, data blocks will be strictly on BC only where as index/bloom
> blocks are on LRU L1 cache.
> {noformat}
> In this situation, the LRU cache's memory is totally on heap.
> If the index and bloom filters on a region server are big, then a big LRU
> cache will introduce more GC cost.
> So we should add a *configuration for user* to choose victim handler.
> A small LRU cache with a big victim offheap bucket cache can reduce GC cost.
>
> For example: a region server has 5GB index, 80 GB bloom filter, and 256GB
> DRAM, its configurations can be as follow:
> {code:xml}
> export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"
> {code}
> {code:xml}
>
> hbase.blockcache.victim.handler
> true
>
>
> hfile.block.cache.size
> 0.3
>
>
> hbase.bucketcache.ioengine
> offheap
>
>
> hbase.bucketcache.size
> 16
>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] carp84 commented on pull request #4404: HBASE-26998 TestRegionProcessRowsWithLocks#testProcessExceptionAndRollBack is broken in branch-1
carp84 commented on PR #4404: URL: https://github.com/apache/hbase/pull/4404#issuecomment-1119307544 The branch-1 test cases seems to be unstable... Have triggered another round of pre-commit check to see the result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-27000) Block cache stats (Misses Caching) display error in RS web UI
[ https://issues.apache.org/jira/browse/HBASE-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-27000. -- Fix Version/s: 2.5.0 3.0.0-alpha-3 2.4.13 Hadoop Flags: Reviewed Resolution: Fixed > Block cache stats (Misses Caching) display error in RS web UI > - > > Key: HBASE-27000 > URL: https://issues.apache.org/jira/browse/HBASE-27000 > Project: HBase > Issue Type: Bug >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13 > > Attachments: image-2022-05-05-20-11-47-884.png > > > Block cache stats (Misses Caching) display error in RS web UI. > !image-2022-05-05-20-11-47-884.png|width=547,height=303! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] virajjasani commented on pull request #4406: HBASE-27000 Block cache stats (Misses Caching) display error in RS web UI
virajjasani commented on PR #4406: URL: https://github.com/apache/hbase/pull/4406#issuecomment-1119304484 Thanks for the contribution @tomscut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27002) Config BucketCache as victim of LRUCache
Xuesen Liang created HBASE-27002:
Summary: Config BucketCache as victim of LRUCache
Key: HBASE-27002
URL: https://issues.apache.org/jira/browse/HBASE-27002
Project: HBase
Issue Type: Improvement
Components: BlockCache, BucketCache
Reporter: Xuesen Liang
Assignee: Xuesen Liang
Bucket cache is no longer as a victim handler for LRU by HBASE-19357.
{noformat}
When BC is used, data blocks will be strictly on BC only where as index/bloom
blocks are on LRU L1 cache.
{noformat}
In this situation, the LRU cache's memory is totally on heap.
If the index and bloom filters on a region server are big, then a big LRU cache
will introduce more GC cost.
So we should add a *configuration for user* to choose victim handler.
A small LRU cache with a big victim offheap bucket cache can reduce GC cost.
For example: a region server has 5GB index, 80 GB bloom filter, and 256GB DRAM,
its configurations can be as follow:
{code:xml}
export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"
{code}
{code:xml}
hbase.bucketcache.as.lrucache.victim
true
hfile.block.cache.size
0.3
hbase.bucketcache.ioengine
offheap
hbase.bucketcache.size
16
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] virajjasani merged pull request #4406: HBASE-27000 Block cache stats (Misses Caching) display error in RS web UI
virajjasani merged PR #4406: URL: https://github.com/apache/hbase/pull/4406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a diff in pull request #4391: HBASE-26994 MasterFileSystem create directory without permission check
virajjasani commented on code in PR #4391:
URL: https://github.com/apache/hbase/pull/4391#discussion_r866525469
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java:
##
@@ -355,9 +354,9 @@ private void checkStagingDir() throws IOException {
if (!this.fs.mkdirs(p, HiddenDirPerms)) {
throw new IOException("Failed to create staging directory " +
p.toString());
}
- } else {
-this.fs.setPermission(p, HiddenDirPerms);
}
+ this.fs.setPermission(p, HiddenDirPerms);
Review Comment:
Makes sense, thanks for the detailed explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a diff in pull request #4390: HBASE-26987 The length of compact queue grows too big when the compac…
bsglz commented on code in PR #4390: URL: https://github.com/apache/hbase/pull/4390#discussion_r866444359 ## hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java: ## @@ -347,7 +347,6 @@ public class HRegion implements HeapSize, PropagatingConfigurationObserver, Regi final LongAdder compactionsFailed = new LongAdder(); final LongAdder compactionNumFilesCompacted = new LongAdder(); final LongAdder compactionNumBytesCompacted = new LongAdder(); - final LongAdder compactionsQueued = new LongAdder(); Review Comment: Yeah, you are right, i thought about it too, but did not find a better way, maybe all that metrics should be moved to HStore? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
