[jira] [Updated] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Manning updated HBASE-27054:
--
Status: Patch Available (was: Open)
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: David Manning
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] d-c-manning opened a new pull request, #4454: HBASE-27054 testRegionReplicasOnLargeCluster flake fix
d-c-manning opened a new pull request, #4454: URL: https://github.com/apache/hbase/pull/4454 [HBASE-27054](https://issues.apache.org/jira/browse/HBASE-27054) Testing locally, I was able to see 100% pass rate with 15 second `maxRunningTime` after making these changes. So I changed to inherit from `StochasticBalancerTestBase` instead of `StochasticBalancerTestBase2`. The major difference in `StochasticBalancerTestBase2` for the purposes of this test is a 180 seconds `maxRunningTime` for the balancer, which should now be wasteful for this test. We can use the default 30 seconds, which we will inherit by using `StochasticBalancerTestBase`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27006) cordon off large ci worker nodes
[
https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539927#comment-17539927
]
Hudson commented on HBASE-27006:
Results for branch branch-2
[build #544 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/544/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/544/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/544/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/544/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/544/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> cordon off large ci worker nodes
>
>
> Key: HBASE-27006
> URL: https://issues.apache.org/jira/browse/HBASE-27006
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>
>
> Make sure our node labeling keeps test workloads that off of the new large
> spec nodes so that they're only used for the nightly k8s based tests
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] virajjasani commented on pull request #4390: HBASE-26987 The length of compact queue grows too big when the compac…
virajjasani commented on PR #4390: URL: https://github.com/apache/hbase/pull/4390#issuecomment-1132456029 +1 overall, also the idea seems good. @bsglz let's wait for one more review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26855) Delete unnecessary dependency on jaxb-runtime jar
[
https://issues.apache.org/jira/browse/HBASE-26855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539922#comment-17539922
]
Hudson commented on HBASE-26855:
Results for branch branch-2.4
[build #354 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Delete unnecessary dependency on jaxb-runtime jar
> -
>
> Key: HBASE-26855
> URL: https://issues.apache.org/jira/browse/HBASE-26855
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
> Since we've moved to using only shaded versions of our jersey stuff, we have
> no need for this explicit dependency.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26523) Upgrade hbase-thirdparty dependency to 4.0.1
[
https://issues.apache.org/jira/browse/HBASE-26523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539920#comment-17539920
]
Hudson commented on HBASE-26523:
Results for branch branch-2.4
[build #354 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Upgrade hbase-thirdparty dependency to 4.0.1
>
>
> Key: HBASE-26523
> URL: https://issues.apache.org/jira/browse/HBASE-26523
> Project: HBase
> Issue Type: Task
> Components: thirdparty
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27006) cordon off large ci worker nodes
[
https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539923#comment-17539923
]
Hudson commented on HBASE-27006:
Results for branch branch-2.4
[build #354 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> cordon off large ci worker nodes
>
>
> Key: HBASE-27006
> URL: https://issues.apache.org/jira/browse/HBASE-27006
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>
>
> Make sure our node labeling keeps test workloads that off of the new large
> spec nodes so that they're only used for the nightly k8s based tests
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-25465) Use javac --release option for supporting cross version compilation
[
https://issues.apache.org/jira/browse/HBASE-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539921#comment-17539921
]
Hudson commented on HBASE-25465:
Results for branch branch-2.4
[build #354 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/354/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use javac --release option for supporting cross version compilation
> ---
>
> Key: HBASE-25465
> URL: https://issues.apache.org/jira/browse/HBASE-25465
> Project: HBase
> Issue Type: Improvement
> Components: create-release
>Affects Versions: 3.0.0-alpha-3
>Reporter: Andrew Kyle Purtell
>Assignee: Duo Zhang
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
> See https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
> {quote}
> the Java compiler’s --release parameter, which was introduced via JEP 247
> ("Compile for Older Platform Versions"), added to the platform also in JDK 9.
> In contrast to the more widely known pair of --source and --target, the
> --release switch will ensure that only byte code is produced which actually
> will be usable with the specified Java version. For this purpose, the JDK
> contains the signature data for all supported Java versions (stored in the
> $JAVA_HOME/lib/ct.sym file).
> {quote}
> Using one JDK (i.e. Java 11) to build Java 8-and-up and Java 11-and-up
> compatible release artifacts would reduce some sources of accidental
> complexity, assuming the --release parameter actually works as advertised. To
> produce Java 8-and-up compatible artifacts, supply --release=8. To produce
> Java 11-and-up compatible release artifacts, supply --release=11. Maven
> activations based on JDK version and command line defined profiles can
> control what --release parameter, if any, should be passed to the compiler.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4453: HBASE-27053 IOException during caching of uncompressed block to the b…
Apache-HBase commented on PR #4453: URL: https://github.com/apache/hbase/pull/4453#issuecomment-1132430331 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 53s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-2 passed | | +1 :green_heart: | compile | 0m 30s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 15s | branch-2 passed | | +1 :green_heart: | spotless | 0m 46s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 57s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 4s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | -1 :x: | spotless | 0m 11s | patch has 22 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 23m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4453 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 7ed89deb9667 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 8a3a94205b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4453: HBASE-27053 IOException during caching of uncompressed block to the b…
Apache-HBase commented on PR #4453: URL: https://github.com/apache/hbase/pull/4453#issuecomment-1132428460 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | branch-2 passed | | +1 :green_heart: | compile | 0m 18s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 26s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | shadedjars | 3m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. | | | | 18m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4453 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 555d363302f7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 8a3a94205b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/testReport/ | | Max. process+thread count | 231 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4453: HBASE-27053 IOException during caching of uncompressed block to the b…
Apache-HBase commented on PR #4453: URL: https://github.com/apache/hbase/pull/4453#issuecomment-1132428371 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | branch-2 passed | | +1 :green_heart: | compile | 0m 14s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 15s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 13s | the patch passed | | +1 :green_heart: | javac | 0m 13s | the patch passed | | +1 :green_heart: | shadedjars | 3m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 22s | hbase-common in the patch passed. | | | | 17m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4453 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f4525da6e78b 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 8a3a94205b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/testReport/ | | Max. process+thread count | 246 (vs. ulimit of 12500) | | modules | C: hbase-common U: hbase-common | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4453/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-27053) IOException during caching of uncompressed block to the block cache.
[
https://issues.apache.org/jira/browse/HBASE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-27053 started by Sergey Soldatov.
---
> IOException during caching of uncompressed block to the block cache.
>
>
> Key: HBASE-27053
> URL: https://issues.apache.org/jira/browse/HBASE-27053
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Affects Versions: 2.4.12
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
>
> When prefetch to block cache is enabled and blocks are compressed sometimes
> caching fails with the exception:
> {noformat}
> 2022-05-18 21:37:29,597 ERROR [RS_OPEN_REGION-regionserver/x1:16020-2]
> regionserver.HRegion: Could not initialize all stores for the
> region=cluster_test,,1652935047946.a57ca5f9e7bebb4855a44523063f79c7.
> 2022-05-18 21:37:29,598 WARN [RS_OPEN_REGION-regionserver/x1:16020-2]
> regionserver.HRegion: Failed initialize of region=
> cluster_test,,1652935047946.a57ca5f9e7bebb4855a44523063f79c7.,
> starting to roll back memstore
> java.io.IOException: java.io.IOException: java.lang.RuntimeException: Cached
> block contents differ, which should not have
> happened.cacheKey:19307adf1c2248ebb5675116ea640712.c3a21f2005abf308e4a8c9759d4e05fe_0
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1149)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1092)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:996)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:946)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7240)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7199)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7175)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7090)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:147)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:100)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.RuntimeException: Cached block
> contents differ, which should not have
> happened.cacheKey:19307adf1c2248ebb5675116ea640712.c3a21f2005abf308e4a8c9759d4e05fe_0
> at
> org.apache.hadoop.hbase.regionserver.StoreEngine.openStoreFiles(StoreEngine.java:294)
> at
> org.apache.hadoop.hbase.regionserver.StoreEngine.initialize(StoreEngine.java:344)
> at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:294)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6375)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1115)
> at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1112)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: java.lang.RuntimeException: Cached block contents differ, which
> should not have
> happened.cacheKey:19307adf1c2248ebb5675116ea640712.c3a21f2005abf308e4a8c9759d4e05fe_0
> at
> org.apache.hadoop.hbase.io.hfile.BlockCacheUtil.validateBlockAddition(BlockCacheUtil.java:199)
> at
> org.apache.hadoop.hbase.io.hfile.BlockCacheUtil.shouldReplaceExistingCacheBlock(BlockCacheUtil.java:231)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.shouldReplaceExistingCacheBlock(BucketCache.java:447)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.cacheBlockWithWait(BucketCache.java:432)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.cacheBlock(BucketCache.java:418)
> at
> org.apache.hadoop.hbase.io.hfile.CombinedBlockCache.cacheBlock(CombinedBlockCache.java:60)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.lambda$readBlock$2(HFileReaderImpl.java:1319)
> at java.util.Optional.ifPresent(Optional.java:159)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1317)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.readAndUpdateNewBlock(HFileReaderImpl.java:942)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekTo(HFileReaderImpl.java:931)
> at
>
[GitHub] [hbase] ss77892 opened a new pull request, #4453: HBASE-27053 IOException during caching of uncompressed block to the b…
ss77892 opened a new pull request, #4453: URL: https://github.com/apache/hbase/pull/4453 …lock cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539878#comment-17539878
]
David Manning commented on HBASE-27054:
---
I see some good results by changing the cost function weights. I will propose a
PR with those changes.
{code:java}
conf.setFloat("hbase.master.balancer.stochastic.moveCost", 0f);
conf.setFloat("hbase.master.balancer.stochastic.tableSkewCost", 0f); {code}
If I make one change, with {{maxRunningTime}} from 180s to 30s, I see 100%
failure rate. If I make the above cost function weight updates, I see 100% pass
rate, even with a {{maxRunningTime}} of 15s.
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Assigned] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Manning reassigned HBASE-27054:
-
Assignee: David Manning
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: David Manning
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Comment Edited] (HBASE-27026) Disable Style/FrozenStringLiteralComment for ruby
[ https://issues.apache.org/jira/browse/HBASE-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539867#comment-17539867 ] Tao Li edited comment on HBASE-27026 at 5/20/22 1:27 AM: - Thanks [~janh] for your comments. Yes, I tried in this PR [https://github.com/apache/hbase/pull/4416], but there are still Rubocop warns. I'm also a little confused, so I started this discussion. !image-2022-05-20-09-14-36-355.png|width=411,height=337! This is the document: [https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3637.] !image-2022-05-20-09-25-05-608.png|width=493,height=322! Could it be because of the Rubocop version? Or if there's something wrong with the way I'm trying, please help point it out. Thank you very much. I'm not familiar with Rubocop. was (Author: tomscut): Thanks [~janh] for your comments. Yes, I tried in this PR [https://github.com/apache/hbase/pull/4416], but there are still Rubocop warns. I'm also a little confused, so I started this discussion. !image-2022-05-20-09-14-36-355.png|width=411,height=337! Could it be because of the Rubocop version? I'm not familiar with Rubocop. > Disable Style/FrozenStringLiteralComment for ruby > - > > Key: HBASE-27026 > URL: https://issues.apache.org/jira/browse/HBASE-27026 > Project: HBase > Issue Type: Task >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Attachments: image-2022-05-20-09-14-36-355.png, > image-2022-05-20-09-25-05-608.png > > > By default Style/FrozenStringLiteralComment is enabled in rubocop. If we > update a ruby file, rubocop prompts `Missing frozen string literal comment` > (see > [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. > To address this warn, we need to add `# frozen_string_literal: true` to the > top of the ruby file(see > https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3637), > which will be added to the top of the `Apache License` and will look strange. > I don't think this `FrozenStringLiteralComment` check is very necessary. We > can disable it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HBASE-27026) Disable Style/FrozenStringLiteralComment for ruby
[ https://issues.apache.org/jira/browse/HBASE-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Li updated HBASE-27026: --- Description: By default Style/FrozenStringLiteralComment is enabled in rubocop. If we update a ruby file, rubocop prompts `Missing frozen string literal comment` (see [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. To address this warn, we need to add `# frozen_string_literal: true` to the top of the ruby file(see https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3637), which will be added to the top of the `Apache License` and will look strange. I don't think this `FrozenStringLiteralComment` check is very necessary. We can disable it. was: By default Style/FrozenStringLiteralComment is enabled in rubocop. If we update a ruby file, rubocop prompts `Missing frozen string literal comment` (see [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. To address this warn, we need to add `# frozen_string_literal: true` to the top of the ruby file(see [https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3631]), which will be added to the top of the `Apache License` and will look strange. I don't think this `FrozenStringLiteralComment` check is very necessary. We can disable it. > Disable Style/FrozenStringLiteralComment for ruby > - > > Key: HBASE-27026 > URL: https://issues.apache.org/jira/browse/HBASE-27026 > Project: HBase > Issue Type: Task >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Attachments: image-2022-05-20-09-14-36-355.png > > > By default Style/FrozenStringLiteralComment is enabled in rubocop. If we > update a ruby file, rubocop prompts `Missing frozen string literal comment` > (see > [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. > To address this warn, we need to add `# frozen_string_literal: true` to the > top of the ruby file(see > https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3637), > which will be added to the top of the `Apache License` and will look strange. > I don't think this `FrozenStringLiteralComment` check is very necessary. We > can disable it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-27026) Disable Style/FrozenStringLiteralComment for ruby
[ https://issues.apache.org/jira/browse/HBASE-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539867#comment-17539867 ] Tao Li commented on HBASE-27026: Thanks [~janh] for your comments. Yes, I tried in this PR [https://github.com/apache/hbase/pull/4416], but there are still Rubocop warns. I'm also a little confused, so I started this discussion. !image-2022-05-20-09-14-36-355.png|width=411,height=337! Could it be because of the Rubocop version? I'm not familiar with Rubocop. > Disable Style/FrozenStringLiteralComment for ruby > - > > Key: HBASE-27026 > URL: https://issues.apache.org/jira/browse/HBASE-27026 > Project: HBase > Issue Type: Task >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Attachments: image-2022-05-20-09-14-36-355.png > > > By default Style/FrozenStringLiteralComment is enabled in rubocop. If we > update a ruby file, rubocop prompts `Missing frozen string literal comment` > (see > [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. > To address this warn, we need to add `# frozen_string_literal: true` to the > top of the ruby file(see > [https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3631]), > which will be added to the top of the `Apache License` and will look strange. > I don't think this `FrozenStringLiteralComment` check is very necessary. We > can disable it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HBASE-27026) Disable Style/FrozenStringLiteralComment for ruby
[ https://issues.apache.org/jira/browse/HBASE-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Li updated HBASE-27026: --- Attachment: image-2022-05-20-09-14-36-355.png > Disable Style/FrozenStringLiteralComment for ruby > - > > Key: HBASE-27026 > URL: https://issues.apache.org/jira/browse/HBASE-27026 > Project: HBase > Issue Type: Task >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Attachments: image-2022-05-20-09-14-36-355.png > > > By default Style/FrozenStringLiteralComment is enabled in rubocop. If we > update a ruby file, rubocop prompts `Missing frozen string literal comment` > (see > [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. > To address this warn, we need to add `# frozen_string_literal: true` to the > top of the ruby file(see > [https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3631]), > which will be added to the top of the `Apache License` and will look strange. > I don't think this `FrozenStringLiteralComment` check is very necessary. We > can disable it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (HBASE-26856) BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
[
https://issues.apache.org/jira/browse/HBASE-26856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539863#comment-17539863
]
Hudson commented on HBASE-26856:
Results for branch master
[build #590 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
> --
>
> Key: HBASE-26856
> URL: https://issues.apache.org/jira/browse/HBASE-26856
> Project: HBase
> Issue Type: Bug
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3, 2.4.13, 2.5.1
>
>
> In our production cluster we observed the cell value is modified after
> successful scanner read. After analyzing we observed OnheapDecodedCell is not
> created properly.
> We create OnheapDecodedCell with complete valAndTagsBuffer underlying array.
> {code:java}
> return new OnheapDecodedCell(Bytes.copy(keyBuffer, 0, this.keyLength),
> currentKey.getRowLength(), currentKey.getFamilyOffset(),
> currentKey.getFamilyLength(),
> currentKey.getQualifierOffset(), currentKey.getQualifierLength(),
> currentKey.getTimestamp(), currentKey.getTypeByte(),
> valAndTagsBuffer.array(),
> valAndTagsBuffer.arrayOffset() + vOffset, this.valueLength,
> memstoreTS, tagsArray,
> tOffset, this.tagsLength);
> {code}
> Here we are passing valAndTagsBuffer.array() for value extraction.
> The underlying array will be modified if it is altered anywhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27045) Disable TestClusterScopeQuotaThrottle
[
https://issues.apache.org/jira/browse/HBASE-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539864#comment-17539864
]
Hudson commented on HBASE-27045:
Results for branch master
[build #590 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Disable TestClusterScopeQuotaThrottle
> -
>
> Key: HBASE-27045
> URL: https://issues.apache.org/jira/browse/HBASE-27045
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
> We will inject a special EnvironmentEdge to control the return value of
> current time, which will mess up the wal filename and then fail the wal
> archiving.
> Let's disable it first before we find a suitable solution.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27006) cordon off large ci worker nodes
[
https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539865#comment-17539865
]
Hudson commented on HBASE-27006:
Results for branch master
[build #590 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/590/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> cordon off large ci worker nodes
>
>
> Key: HBASE-27006
> URL: https://issues.apache.org/jira/browse/HBASE-27006
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>
>
> Make sure our node labeling keeps test workloads that off of the new large
> spec nodes so that they're only used for the nightly k8s based tests
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4452: HBASE-27055 Fix HBASE_TRACE_OPTS when adding opentelemetry agent
Apache-HBase commented on PR #4452: URL: https://github.com/apache/hbase/pull/4452#issuecomment-1132321548 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | spotless | 1m 10s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 0m 36s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 3m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4452 | | Optional Tests | dupname asflicense spotless shellcheck shelldocs | | uname | Linux 391b2ca6a53e 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c7eb30d910 | | Max. process+thread count | 34 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4452: HBASE-27055 Fix HBASE_TRACE_OPTS when adding opentelemetry agent
Apache-HBase commented on PR #4452: URL: https://github.com/apache/hbase/pull/4452#issuecomment-1132320851 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4452 | | Optional Tests | | | uname | Linux 9a573bd93aab 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c7eb30d910 | | Max. process+thread count | 33 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4452: HBASE-27055 Fix HBASE_TRACE_OPTS when adding opentelemetry agent
Apache-HBase commented on PR #4452: URL: https://github.com/apache/hbase/pull/4452#issuecomment-1132320740 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 52s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4452 | | Optional Tests | | | uname | Linux 01a94c808939 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / c7eb30d910 | | Max. process+thread count | 40 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4452/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] taklwu opened a new pull request, #4452: HBASE-27055 Fix HBASE_TRACE_OPTS when adding opentelemetry agent
taklwu opened a new pull request, #4452: URL: https://github.com/apache/hbase/pull/4452 tested locally and now standalone with these configuration `export HBASE_TRACE_OPTS="-Dotel.traces.exporter=jaegar -Dotel.metrics.exporter=none"` in `conf/hbase-env.sh` no longer failed before the change, it's always using the default and create Otlp metrics exporter but it cannot be done. ``` [opentelemetry.auto.trace 2022-05-19 17:09:13:587 -0700] [grpc-default-executor-3] WARN io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter - Failed to export metrics io.grpc.StatusRuntimeException: UNAVAILABLE: io exception at io.grpc.Status.asRuntimeException(Status.java:534) at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:617) at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:803) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:782) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/0:0:0:0:0:0:0:1:4317 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27053) IOException during caching of uncompressed block to the block cache.
[
https://issues.apache.org/jira/browse/HBASE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539859#comment-17539859
]
Sergey Soldatov commented on HBASE-27053:
-
So, why and when did it happen. The usual scenario is when a region split has
just happened and RS is trying to open both daughters and load those to the
bucket cache. There is a single store that has the split point, so two threads
are reading the same block and trying to store it in the cache. When we
decompress the block, the first thing we are doing is the space allocation:
[https://github.com/apache/hbase/blob/c7eb30d91015de67fb8207ac1818ce2a29dd60a4/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L649]
Here we allocate the space for uncompressed data *AND* checksum. But we fill
the data only, without filling the checksum space which actually might be
filled by garbage from the previous usage. So the block is cached with this
garbage and obviously, that part might and would not match when we try to store
the same block from another thread.
Unfortunately, I was unable to create a reasonable unit test for this scenario,
but there is the manual steps to reproduce:
a single node cluster with following configuration tweaks:
hbase.hregion.memstore.flush.size=100
hbase.hregion.max.filesize=1000
hbase.bucketcache.ioengine=file:/tmp/hbase_cache
hbase.bucketcache.size=20
hbase.hfile.thread.prefetch=4
And the load is generated by:
{noformat}
hbase org.apache.hadoop.hbase.util.LoadTestTool -compression SNAPPY -write
1:10:100 -num_keys 1 {noformat}
Usually, after 700k-1.2m records, the earlier mentioned exception appears in
the RS log.
So, to solve the problem I would suggest adding a code that cleans up the
checksum space when the decompression is completed. It doesn't look like an
optimal solution, but right after decompression, we don't know whether the
checksum space will be used or not, so we could not just trim the bytebuff.
> IOException during caching of uncompressed block to the block cache.
>
>
> Key: HBASE-27053
> URL: https://issues.apache.org/jira/browse/HBASE-27053
> Project: HBase
> Issue Type: Bug
> Components: BlockCache
>Affects Versions: 2.4.12
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
>
> When prefetch to block cache is enabled and blocks are compressed sometimes
> caching fails with the exception:
> {noformat}
> 2022-05-18 21:37:29,597 ERROR [RS_OPEN_REGION-regionserver/x1:16020-2]
> regionserver.HRegion: Could not initialize all stores for the
> region=cluster_test,,1652935047946.a57ca5f9e7bebb4855a44523063f79c7.
> 2022-05-18 21:37:29,598 WARN [RS_OPEN_REGION-regionserver/x1:16020-2]
> regionserver.HRegion: Failed initialize of region=
> cluster_test,,1652935047946.a57ca5f9e7bebb4855a44523063f79c7.,
> starting to roll back memstore
> java.io.IOException: java.io.IOException: java.lang.RuntimeException: Cached
> block contents differ, which should not have
> happened.cacheKey:19307adf1c2248ebb5675116ea640712.c3a21f2005abf308e4a8c9759d4e05fe_0
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1149)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1092)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:996)
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:946)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7240)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7199)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7175)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7134)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7090)
> at
> org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:147)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:100)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.RuntimeException: Cached block
> contents differ, which should not have
> happened.cacheKey:19307adf1c2248ebb5675116ea640712.c3a21f2005abf308e4a8c9759d4e05fe_0
> at
> org.apache.hadoop.hbase.regionserver.StoreEngine.openStoreFiles(StoreEngine.java:294)
> at
> org.apache.hadoop.hbase.regionserver.StoreEngine.initialize(StoreEngine.java:344)
> at
[jira] [Assigned] (HBASE-27055) Fix HBASE_TRACE_OPTS when adding opentelemetry agent
[
https://issues.apache.org/jira/browse/HBASE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak-Lon (Stephen) Wu reassigned HBASE-27055:
Assignee: Tak-Lon (Stephen) Wu
> Fix HBASE_TRACE_OPTS when adding opentelemetry agent
>
>
> Key: HBASE-27055
> URL: https://issues.apache.org/jira/browse/HBASE-27055
> Project: HBase
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
>
> it was a minor bug caused by HBASE-26363 with a typo in the following line
> https://github.com/apache/hbase/blob/master/bin/hbase#L511
> {code}
> HBASE_OPTS="$HBASE_OPTS -javaagent:$agent_jar"
> {code}
> and it should be
> {code}
> HBASE_OPTS="$HBASE_TRACE_OPTS -javaagent:$agent_jar"
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (HBASE-27055) Fix HBASE_TRACE_OPTS when adding opentelemetry agent
Tak-Lon (Stephen) Wu created HBASE-27055:
Summary: Fix HBASE_TRACE_OPTS when adding opentelemetry agent
Key: HBASE-27055
URL: https://issues.apache.org/jira/browse/HBASE-27055
Project: HBase
Issue Type: Bug
Components: tracing
Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
Reporter: Tak-Lon (Stephen) Wu
it was a minor bug caused by HBASE-26363 with a typo in the following line
https://github.com/apache/hbase/blob/master/bin/hbase#L511
{code}
HBASE_OPTS="$HBASE_OPTS -javaagent:$agent_jar"
{code}
and it should be
{code}
HBASE_OPTS="$HBASE_TRACE_OPTS -javaagent:$agent_jar"
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1132307509 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 0m 47s | master passed | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 30s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed | | +1 :green_heart: | javac | 0m 48s | the patch passed | | +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 211m 1s | hbase-server in the patch passed. | | | | 231m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fa54342aa463 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/testReport/ | | Max. process+thread count | 2449 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1132297950 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 30s | master passed | | +1 :green_heart: | compile | 0m 46s | master passed | | +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 192m 2s | hbase-server in the patch failed. | | | | 208m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6c67d3505ace 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/testReport/ | | Max. process+thread count | 2706 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
Apache-HBase commented on PR #4442: URL: https://github.com/apache/hbase/pull/4442#issuecomment-1132294212 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 22s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 24s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 3m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 7s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 215m 6s | hbase-server in the patch passed. | | | | 234m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4442 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 36ee3218025d 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/testReport/ | | Max. process+thread count | 2581 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
Apache-HBase commented on PR #4442: URL: https://github.com/apache/hbase/pull/4442#issuecomment-1132288809 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 13s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 12s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 10s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 205m 27s | hbase-server in the patch passed. | | | | 224m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4442 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 14301b19e1b9 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/testReport/ | | Max. process+thread count | 2356 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539839#comment-17539839
]
David Manning edited comment on HBASE-27054 at 5/19/22 11:06 PM:
-
I ran it 50 times locally using latest {{{}master{}}}, it failed twice, even
with 3-minute timeout, and ~3.9 million stochastic steps. So the 77s appears
irrelevant.
was (Author: dmanning):
I ran it 50 times locally using latest {{master}}, it failed twice, even with
3-minute timeout, and ~3.9 million stochastic steps.
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (HBASE-26981) The CPU usage of the regionserver node where the meta table is located is too high
[ https://issues.apache.org/jira/browse/HBASE-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-26981: - Fix Version/s: 2.5.0 3.0.0-alpha-3 2.4.13 > The CPU usage of the regionserver node where the meta table is located is too > high > -- > > Key: HBASE-26981 > URL: https://issues.apache.org/jira/browse/HBASE-26981 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Affects Versions: 2.3.4 >Reporter: zhengsicheng >Assignee: zhengsicheng >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13 > > Attachments: image-2022-04-27-20-24-33-252.png, > image-2022-04-27-20-45-09-227.png, image-2022-04-28-09-59-40-567.png, jstack > > > When the read and write pressure is high, the CPU usage of the meta table > node is too high > !image-2022-04-28-09-59-40-567.png! > > !image-2022-04-27-20-24-33-252.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (HBASE-27018) Add a tool command list_liveservers
[ https://issues.apache.org/jira/browse/HBASE-27018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-27018. -- Fix Version/s: 2.5.0 3.0.0-alpha-3 2.4.13 Hadoop Flags: Reviewed Resolution: Fixed > Add a tool command list_liveservers > --- > > Key: HBASE-27018 > URL: https://issues.apache.org/jira/browse/HBASE-27018 > Project: HBase > Issue Type: New Feature >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13 > > Attachments: image-2022-05-10-08-34-33-711.png > > > To make it easier for us to query the living region Servers. We can add a > command `list_liveservers`. There are already `list_deadServers` and > `list_Decommissioned_regionServers`. > !image-2022-05-10-08-34-33-711.png|width=457,height=123! > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] virajjasani commented on pull request #4416: HBASE-27018 Add a tool command list_liveservers
virajjasani commented on PR #4416: URL: https://github.com/apache/hbase/pull/4416#issuecomment-1132275153 Thanks for the contribution @tomscut and for the review @pankaj72981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani merged pull request #4416: HBASE-27018 Add a tool command list_liveservers
virajjasani merged PR #4416: URL: https://github.com/apache/hbase/pull/4416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539839#comment-17539839
]
David Manning commented on HBASE-27054:
---
I ran it 50 times locally using latest {{master}}, it failed twice, even with
3-minute timeout, and ~3.9 million stochastic steps.
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1132180963 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 31s | master passed | | +1 :green_heart: | compile | 2m 15s | master passed | | +1 :green_heart: | checkstyle | 0m 32s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 10s | the patch passed | | +1 :green_heart: | compile | 2m 9s | the patch passed | | +1 :green_heart: | javac | 2m 9s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 49s | patch has 22 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 33m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 91c1555a6e14 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
Apache-HBase commented on PR #4442: URL: https://github.com/apache/hbase/pull/4442#issuecomment-1132152201 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 49s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 2m 47s | master passed | | +1 :green_heart: | checkstyle | 0m 42s | master passed | | +1 :green_heart: | spotless | 0m 48s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 2m 48s | the patch passed | | +1 :green_heart: | javac | 2m 48s | the patch passed | | +1 :green_heart: | checkstyle | 0m 38s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 45s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 32s | patch has 22 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 2m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 36m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4442 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux c69e314cc9bc 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4442/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a diff in pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
sunhelly commented on code in PR #4403: URL: https://github.com/apache/hbase/pull/4403#discussion_r877490131 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java: ## @@ -384,6 +387,9 @@ public class HMaster extends HBaseServerBase implements Maste private DirScanPool logCleanerPool; private LogCleaner logCleaner; private HFileCleaner hfileCleaner; + private HFileCleaner[] customHFileCleaners; + private Path[] customHFilePaths; + private DirScanPool customHFileCleanerPool; Review Comment: Thanks. All the custom paths cleaners use a shared DirScanPool, while the origin archive cleaner use an exclusive DirScanPool. I think currently it is not necessary for each custom path use its unique DirScanPool, since all the customs paths are subdirectories under the hfile archive path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a diff in pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
sunhelly commented on code in PR #4403: URL: https://github.com/apache/hbase/pull/4403#discussion_r876970377 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java: ## @@ -384,6 +387,9 @@ public class HMaster extends HBaseServerBase implements Maste private DirScanPool logCleanerPool; private LogCleaner logCleaner; private HFileCleaner hfileCleaner; + private HFileCleaner[] customHFileCleaners; Review Comment: Collected the hfileCleaner and customHFileCleaners to a linked list, and let the default hfileCleaner be the first element. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26856) BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
[
https://issues.apache.org/jira/browse/HBASE-26856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539766#comment-17539766
]
Hudson commented on HBASE-26856:
Results for branch branch-2.5
[build #121 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted
> --
>
> Key: HBASE-26856
> URL: https://issues.apache.org/jira/browse/HBASE-26856
> Project: HBase
> Issue Type: Bug
>Reporter: Mohammad Arshad
>Assignee: Mohammad Arshad
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3, 2.4.13, 2.5.1
>
>
> In our production cluster we observed the cell value is modified after
> successful scanner read. After analyzing we observed OnheapDecodedCell is not
> created properly.
> We create OnheapDecodedCell with complete valAndTagsBuffer underlying array.
> {code:java}
> return new OnheapDecodedCell(Bytes.copy(keyBuffer, 0, this.keyLength),
> currentKey.getRowLength(), currentKey.getFamilyOffset(),
> currentKey.getFamilyLength(),
> currentKey.getQualifierOffset(), currentKey.getQualifierLength(),
> currentKey.getTimestamp(), currentKey.getTypeByte(),
> valAndTagsBuffer.array(),
> valAndTagsBuffer.arrayOffset() + vOffset, this.valueLength,
> memstoreTS, tagsArray,
> tOffset, this.tagsLength);
> {code}
> Here we are passing valAndTagsBuffer.array() for value extraction.
> The underlying array will be modified if it is altered anywhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27024) The User API and Developer API links are broken on hbase.apache.org
[
https://issues.apache.org/jira/browse/HBASE-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539769#comment-17539769
]
Hudson commented on HBASE-27024:
Results for branch branch-2.5
[build #121 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The User API and Developer API links are broken on hbase.apache.org
> ---
>
> Key: HBASE-27024
> URL: https://issues.apache.org/jira/browse/HBASE-27024
> Project: HBase
> Issue Type: Sub-task
> Components: website
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27045) Disable TestClusterScopeQuotaThrottle
[
https://issues.apache.org/jira/browse/HBASE-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539768#comment-17539768
]
Hudson commented on HBASE-27045:
Results for branch branch-2.5
[build #121 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Disable TestClusterScopeQuotaThrottle
> -
>
> Key: HBASE-27045
> URL: https://issues.apache.org/jira/browse/HBASE-27045
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.13
>
>
> We will inject a special EnvironmentEdge to control the return value of
> current time, which will mess up the wal filename and then fail the wal
> archiving.
> Let's disable it first before we find a suitable solution.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26986) Trace a one-shot execution of a Master procedure
[
https://issues.apache.org/jira/browse/HBASE-26986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539767#comment-17539767
]
Hudson commented on HBASE-26986:
Results for branch branch-2.5
[build #121 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/121/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Trace a one-shot execution of a Master procedure
>
>
> Key: HBASE-26986
> URL: https://issues.apache.org/jira/browse/HBASE-26986
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> A self-contained effort towards wrangling the stray traces identified in
> HBASE-26366. This one adds a tracing context to the Master's ProcedureV2
> execution. This ties all the spans that result from a procedure execution
> back to a procedure by name.
> Maybe in a follow-on issue, we can serialize span context along with
> procedures and display procedure start/stop/retry events and parent-child
> hierarchy in tracing visualization.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27026) Disable Style/FrozenStringLiteralComment for ruby
[ https://issues.apache.org/jira/browse/HBASE-27026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539763#comment-17539763 ] Jan Hentschel commented on HBASE-27026: --- Did you try it out? Based on the documentation I would be surprised if adding it after the license wouldn't work as examples in the Rubocop documentation leave the impression that you can put it anywhere in a file, marking kind of the starting point, multiple times. I would prefer handling the warning instead of ignoring it. I would recommend to put # frozen_string_literal: true after the Apache license header in your existing PR and just give it a try. > Disable Style/FrozenStringLiteralComment for ruby > - > > Key: HBASE-27026 > URL: https://issues.apache.org/jira/browse/HBASE-27026 > Project: HBase > Issue Type: Task >Reporter: Tao Li >Assignee: Tao Li >Priority: Major > > By default Style/FrozenStringLiteralComment is enabled in rubocop. If we > update a ruby file, rubocop prompts `Missing frozen string literal comment` > (see > [https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4416/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt]. > To address this warn, we need to add `# frozen_string_literal: true` to the > top of the ruby file(see > [https://github.com/rubocop/rubocop/blob/master/config/default.yml#L3631]), > which will be added to the top of the `Apache License` and will look strange. > I don't think this `FrozenStringLiteralComment` check is very necessary. We > can disable it. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] huaxiangsun commented on a diff in pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
huaxiangsun commented on code in PR #4442:
URL: https://github.com/apache/hbase/pull/4442#discussion_r877453331
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ClientMetaTableAccessor.java:
##
@@ -257,7 +259,37 @@ private static CompletableFuture
scanMeta(AsyncTable future = new CompletableFuture();
-metaTable.scan(scan, new MetaTableScanResultConsumer(rowUpperLimit,
visitor, future));
+// Get the region locator's meta replica mode.
+CatalogReplicaMode metaReplicaMode =
CatalogReplicaMode.fromString(metaTable.getConfiguration()
+ .get(LOCATOR_META_REPLICAS_MODE, CatalogReplicaMode.NONE.toString()));
+
+if (metaReplicaMode == CatalogReplicaMode.LOAD_BALANCE) {
+ addListener(metaTable.getDescriptor(), (desc, error) -> {
+if (error != null) {
+ LOG.warn("Failed to get meta table descriptotr, error={}", error);
Review Comment:
We need the stacktrace, update in the new patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539754#comment-17539754
]
David Manning commented on HBASE-27054:
---
With a lower timeout, like 60 seconds, or on slower hardware, we could get
fewer iterations. I suppose in that sense we may just get unlucky in not being
able to get to fully balanced state given current configuration.
50,000 regions have to move, and the {{RegionReplicaCandidateGenerator}} is
doing most of that work, which is chosen roughly 25% of the time. There are
likely some missteps. Conservatively, it seems like we may need 200,000 calls
to guarantee the work gets done. That means 800,000 iterations. Running
locally, if I had set a timeout of 60 seconds, I'd see 1.3 million iterations.
It's close enough that we may see the occasional problem. The tests should
ensure that even on slow hardware, with unlucky random choices, we are still
virtually guaranteed success. We may not be doing that here. But a 3 minute
timeout should make it much more likely. So I'm interested in the test message
that says it ran 77 seconds, even though I'm sure the test could be improved to
be more deterministic.
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
> ---
>
> Key: HBASE-27054
> URL: https://issues.apache.org/jira/browse/HBASE-27054
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> . Looks like we can be off by one on either side of an expected value.
> Any idea what is going on here [~dmanning]?
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.779 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no less than 60.
> server=srv1351292323,46522,-3543799643652531264 , load=59
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:200)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
> {noformat}
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> Time elapsed: 77.781 s <<< FAILURE!
> java.lang.AssertionError: All servers should have load no more than 60.
> server=srv1402325691,7995,26308078476749652 , load=61
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.assertTrue(Assert.java:42)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.assertClusterAsBalanced(BalancerTestBase.java:198)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:577)
> at
> org.apache.hadoop.hbase.master.balancer.BalancerTestBase.testWithCluster(BalancerTestBase.java:544)
> at
> org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster(TestStochasticLoadBalancerRegionReplicaLargeCluster.java:41)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-27054) TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539749#comment-17539749
]
David Manning commented on HBASE-27054:
---
[~apurtell] Do we know if this is a recent regression, or has it always been
flaky? My initial thought is that there may be some randomness (it is a
stochastic balancer after all) which leads to this end result. I don't believe
any recent changes would have caused this to become more flaky, but I suppose
it's possible. HBASE-26311 is interesting, since it changes calculations to use
standard deviation. [~claraxiong]
Why does the error message say it failed after 77 seconds? The test takes 3
minutes to run for me locally, which is the configured timeout for the balancer
in {{StochasticBalancerTestBase2}}. Is there a link to a test failure with full
logs that I can inspect? (Note, 3 minute timeout was updated in HBASE-25873.
Previous value was 90 seconds.)
With region replicas involved, the {{RegionReplicaCandidateGenerator}} will
just move a colocated replica to a random server, without consideration of how
many regions that target server is hosting. The cost functions will allow it in
basically every case, since it heavily prioritizes resolving colocated
replicas. So maybe by the time all the region replicas have been resolved, the
number of moves is already pushing limits of one balancer iteration, with
having randomly overloaded one regionserver.
A situation that the balancer will have a difficult time getting out of is if
one regionserver is hosting 61 replicas of 61 regions, and another regionserver
is hosting 59 regions, which are replicas of those 61 regions. The
{{LoadCandidateGenerator}} will keep trying to take a region from the server
with 61 and give it to the server with 59, but because there is already a
replica that matches, it will be too expensive to move. But as long as we can
process enough iterations, probabilistically speaking we should be able to get
to one of the 2 safe regions to move... when I run this test locally I see
nearly 4 million iterations, and with 1/4 of those using the
{{LoadCandidateGenerator}} it seems like we should generally find a solution
that moves them all.
{code}
Finished computing new moving plan. Computation took 180001 ms to try 3975554
different iterations. Found a solution that moves 50006 regions; Going from a
computed imbalance of 0.9026309610781538 to a new imbalance of
5.252006025578701E-5. funtionCost=RegionCountSkewCostFunction :
(multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction :
(multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0,
imbalance=0.83343334, need balance); RackLocalityCostFunction :
(multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0,
imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=10.0,
imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=1.0,
imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0);
CPRequestCostFunction : (multiplier=5.0, imbalance=0.0);
WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0);
MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0);
StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);
{code}
Since the test case is also using 100 tables, and there is a
{{TableSkewCostFunction}} involved, it's also possible that the balancer is
happy with a slightly uneven region count balance, because balancing the last
region would push towards an imbalance of tables if the target regionserver
already has too many regions of that table for every region that is chosen. I
don't know if the math would support this, though. If it does, it's possible
that out of the last 61 regions, moving any region to the server with 59 would
either cause table skew or colocated replicas, and so the balancer cannot fully
balance based on the simple {{LoadCandidateGenerator}} alone.
This is all hypothetical, without yet trying to debug. Given the large size of
the test, the number of balancer iterations, and the flakiness, it may be
difficult to debug. I ran it 10+ times locally so far, and it passes each time.
So, some ideas to explore:
# Don't assert that the cluster is fully balanced in this test case, just
assert that there are no colocated replicas. Arguably this is the purpose of
the test, and the test framework already appears to allow for this.
# Change cost function weights for everything else, other than region counts
and replica counts, to be 0. In this way, nothing prevents the balancer
optimizing for these variables, which the test is expecting to validate.
Specifically, set TableSkew and MoveCost functions to 0.
# Use fewer than 100 tables, if table skew is a contributing factor.
> TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster
> is flaky
>
[GitHub] [hbase] huaxiangsun commented on a diff in pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
huaxiangsun commented on code in PR #4442:
URL: https://github.com/apache/hbase/pull/4442#discussion_r877404672
##
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplication.java:
##
@@ -477,7 +495,7 @@ public void testHBaseMetaReplicaGets() throws Exception {
// There are read requests increase for primary meta replica.
// For rest of meta replicas, there is no change as regionMove will tell
the new location
- primaryIncreaseReplicaNoChange(readReqsForMetaReplicasAfterGet,
+
primaryIncreaseReplicaNoChange(readReqsForMetaReplicasAfterGetAllLocations,
Review Comment:
All locations are cached at the client side, when a region move, for a short
period of time, the previous RS maintains a moved region list. Get requests to
previous RS result in RegionMovedException, which has the new RS info. There is
no new requests to meta replica regions in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a diff in pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
huaxiangsun commented on code in PR #4442: URL: https://github.com/apache/hbase/pull/4442#discussion_r877396560 ## hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncNonMetaRegionLocator.java: ## @@ -50,6 +50,7 @@ import java.util.concurrent.TimeUnit; import org.apache.commons.lang3.ObjectUtils; import org.apache.hadoop.hbase.CatalogFamilyFormat; +import org.apache.hadoop.hbase.CatalogReplicaMode; Review Comment: Yes, since it is moved to another package, it needs to be imported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a diff in pull request #4442: HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#g…
huaxiangsun commented on code in PR #4442:
URL: https://github.com/apache/hbase/pull/4442#discussion_r877395823
##
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplication.java:
##
@@ -385,6 +386,12 @@ private void primaryNoChangeReplicaIncrease(final long[]
before, final long[] af
}
}
+ private void PrimaryIncreaseReplicaIncrease(final long[] before, final
long[] after) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
Apache-HBase commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1132007997 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 58s | branch-1 passed | | +1 :green_heart: | compile | 0m 23s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 28s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 6s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 24s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 2m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 0s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 17s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 53s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 1m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 3m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 1m 49s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 238m 49s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 271m 29s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort | | | hadoop.hbase.coprocessor.TestMasterObserver | | | hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController | | | hadoop.hbase.regionserver.TestPerColumnFamilyFlush | | | hadoop.hbase.client.TestClientScannerRPCTimeout | | | hadoop.hbase.client.TestSnapshotCloneIndependence | | | hadoop.hbase.replication.TestReplicationEndpoint | | | hadoop.hbase.regionserver.TestRegionServerHostname | | | hadoop.hbase.snapshot.TestExportSnapshot | | | hadoop.hbase.regionserver.wal.TestLogRollPeriod | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4377 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux cecf0c8ca243 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4377/out/precommit/personality/provided.sh | | git revision | branch-1 / 1599a48687 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/12/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/12/testReport/ | | Max. process+thread count | 4028 (vs. ulimit of 1) | | modules |
[GitHub] [hbase] Apache-HBase commented on pull request #4451: HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky
Apache-HBase commented on PR #4451: URL: https://github.com/apache/hbase/pull/4451#issuecomment-1131997192 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 44s | master passed | | +1 :green_heart: | compile | 0m 47s | master passed | | +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 37s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 196m 54s | hbase-server in the patch passed. | | | | 214m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4451 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux df2afea24ded 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/testReport/ | | Max. process+thread count | 2457 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4451: HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky
Apache-HBase commented on PR #4451: URL: https://github.com/apache/hbase/pull/4451#issuecomment-1131994799 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | compile | 0m 33s | master passed | | +1 :green_heart: | shadedjars | 3m 57s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 0m 34s | the patch passed | | +1 :green_heart: | javac | 0m 34s | the patch passed | | +1 :green_heart: | shadedjars | 3m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 193m 47s | hbase-server in the patch passed. | | | | 212m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4451 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0fea240e1b91 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/testReport/ | | Max. process+thread count | 2684 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-27006) cordon off large ci worker nodes
[ https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-27006. - Fix Version/s: 2.5.0 2.6.0 3.0.0-alpha-3 2.4.13 Assignee: Sean Busbey Resolution: Fixed > cordon off large ci worker nodes > > > Key: HBASE-27006 > URL: https://issues.apache.org/jira/browse/HBASE-27006 > Project: HBase > Issue Type: Sub-task >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13 > > > Make sure our node labeling keeps test workloads that off of the new large > spec nodes so that they're only used for the nightly k8s based tests -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1131881628 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 0m 35s | master passed | | +1 :green_heart: | shadedjars | 3m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 0m 34s | the patch passed | | +1 :green_heart: | javac | 0m 34s | the patch passed | | +1 :green_heart: | shadedjars | 3m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 197m 22s | hbase-server in the patch failed. | | | | 216m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 043bb0df2cf3 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/testReport/ | | Max. process+thread count | 2807 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1131876853 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 0m 47s | master passed | | +1 :green_heart: | shadedjars | 3m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 3m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 193m 33s | hbase-server in the patch failed. | | | | 211m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1c1fa43a1235 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/testReport/ | | Max. process+thread count | 2730 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4451: HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky
Apache-HBase commented on PR #4451: URL: https://github.com/apache/hbase/pull/4451#issuecomment-1131785538 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 44s | master passed | | +1 :green_heart: | compile | 2m 14s | master passed | | +1 :green_heart: | checkstyle | 0m 32s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 9s | the patch passed | | +1 :green_heart: | compile | 2m 9s | the patch passed | | +1 :green_heart: | javac | 2m 9s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 33m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4451 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 905eb1087bec 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4451/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26981) The CPU usage of the regionserver node where the meta table is located is too high
[ https://issues.apache.org/jira/browse/HBASE-26981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539566#comment-17539566 ] Bryan Beaudreault commented on HBASE-26981: --- Hi [~zhengsicheng] , thanks for your work here. Just curious if you plan to work on a solution soon? We are also seeing this issue > The CPU usage of the regionserver node where the meta table is located is too > high > -- > > Key: HBASE-26981 > URL: https://issues.apache.org/jira/browse/HBASE-26981 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Affects Versions: 2.3.4 >Reporter: zhengsicheng >Assignee: zhengsicheng >Priority: Major > Attachments: image-2022-04-27-20-24-33-252.png, > image-2022-04-27-20-45-09-227.png, image-2022-04-28-09-59-40-567.png, jstack > > > When the read and write pressure is high, the CPU usage of the meta table > node is too high > !image-2022-04-28-09-59-40-567.png! > > !image-2022-04-27-20-24-33-252.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (HBASE-27052) TestAsyncTableScanner.testScanWrongColumnFamily is flaky
[
https://issues.apache.org/jira/browse/HBASE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-27052:
-
Status: Patch Available (was: Open)
PR for master is posted.
> TestAsyncTableScanner.testScanWrongColumnFamily is flaky
>
>
> Key: HBASE-27052
> URL: https://issues.apache.org/jira/browse/HBASE-27052
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> [~ndimiduk] have you seen something like this before?
> {noformat}
> [ERROR]
> org.apache.hadoop.hbase.client.TestAsyncTableScanner.testScanWrongColumnFamily[3:
> table=raw, scan=batchSmallResultSize] Time elapsed: 0.018 s <<< FAILURE!
> java.lang.AssertionError:
> Expected:
> a collection containing (SpanKind with a name that a string starting with
> "SCAN default:async" and
> SpanKind with a parentSpanId that "612b6a689e063c9b" and
> SpanData with StatusCode that is and
> SpanData having Exception with Attributes that Attributes containing [is
> ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] and
> SpanData that hasEnded)
> but:
> SpanKind with a name that a string starting with "SCAN default:async" name was
> "AsyncRegionLocator.getRegionLocation", SpanKind with a name that a string
> starting with "SCAN default:async" name was
> "Region.getScanner", SpanKind with a name that a string starting with "SCAN
> default:async" name was
> "hbase.pb.ClientService/Scan", SpanKind with a name that a string starting
> with "SCAN default:async" name was
> "RpcServer.process", SpanData having Exception with Attributes that
> Attributes containing [is ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] exception
> attributes
> Attributes was
> [ Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3544)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> >,
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
>
[GitHub] [hbase] ndimiduk opened a new pull request, #4451: HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky
ndimiduk opened a new pull request, #4451: URL: https://github.com/apache/hbase/pull/4451 I've looped this test 100+ times and it doesn't reproduce locally for me. This is a patch that I think will make the test resilient to what happened in this stack trace. There are likely other places where this kind up exception unwrapping needs to happen, so there could be others... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27052) TestAsyncTableScanner.testScanWrongColumnFamily is flaky
[
https://issues.apache.org/jira/browse/HBASE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539555#comment-17539555
]
Nick Dimiduk commented on HBASE-27052:
--
I've looped this test 100+ times and it doesn't reproduce locally for me. Let
me post up a patch that I think will make the test resilient to what happened
in this stack trace. There are likely other places where this kind up exception
unwrapping needs to happen, so there could be others...
> TestAsyncTableScanner.testScanWrongColumnFamily is flaky
>
>
> Key: HBASE-27052
> URL: https://issues.apache.org/jira/browse/HBASE-27052
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> [~ndimiduk] have you seen something like this before?
> {noformat}
> [ERROR]
> org.apache.hadoop.hbase.client.TestAsyncTableScanner.testScanWrongColumnFamily[3:
> table=raw, scan=batchSmallResultSize] Time elapsed: 0.018 s <<< FAILURE!
> java.lang.AssertionError:
> Expected:
> a collection containing (SpanKind with a name that a string starting with
> "SCAN default:async" and
> SpanKind with a parentSpanId that "612b6a689e063c9b" and
> SpanData with StatusCode that is and
> SpanData having Exception with Attributes that Attributes containing [is
> ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] and
> SpanData that hasEnded)
> but:
> SpanKind with a name that a string starting with "SCAN default:async" name was
> "AsyncRegionLocator.getRegionLocation", SpanKind with a name that a string
> starting with "SCAN default:async" name was
> "Region.getScanner", SpanKind with a name that a string starting with "SCAN
> default:async" name was
> "hbase.pb.ClientService/Scan", SpanKind with a name that a string starting
> with "SCAN default:async" name was
> "RpcServer.process", SpanData having Exception with Attributes that
> Attributes containing [is ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] exception
> attributes
> Attributes was
> [ Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3544)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> >,
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
>
[GitHub] [hbase] Apache-HBase commented on pull request #4449: HBASE-27046 The filenum in AbstractFSWAL should be monotone increasing
Apache-HBase commented on PR #4449: URL: https://github.com/apache/hbase/pull/4449#issuecomment-1131632996 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 21s | master passed | | +1 :green_heart: | compile | 0m 39s | master passed | | +1 :green_heart: | shadedjars | 3m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 242m 40s | hbase-server in the patch failed. | | | | 259m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4449 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 41330ea4a14f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/testReport/ | | Max. process+thread count | 2529 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
Apache-HBase commented on PR #4403: URL: https://github.com/apache/hbase/pull/4403#issuecomment-1131628243 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 34s | master passed | | +1 :green_heart: | compile | 2m 17s | master passed | | +1 :green_heart: | checkstyle | 0m 31s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 18s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 2m 13s | the patch passed | | +1 :green_heart: | javac | 2m 13s | the patch passed | | +1 :green_heart: | checkstyle | 0m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 1s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 32m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4403 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 6992139158bd 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4403/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a diff in pull request #4403: HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner
sunhelly commented on code in PR #4403: URL: https://github.com/apache/hbase/pull/4403#discussion_r876970377 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java: ## @@ -384,6 +387,9 @@ public class HMaster extends HBaseServerBase implements Maste private DirScanPool logCleanerPool; private LogCleaner logCleaner; private HFileCleaner hfileCleaner; + private HFileCleaner[] customHFileCleaners; Review Comment: Collected the hfileCleaner and customHFileCleaners in a linked list, the head is the default hfileCleaner. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27052) TestAsyncTableScanner.testScanWrongColumnFamily is flaky
[
https://issues.apache.org/jira/browse/HBASE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539480#comment-17539480
]
Nick Dimiduk commented on HBASE-27052:
--
I've not seen it, but it's entirely reasonable. It appears that the expected
{{NoSuchColumnFamilyException}} is wrapped in a {{RemoteWithExtrasException}}
at the time it is annotated onto the span. Our client goes out of its way to
unwrap this kind of thing, let me take a pass over span error handling and make
sure it does the same.
> TestAsyncTableScanner.testScanWrongColumnFamily is flaky
>
>
> Key: HBASE-27052
> URL: https://issues.apache.org/jira/browse/HBASE-27052
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> [~ndimiduk] have you seen something like this before?
> {noformat}
> [ERROR]
> org.apache.hadoop.hbase.client.TestAsyncTableScanner.testScanWrongColumnFamily[3:
> table=raw, scan=batchSmallResultSize] Time elapsed: 0.018 s <<< FAILURE!
> java.lang.AssertionError:
> Expected:
> a collection containing (SpanKind with a name that a string starting with
> "SCAN default:async" and
> SpanKind with a parentSpanId that "612b6a689e063c9b" and
> SpanData with StatusCode that is and
> SpanData having Exception with Attributes that Attributes containing [is
> ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] and
> SpanData that hasEnded)
> but:
> SpanKind with a name that a string starting with "SCAN default:async" name was
> "AsyncRegionLocator.getRegionLocation", SpanKind with a name that a string
> starting with "SCAN default:async" name was
> "Region.getScanner", SpanKind with a name that a string starting with "SCAN
> default:async" name was
> "hbase.pb.ClientService/Scan", SpanKind with a name that a string starting
> with "SCAN default:async" name was
> "RpcServer.process", SpanData having Exception with Attributes that
> Attributes containing [is ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] exception
> attributes
> Attributes was
> [ Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3544)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> >,
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
>
[GitHub] [hbase] Apache-HBase commented on pull request #4450: HBASE-18045 Added default SSH connection timeout in Integration tests
Apache-HBase commented on PR #4450: URL: https://github.com/apache/hbase/pull/4450#issuecomment-1131571514 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | compile | 0m 26s | master passed | | +1 :green_heart: | checkstyle | 0m 9s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 8s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 28m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4450 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux ad1a0df6f2a8 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4450: HBASE-18045 Added default SSH connection timeout in Integration tests
Apache-HBase commented on PR #4450: URL: https://github.com/apache/hbase/pull/4450#issuecomment-1131563079 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 59s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 4m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 43s | the patch passed | | +1 :green_heart: | compile | 0m 20s | the patch passed | | +1 :green_heart: | javac | 0m 20s | the patch passed | | +1 :green_heart: | shadedjars | 4m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 44s | hbase-it in the patch passed. | | | | 18m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 91b45861f999 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/testReport/ | | Max. process+thread count | 573 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4450: HBASE-18045 Added default SSH connection timeout in Integration tests
Apache-HBase commented on PR #4450: URL: https://github.com/apache/hbase/pull/4450#issuecomment-1131562731 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | compile | 0m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | shadedjars | 4m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 37s | hbase-it in the patch passed. | | | | 18m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 624e87796098 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/testReport/ | | Max. process+thread count | 596 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4450/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26366) Provide meaningful parent spans to ZK interactions
[
https://issues.apache.org/jira/browse/HBASE-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-26366:
-
Summary: Provide meaningful parent spans to ZK interactions (was: ZK
interaction during Master startup produces loads of root-less spans)
> Provide meaningful parent spans to ZK interactions
> --
>
> Key: HBASE-26366
> URL: https://issues.apache.org/jira/browse/HBASE-26366
> Project: HBase
> Issue Type: Improvement
> Components: tracing
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
> Attachments: image.png,
> regroup-spans-for-recoverablezookeeper.create.png
>
>
> Enable tracing with {{-Dotel.traces.sampler=always_on}} and launch the
> standalone master, check the trace store. We see lots of single-span traces,
> each a call to different {{RecoverableZooKeeper}} methods, like {{create}}
> and {{exists}}. Whatever master startup thread this is should define its own
> root span.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (HBASE-26366) ZK interaction during Master startup produces loads of root-less spans
[
https://issues.apache.org/jira/browse/HBASE-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539463#comment-17539463
]
Nick Dimiduk commented on HBASE-26366:
--
In fact this is not just Master startup, but Region Server startup as well.
> ZK interaction during Master startup produces loads of root-less spans
> --
>
> Key: HBASE-26366
> URL: https://issues.apache.org/jira/browse/HBASE-26366
> Project: HBase
> Issue Type: Improvement
> Components: tracing
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
> Attachments: image.png,
> regroup-spans-for-recoverablezookeeper.create.png
>
>
> Enable tracing with {{-Dotel.traces.sampler=always_on}} and launch the
> standalone master, check the trace store. We see lots of single-span traces,
> each a call to different {{RecoverableZooKeeper}} methods, like {{create}}
> and {{exists}}. Whatever master startup thread this is should define its own
> root span.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[GitHub] [hbase] Apache-HBase commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
Apache-HBase commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1131527030 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 36s | branch-1 passed | | +1 :green_heart: | compile | 0m 26s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 29s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 3s | branch-1 passed | | +1 :green_heart: | shadedjars | 2m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 24s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 2m 8s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 6s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 18s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 53s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 1m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 3m 11s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 24s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 1m 48s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 143m 18s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 181m 54s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestHTableMultiplexerFlushCache | | | hadoop.hbase.client.TestSnapshotCloneIndependence | | | hadoop.hbase.replication.TestReplicationEndpoint | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4377 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d6660e21bfdd 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4377/out/precommit/personality/provided.sh | | git revision | branch-1 / 1599a48687 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/11/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/11/testReport/ | | Max. process+thread count | 4323 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/11/console | | versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the
[jira] [Assigned] (HBASE-27052) TestAsyncTableScanner.testScanWrongColumnFamily is flaky
[
https://issues.apache.org/jira/browse/HBASE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reassigned HBASE-27052:
Assignee: Nick Dimiduk
> TestAsyncTableScanner.testScanWrongColumnFamily is flaky
>
>
> Key: HBASE-27052
> URL: https://issues.apache.org/jira/browse/HBASE-27052
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> [~ndimiduk] have you seen something like this before?
> {noformat}
> [ERROR]
> org.apache.hadoop.hbase.client.TestAsyncTableScanner.testScanWrongColumnFamily[3:
> table=raw, scan=batchSmallResultSize] Time elapsed: 0.018 s <<< FAILURE!
> java.lang.AssertionError:
> Expected:
> a collection containing (SpanKind with a name that a string starting with
> "SCAN default:async" and
> SpanKind with a parentSpanId that "612b6a689e063c9b" and
> SpanData with StatusCode that is and
> SpanData having Exception with Attributes that Attributes containing [is
> ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] and
> SpanData that hasEnded)
> but:
> SpanKind with a name that a string starting with "SCAN default:async" name was
> "AsyncRegionLocator.getRegionLocation", SpanKind with a name that a string
> starting with "SCAN default:async" name was
> "Region.getScanner", SpanKind with a name that a string starting with "SCAN
> default:async" name was
> "hbase.pb.ClientService/Scan", SpanKind with a name that a string starting
> with "SCAN default:async" name was
> "RpcServer.process", SpanData having Exception with Attributes that
> Attributes containing [is ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] exception
> attributes
> Attributes was
> [ Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3544)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> >,
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
>
[jira] [Commented] (HBASE-27052) TestAsyncTableScanner.testScanWrongColumnFamily is flaky
[
https://issues.apache.org/jira/browse/HBASE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539424#comment-17539424
]
Nick Dimiduk commented on HBASE-27052:
--
I have not. It looks like the assertion expect an exception to be present in
the span, but it's not found. Let me spend some time with it.
> TestAsyncTableScanner.testScanWrongColumnFamily is flaky
>
>
> Key: HBASE-27052
> URL: https://issues.apache.org/jira/browse/HBASE-27052
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.5.0
>Reporter: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> [~ndimiduk] have you seen something like this before?
> {noformat}
> [ERROR]
> org.apache.hadoop.hbase.client.TestAsyncTableScanner.testScanWrongColumnFamily[3:
> table=raw, scan=batchSmallResultSize] Time elapsed: 0.018 s <<< FAILURE!
> java.lang.AssertionError:
> Expected:
> a collection containing (SpanKind with a name that a string starting with
> "SCAN default:async" and
> SpanKind with a parentSpanId that "612b6a689e063c9b" and
> SpanData with StatusCode that is and
> SpanData having Exception with Attributes that Attributes containing [is
> ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] and
> SpanData that hasEnded)
> but:
> SpanKind with a name that a string starting with "SCAN default:async" name was
> "AsyncRegionLocator.getRegionLocation", SpanKind with a name that a string
> starting with "SCAN default:async" name was
> "Region.getScanner", SpanKind with a name that a string starting with "SCAN
> default:async" name was
> "hbase.pb.ClientService/Scan", SpanKind with a name that a string starting
> with "SCAN default:async" name was
> "RpcServer.process", SpanData having Exception with Attributes that
> Attributes containing [is ->a string ending with
> "org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException"] exception
> attributes
> Attributes was
> [ Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2991)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.newRegionScanner(RSRpcServices.java:3187)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3544)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45819)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> >,
> > org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException:
> Column family WrongColumnFamily does not exist in region
> async,,1652930099982.5fe23e4a8e57156da060c1ecfade4b97. in table 'async',
> {TABLE_ATTRIBUTES => {METADATA => {'hbase.store.file-tracker.impl' =>
> 'DEFAULT'}}}, {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false',
> VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING =>
> 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0',
> BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7831)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.lambda$getScanner$3(HRegion.java:3013)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:3002)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2997)
> at
>
[GitHub] [hbase] Apache-HBase commented on pull request #4449: HBASE-27046 The filenum in AbstractFSWAL should be monotone increasing
Apache-HBase commented on PR #4449: URL: https://github.com/apache/hbase/pull/4449#issuecomment-1131415248 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | compile | 2m 16s | master passed | | +1 :green_heart: | checkstyle | 0m 32s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 6s | the patch passed | | +1 :green_heart: | compile | 2m 10s | the patch passed | | +1 :green_heart: | javac | 2m 10s | the patch passed | | +1 :green_heart: | checkstyle | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 8m 42s | The patch causes 10 errors with Hadoop v3.3.1. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 26m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4449 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux f423c6c9e276 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/artifact/yetus-general-check/output/patch-javac-3.3.1.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4449/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #4390: HBASE-26987 The length of compact queue grows too big when the compac…
bsglz commented on PR #4390: URL: https://github.com/apache/hbase/pull/4390#issuecomment-1131363792 > The idea looks good overall. @bsglz do you have any recommendation on the max files to compact value? Do you mean the value for hbase.hstore.compaction.max? The default value is 10, i think its ok. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on a diff in pull request #4390: HBASE-26987 The length of compact queue grows too big when the compac…
bsglz commented on code in PR #4390: URL: https://github.com/apache/hbase/pull/4390#discussion_r876727305 ## hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java: ## @@ -347,7 +347,6 @@ public class HRegion implements HeapSize, PropagatingConfigurationObserver, Regi final LongAdder compactionsFailed = new LongAdder(); final LongAdder compactionNumFilesCompacted = new LongAdder(); final LongAdder compactionNumBytesCompacted = new LongAdder(); - final LongAdder compactionsQueued = new LongAdder(); Review Comment: > I think moving compaction metrics to HStore would be better. I can move the other compaction metrics in a follow-on issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4447: HBASE-27050 Support unit test pattern matching again
Apache-HBase commented on PR #4447: URL: https://github.com/apache/hbase/pull/4447#issuecomment-1131353282 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 4m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 1m 28s | the patch passed | | +1 :green_heart: | javac | 1m 28s | the patch passed | | +1 :green_heart: | shadedjars | 3m 57s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 1m 21s | root generated 6 new + 78 unchanged - 6 fixed = 84 total (was 84) | ||| _ Other Tests _ | | -1 :x: | unit | 393m 2s | root in the patch failed. | | | | 414m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4447/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bfdfe8fe2216 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4447/1/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4447/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4447/1/testReport/ | | Max. process+thread count | 2229 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4447/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] comnetwork commented on a diff in pull request #4392: HBASE-26993 Make the new framework for region replication could work …
comnetwork commented on code in PR #4392:
URL: https://github.com/apache/hbase/pull/4392#discussion_r872196855
##
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:
##
@@ -4142,15 +4147,58 @@ private static long getLongValue(final Cell cell)
throws DoNotRetryIOException {
@Override
public WriteEntry writeMiniBatchOperationsToMemStore(
- final MiniBatchOperationInProgress miniBatchOp, @Nullable
WriteEntry writeEntry)
- throws IOException {
+ final MiniBatchOperationInProgress miniBatchOp, @Nullable
WriteEntry writeEntry,
+ long now) throws IOException {
+ boolean newWriteEntry = false;
if (writeEntry == null) {
writeEntry = region.mvcc.begin();
+newWriteEntry = true;
}
super.writeMiniBatchOperationsToMemStore(miniBatchOp,
writeEntry.getWriteNumber());
+ if (newWriteEntry) {
Review Comment:
@Apache9 ，There are two cases:
case 1 is partial Mutations are Durability#SKIP_WAL , which is processed in
`HRegion.doWALAppend`
case 2 is all Mutations are Durability#SKIP_WAL, which is processed here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4448: HBASE-27051 TestReplicationSource.testReplicationSourceInitializingMetric is flaky
Apache-HBase commented on PR #4448: URL: https://github.com/apache/hbase/pull/4448#issuecomment-1131297795 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 46s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | master passed | | +1 :green_heart: | compile | 0m 50s | master passed | | +1 :green_heart: | shadedjars | 5m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 274m 52s | hbase-server in the patch failed. | | | | 300m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4448/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4448 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a1e6bd941b54 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4448/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4448/1/testReport/ | | Max. process+thread count | 2390 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4448/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4377: HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false
Apache-HBase commented on PR #4377: URL: https://github.com/apache/hbase/pull/4377#issuecomment-1131297501 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | branch-1 passed | | +1 :green_heart: | compile | 0m 27s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 32s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 0m 56s | branch-1 passed | | +1 :green_heart: | shadedjars | 1m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 27s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 1m 44s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 1m 41s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 20s | the patch passed | | +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 27s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 54s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 1m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 2m 59s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 1m 48s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 183m 23s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 215m 10s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestMasterBalanceThrottling | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles | | | hadoop.hbase.regionserver.TestRegionReplicaFailover | | | hadoop.hbase.client.TestAdmin1 | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.client.TestSnapshotCloneIndependence | | | hadoop.hbase.replication.TestReplicationEndpoint | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4377 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2258c0b1feff 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4377/out/precommit/personality/provided.sh | | git revision | branch-1 / 1599a48687 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/10/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/10/testReport/ | | Max. process+thread count | 4023 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4377/10/console | | versions | git=2.17.1 maven=3.6.0 findbugs=3.0.1 | | Powered by |
[GitHub] [hbase] Apache-HBase commented on pull request #4437: HBASE-27043 Let lock wait timeout to improve performance of SnapshotHFileCleaner
Apache-HBase commented on PR #4437: URL: https://github.com/apache/hbase/pull/4437#issuecomment-1131290409 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | master passed | | +1 :green_heart: | compile | 0m 40s | master passed | | +1 :green_heart: | shadedjars | 3m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 206m 13s | hbase-server in the patch passed. | | | | 222m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4437 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 829471ed946a 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/testReport/ | | Max. process+thread count | 2580 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4437: HBASE-27043 Let lock wait timeout to improve performance of SnapshotHFileCleaner
Apache-HBase commented on PR #4437: URL: https://github.com/apache/hbase/pull/4437#issuecomment-1131265941 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 0m 39s | master passed | | +1 :green_heart: | shadedjars | 3m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 22s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | shadedjars | 3m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 187m 14s | hbase-server in the patch passed. | | | | 204m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4437 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1ac84f082998 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76d20fc48a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/testReport/ | | Max. process+thread count | 2404 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4437/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
