[jira] [Commented] (HBASE-26802) Backport the log4j2 changes to branch-2
[
https://issues.apache.org/jira/browse/HBASE-26802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505203#comment-17505203
]
Hudson commented on HBASE-26802:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Backport the log4j2 changes to branch-2
> ---
>
> Key: HBASE-26802
> URL: https://issues.apache.org/jira/browse/HBASE-26802
> Project: HBase
> Issue Type: Task
> Components: logging
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> One thing is that, we'd better make sure that users can still use the old
> log4j.properties for logging.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26819) Minor code cleanup in and around RpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-26819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505202#comment-17505202
]
Hudson commented on HBASE-26819:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Minor code cleanup in and around RpcScheduler
> -
>
> Key: HBASE-26819
> URL: https://issues.apache.org/jira/browse/HBASE-26819
> Project: HBase
> Issue Type: Task
> Components: IPC/RPC
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> Following the coattails of HBASE-26782.
> See detailed compatibility discussion on
> https://github.com/apache/hbase/pull/4173
> See a related concern regarding {{IA.LimitedPrivate}} annotations on class
> hierarchies on HBASE-26817.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26824) TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
[
https://issues.apache.org/jira/browse/HBASE-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505198#comment-17505198
]
Hudson commented on HBASE-26824:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
> --
>
> Key: HBASE-26824
> URL: https://issues.apache.org/jira/browse/HBASE-26824
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26816) Fix CME in ReplicationSourceManager
[
https://issues.apache.org/jira/browse/HBASE-26816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505197#comment-17505197
]
Hudson commented on HBASE-26816:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Fix CME in ReplicationSourceManager
> ---
>
> Key: HBASE-26816
> URL: https://issues.apache.org/jira/browse/HBASE-26816
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Exception in thread "regionserver/hostname/ip:port"
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:832)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:162)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:155)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2623)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1175)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26817) Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
[
https://issues.apache.org/jira/browse/HBASE-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505201#comment-17505201
]
Hudson commented on HBASE-26817:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
>
>
> Key: HBASE-26817
> URL: https://issues.apache.org/jira/browse/HBASE-26817
> Project: HBase
> Issue Type: Task
> Components: compatibility
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0, 2.4.10
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {{RpcExecutor}}, an abstract base class, is marked as {{IA.Private}}.
> However, it has several subclasses that are marked as
> {{IA.LimitedPrivate(COPROC, PHOENIX)}}. I think that the base class needs to
> match the highest exposure level of any of its subclasses.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26761) TestMobStoreScanner (testGetMassive) can OOME
[
https://issues.apache.org/jira/browse/HBASE-26761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505200#comment-17505200
]
Hudson commented on HBASE-26761:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> TestMobStoreScanner (testGetMassive) can OOME
> -
>
> Key: HBASE-26761
> URL: https://issues.apache.org/jira/browse/HBASE-26761
> Project: HBase
> Issue Type: Bug
> Components: mob, test
>Affects Versions: 2.4.10
> Environment: Apache Maven 3.6.3
> Maven home: /usr/share/maven
> Java version: 1.8.0_322, vendor: Azul Systems, Inc., runtime:
> /usr/lib/jvm/zulu-8-arm64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "5.10.0-11-arm64", arch: "aarch64", family: "unix"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Running TestMobStoreScanner, looks like it may need to go on a diet. The test
> case testGetMassive will reliably OOME for me. It might be because I am using
> an aarch64 Java 8 JVM and surefire.Xmx as currently defined in pom.xml at
> 2200m is not quite enough.
> {noformat}
> 2022-02-18T17:04:04,886 ERROR [RS:0;buildbox:45315] util.OOMEChecker(40): Run
> out of memory; RSRpcServices will abort itself immediate
> ly
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> Where the OOME is thrown is usually not germane so the rest of the stacktrace
> is not included.
> A repeatable failure for me.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26582) Prune use of Random and SecureRandom objects
[
https://issues.apache.org/jira/browse/HBASE-26582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505199#comment-17505199
]
Hudson commented on HBASE-26582:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Prune use of Random and SecureRandom objects
>
>
> Key: HBASE-26582
> URL: https://issues.apache.org/jira/browse/HBASE-26582
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> While looking over Findbugs/Spotbugs findings on HBASE-26555 I noticed we
> have some violations of Spotbugs rules regarding lack of Random object
> re-use, and generally have a lot of static Random and SecureRandom objects in
> our code, especially in tests, where ThreadLocalRandom would be more than
> sufficient.
> This is nitty stuff but let's clean it up. It will reduce noise on the
> analysis reports and SecureRandom is unnecessarily slow when used wherever we
> don't need it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26715) Blocked on SyncFuture in AsyncProtobufLogWriter#write
[
https://issues.apache.org/jira/browse/HBASE-26715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505196#comment-17505196
]
Hudson commented on HBASE-26715:
Results for branch branch-2
[build #481 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/481//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Blocked on SyncFuture in AsyncProtobufLogWriter#write
> -
>
> Key: HBASE-26715
> URL: https://issues.apache.org/jira/browse/HBASE-26715
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: WAL
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Ran into an issue on hbase 2.4.6, I think related to HBASE-26679. Individual
> writes are blocking on SyncFuture, which never gets completed. Eventually
> (5m) the writes timeout and fail. But the regionserver hung on like this
> basically forever until I killed it about 14 hours later. While 26679 may fix
> the hang bug, I think we should have additional protection against such
> zombie states. In this case I think what happened is that the rollWAL was
> requested due to failed appends, but it also hung forever. See the below
> stack trace:
>
> {code:java}
> Thread 240 (regionserver/host:60020.logRoller):
> State: WAITING
> Blocked count: 38
> Waited count: 293
> Waiting on java.util.concurrent.CompletableFuture$Signaller@13342c6d
> Stack:
> java.base@11.0.5/jdk.internal.misc.Unsafe.park(Native Method)
>
> java.base@11.0.5/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796)
>
> java.base@11.0.5/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:189)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:202)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:170)
>
> app//org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:268)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:187)
> {code}
>
> The wall roller thread was stuck on this wait seemingly forever, so it was
> never able to roll the wal and get writes working again. I think we should
> add a timeout here, and abort the regionserver if a WAL cannot be rolled in a
> timely manner.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26817) Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
[
https://issues.apache.org/jira/browse/HBASE-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505194#comment-17505194
]
Hudson commented on HBASE-26817:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
>
>
> Key: HBASE-26817
> URL: https://issues.apache.org/jira/browse/HBASE-26817
> Project: HBase
> Issue Type: Task
> Components: compatibility
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0, 2.4.10
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {{RpcExecutor}}, an abstract base class, is marked as {{IA.Private}}.
> However, it has several subclasses that are marked as
> {{IA.LimitedPrivate(COPROC, PHOENIX)}}. I think that the base class needs to
> match the highest exposure level of any of its subclasses.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26761) TestMobStoreScanner (testGetMassive) can OOME
[
https://issues.apache.org/jira/browse/HBASE-26761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505193#comment-17505193
]
Hudson commented on HBASE-26761:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMobStoreScanner (testGetMassive) can OOME
> -
>
> Key: HBASE-26761
> URL: https://issues.apache.org/jira/browse/HBASE-26761
> Project: HBase
> Issue Type: Bug
> Components: mob, test
>Affects Versions: 2.4.10
> Environment: Apache Maven 3.6.3
> Maven home: /usr/share/maven
> Java version: 1.8.0_322, vendor: Azul Systems, Inc., runtime:
> /usr/lib/jvm/zulu-8-arm64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "5.10.0-11-arm64", arch: "aarch64", family: "unix"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Running TestMobStoreScanner, looks like it may need to go on a diet. The test
> case testGetMassive will reliably OOME for me. It might be because I am using
> an aarch64 Java 8 JVM and surefire.Xmx as currently defined in pom.xml at
> 2200m is not quite enough.
> {noformat}
> 2022-02-18T17:04:04,886 ERROR [RS:0;buildbox:45315] util.OOMEChecker(40): Run
> out of memory; RSRpcServices will abort itself immediate
> ly
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> Where the OOME is thrown is usually not germane so the rest of the stacktrace
> is not included.
> A repeatable failure for me.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26816) Fix CME in ReplicationSourceManager
[
https://issues.apache.org/jira/browse/HBASE-26816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505190#comment-17505190
]
Hudson commented on HBASE-26816:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix CME in ReplicationSourceManager
> ---
>
> Key: HBASE-26816
> URL: https://issues.apache.org/jira/browse/HBASE-26816
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Exception in thread "regionserver/hostname/ip:port"
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:832)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:162)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:155)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2623)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1175)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26582) Prune use of Random and SecureRandom objects
[
https://issues.apache.org/jira/browse/HBASE-26582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505192#comment-17505192
]
Hudson commented on HBASE-26582:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Prune use of Random and SecureRandom objects
>
>
> Key: HBASE-26582
> URL: https://issues.apache.org/jira/browse/HBASE-26582
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> While looking over Findbugs/Spotbugs findings on HBASE-26555 I noticed we
> have some violations of Spotbugs rules regarding lack of Random object
> re-use, and generally have a lot of static Random and SecureRandom objects in
> our code, especially in tests, where ThreadLocalRandom would be more than
> sufficient.
> This is nitty stuff but let's clean it up. It will reduce noise on the
> analysis reports and SecureRandom is unnecessarily slow when used wherever we
> don't need it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26824) TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
[
https://issues.apache.org/jira/browse/HBASE-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505191#comment-17505191
]
Hudson commented on HBASE-26824:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
> --
>
> Key: HBASE-26824
> URL: https://issues.apache.org/jira/browse/HBASE-26824
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26715) Blocked on SyncFuture in AsyncProtobufLogWriter#write
[
https://issues.apache.org/jira/browse/HBASE-26715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505189#comment-17505189
]
Hudson commented on HBASE-26715:
Results for branch branch-2.4
[build #305 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/305/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Blocked on SyncFuture in AsyncProtobufLogWriter#write
> -
>
> Key: HBASE-26715
> URL: https://issues.apache.org/jira/browse/HBASE-26715
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: WAL
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Ran into an issue on hbase 2.4.6, I think related to HBASE-26679. Individual
> writes are blocking on SyncFuture, which never gets completed. Eventually
> (5m) the writes timeout and fail. But the regionserver hung on like this
> basically forever until I killed it about 14 hours later. While 26679 may fix
> the hang bug, I think we should have additional protection against such
> zombie states. In this case I think what happened is that the rollWAL was
> requested due to failed appends, but it also hung forever. See the below
> stack trace:
>
> {code:java}
> Thread 240 (regionserver/host:60020.logRoller):
> State: WAITING
> Blocked count: 38
> Waited count: 293
> Waiting on java.util.concurrent.CompletableFuture$Signaller@13342c6d
> Stack:
> java.base@11.0.5/jdk.internal.misc.Unsafe.park(Native Method)
>
> java.base@11.0.5/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796)
>
> java.base@11.0.5/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:189)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:202)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:170)
>
> app//org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:268)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:187)
> {code}
>
> The wall roller thread was stuck on this wait seemingly forever, so it was
> never able to roll the wal and get writes working again. I think we should
> add a timeout here, and abort the regionserver if a WAL cannot be rolled in a
> timely manner.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4181: HBASE-26810 Add dynamic configuration support for system coprocessors
Apache-HBase commented on pull request #4181: URL: https://github.com/apache/hbase/pull/4181#issuecomment-1065819013 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 50s | master passed | | +1 :green_heart: | compile | 3m 0s | master passed | | +1 :green_heart: | shadedjars | 5m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 11s | the patch passed | | +1 :green_heart: | compile | 3m 5s | the patch passed | | +1 :green_heart: | javac | 3m 5s | the patch passed | | +1 :green_heart: | shadedjars | 5m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 46s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 438m 58s | root in the patch failed. | | | | 476m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4181 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9ef3238c331c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/testReport/ | | Max. process+thread count | 2244 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065799914 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 48s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 48s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 47s | branch-2 passed | | +1 :green_heart: | compile | 1m 8s | branch-2 passed | | +1 :green_heart: | shadedjars | 0m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 0m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 26s | hbase-http in the patch passed. | | -1 :x: | unit | 372m 20s | hbase-server in the patch failed. | | | | 391m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 03ad90ec9f3c 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 340cc6c6f1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/testReport/ | | Max. process+thread count | 2004 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505167#comment-17505167
]
Andrew Kyle Purtell commented on HBASE-26042:
-
[~vjasani] You might be interested in the recent attachments here.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> -
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065796872 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 53s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 6s | branch-2.5 passed | | +1 :green_heart: | compile | 1m 0s | branch-2.5 passed | | +1 :green_heart: | shadedjars | 0m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 48s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 0m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 55s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 329m 46s | hbase-server in the patch passed. | | | | 347m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ad425f8cd67a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / 5844b53dea | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/testReport/ | | Max. process+thread count | 1991 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065795580 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 24s | branch-2.5 passed | | +1 :green_heart: | compile | 0m 52s | branch-2.5 passed | | +1 :green_heart: | shadedjars | 0m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 52s | the patch passed | | +1 :green_heart: | javac | 0m 52s | the patch passed | | +1 :green_heart: | shadedjars | 0m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 49s | hbase-http in the patch passed. | | -1 :x: | unit | 327m 51s | hbase-server in the patch failed. | | | | 339m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 91cdec361604 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / 5844b53dea | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/testReport/ | | Max. process+thread count | 2449 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065795066 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 28s | branch-2 passed | | +1 :green_heart: | compile | 1m 1s | branch-2 passed | | +1 :green_heart: | shadedjars | 0m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 0m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 51s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 344m 33s | hbase-server in the patch passed. | | | | 363m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9faf91bb233b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 340cc6c6f1 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/testReport/ | | Max. process+thread count | 1888 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] frostruan commented on pull request #4115: Introduce a Snapshot Procedure
frostruan commented on pull request #4115: URL: https://github.com/apache/hbase/pull/4115#issuecomment-1065783615 Really appreciate your help, Duo @Apache9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065777415 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 1m 15s | master passed | | +1 :green_heart: | shadedjars | 5m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 33s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 5m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 451m 39s | hbase-server in the patch passed. | | | | 479m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4207 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2cac6eeb735f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/testReport/ | | Max. process+thread count | 1718 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #4115: Introduce a Snapshot Procedure
Apache9 commented on a change in pull request #4115:
URL: https://github.com/apache/hbase/pull/4115#discussion_r825218867
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
##
@@ -1264,4 +1424,55 @@ private SnapshotDescription
toSnapshotDescription(ProcedureDescription desc)
builder.setType(SnapshotDescription.Type.FLUSH);
return builder.build();
}
+
+ public void registerSnapshotProcedure(SnapshotDescription snapshot, long
procId) {
+snapshotToProcIdMap.put(snapshot, procId);
+LOG.debug("register snapshot={}, snapshot procedure id = {}",
+ ClientSnapshotDescriptionUtils.toString(snapshot), procId);
+ }
+
+ public void unregisterSnapshotProcedure(SnapshotDescription snapshot, long
procId) {
+snapshotToProcIdMap.remove(snapshot, procId);
+LOG.debug("unregister snapshot={}, snapshot procedure id = {}",
+ ClientSnapshotDescriptionUtils.toString(snapshot), procId);
+ }
+
+ public boolean snapshotProcedureEnabled() {
+return master.getConfiguration()
+ .getBoolean(SNAPSHOT_PROCEDURE_ENABLED,
SNAPSHOT_PROCEDURE_ENABLED_DEFAULT);
+ }
+
+ public ServerName acquireSnapshotVerifyWorker(SnapshotVerifyProcedure
procedure)
+ throws ProcedureSuspendedException {
+Optional worker = verifyWorkerAssigner.acquire();
+if (worker.isPresent()) {
+ LOG.debug("{} Acquired verify snapshot worker={}", procedure,
worker.get());
+ return worker.get();
+}
+verifyWorkerAssigner.suspend(procedure);
+throw new ProcedureSuspendedException();
+ }
+
+ public void releaseSnapshotVerifyWorker(SnapshotVerifyProcedure procedure,
+ ServerName worker, MasterProcedureScheduler scheduler) {
+LOG.debug("{} Release verify snapshot worker={}", procedure, worker);
+verifyWorkerAssigner.release(worker);
+verifyWorkerAssigner.wake(scheduler);
+ }
+
+ private void restoreWorkers() {
+master.getMasterProcedureExecutor().getActiveProceduresNoCopy().stream()
+ .filter(p -> p instanceof SnapshotVerifyProcedure)
+ .map(p -> (SnapshotVerifyProcedure) p)
+ .filter(p -> !p.isFinished())
+ .filter(p -> p.getServerName() != null)
+ .forEach(p -> {
+verifyWorkerAssigner.addUsedWorker(p.getServerName());
Review comment:
Seems this is the behavior of the SplitWALManager, so I guess it is OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4209: HBASE-26822 Revert the changes on hbase-daemon.sh after switching to …
Apache-HBase commented on pull request #4209: URL: https://github.com/apache/hbase/pull/4209#issuecomment-1065769905 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 18s | The patch does not generate ASF License warnings. | | | | 2m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4209 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux bd9ba8221de8 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Max. process+thread count | 32 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4209: HBASE-26822 Revert the changes on hbase-daemon.sh after switching to …
Apache-HBase commented on pull request #4209: URL: https://github.com/apache/hbase/pull/4209#issuecomment-1065768848 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4209 | | Optional Tests | | | uname | Linux 9f24cb18b01e 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Max. process+thread count | 28 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4209: HBASE-26822 Revert the changes on hbase-daemon.sh after switching to …
Apache-HBase commented on pull request #4209: URL: https://github.com/apache/hbase/pull/4209#issuecomment-1065768088 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 1m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4209 | | Optional Tests | | | uname | Linux fc4a1adcfd04 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Max. process+thread count | 39 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4209/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26828 started by Duo Zhang. - > Increase the concurrency when running UTs in pre commit job > --- > > Key: HBASE-26828 > URL: https://issues.apache.org/jira/browse/HBASE-26828 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > The default forkCount is 0.25C. We used to run on a 16 cores machine so the > concurrency is 4, but now our build nodes are all 8 cores, so the concurrency > is 2. The build time is way more slower than before. > Since we only have 2 slots on the build node, I think it is OK to increase > the forkCount to 0.5C. But I'm not sure whether we should change the default > value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
[ https://issues.apache.org/jira/browse/HBASE-26828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-26828: - Assignee: Duo Zhang > Increase the concurrency when running UTs in pre commit job > --- > > Key: HBASE-26828 > URL: https://issues.apache.org/jira/browse/HBASE-26828 > Project: HBase > Issue Type: Improvement > Components: jenkins, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > The default forkCount is 0.25C. We used to run on a 16 cores machine so the > concurrency is 4, but now our build nodes are all 8 cores, so the concurrency > is 2. The build time is way more slower than before. > Since we only have 2 slots on the build node, I think it is OK to increase > the forkCount to 0.5C. But I'm not sure whether we should change the default > value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26821) Bump dependencies in /dev-support/git-jira-release-audit
[
https://issues.apache.org/jira/browse/HBASE-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505157#comment-17505157
]
Hudson commented on HBASE-26821:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Bump dependencies in /dev-support/git-jira-release-audit
>
>
> Key: HBASE-26821
> URL: https://issues.apache.org/jira/browse/HBASE-26821
> Project: HBase
> Issue Type: Bug
> Components: dependabot
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 3.0.0-alpha-3
>
>
> Bumps urllib3 from 1.25.8 to 1.26.5 to resolve two dependabot warnings
> - CRLF injection (Moderate) urllib3 (pip) ·
> dev-support/git-jira-release-audit/requirements.txt
> - Catastrophic backtracking in URL authority parser when passed URL
> containing many @ characters (High) urllib3 (pip) ·
> dev-support/git-jira-release-audit/requirements.txt
> Bumps cryptography from 2.8 to 3.3.2 to resolve one dependabot warning
> - RSA decryption vulnerable to Bleichenbacher timing vulnerability
> (Moderate) cryptography (pip) ·
> dev-support/git-jira-release-audit/requirements.txt
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache9 opened a new pull request #4209: HBASE-26822 Revert the changes on hbase-daemon.sh after switching to …
Apache9 opened a new pull request #4209: URL: https://github.com/apache/hbase/pull/4209 …log4j2 properties file -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26802) Backport the log4j2 changes to branch-2
[ https://issues.apache.org/jira/browse/HBASE-26802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26802: -- Release Note: Use log4j2 instead of log4j for logging. Exclude log4j dependency from hbase and transitive dependencies, use log4j-1.2-api as test dependency for bridging as hadoop still need log4j for some reasons. Copy FileAppender implementation in hbase-logging as the ContainerLogAppender for YARN NodeManager extends it. All log4j.properties files have been replaced by log4j2.properties. was: Use log4j2 instead of log4j for logging. Exclude log4j dependency from hbase and transitive dependencies, use log4j-1.2-api as test dependency for bridging as hadoop still need log4j for some reasons. Copy FileAppender implementation in hbase-logging as the ContainerLogAppender for YARN NodeManager extends it. All log4j.properties files have been replaced by log4j2.xml. For log4j2, there is no 'log4j.rootLogger' config, so we need to config level and appender separately, the system properties are now 'hbase.root.logger.level' and 'hbase.root.logger.appender', for security loggers they are 'hbase.security.logger.level' and 'hbase.security.logger.appender'. But for setting them from command line, you could still use something like 'HBASE_ROOT_LOGGER=INFO,console' as we will split it and set level and appender separately. > Backport the log4j2 changes to branch-2 > --- > > Key: HBASE-26802 > URL: https://issues.apache.org/jira/browse/HBASE-26802 > Project: HBase > Issue Type: Task > Components: logging >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.5.0, 2.6.0 > > > One thing is that, we'd better make sure that users can still use the old > log4j.properties for logging. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26761) TestMobStoreScanner (testGetMassive) can OOME
[
https://issues.apache.org/jira/browse/HBASE-26761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505161#comment-17505161
]
Hudson commented on HBASE-26761:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> TestMobStoreScanner (testGetMassive) can OOME
> -
>
> Key: HBASE-26761
> URL: https://issues.apache.org/jira/browse/HBASE-26761
> Project: HBase
> Issue Type: Bug
> Components: mob, test
>Affects Versions: 2.4.10
> Environment: Apache Maven 3.6.3
> Maven home: /usr/share/maven
> Java version: 1.8.0_322, vendor: Azul Systems, Inc., runtime:
> /usr/lib/jvm/zulu-8-arm64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "5.10.0-11-arm64", arch: "aarch64", family: "unix"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Running TestMobStoreScanner, looks like it may need to go on a diet. The test
> case testGetMassive will reliably OOME for me. It might be because I am using
> an aarch64 Java 8 JVM and surefire.Xmx as currently defined in pom.xml at
> 2200m is not quite enough.
> {noformat}
> 2022-02-18T17:04:04,886 ERROR [RS:0;buildbox:45315] util.OOMEChecker(40): Run
> out of memory; RSRpcServices will abort itself immediate
> ly
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> Where the OOME is thrown is usually not germane so the rest of the stacktrace
> is not included.
> A repeatable failure for me.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26824) TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
[
https://issues.apache.org/jira/browse/HBASE-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505159#comment-17505159
]
Hudson commented on HBASE-26824:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
> --
>
> Key: HBASE-26824
> URL: https://issues.apache.org/jira/browse/HBASE-26824
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26814) Default StoreHotnessProtector to off, with logs to guide when to turn it on
[
https://issues.apache.org/jira/browse/HBASE-26814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505155#comment-17505155
]
Hudson commented on HBASE-26814:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Default StoreHotnessProtector to off, with logs to guide when to turn it on
> ---
>
> Key: HBASE-26814
> URL: https://issues.apache.org/jira/browse/HBASE-26814
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> As discussed in HBASE-26522 and somewhat in HBASE-26575,
> StoreHotnessProtector is too conservative. One can easily trigger
> RegionTooBusyException on an otherwise idle regionserver with relatively
> small amount of traffic. The idea behind this protection is good, but it
> needs to take into account current load in order to warrant being enabled by
> default.
> Until it's improved as such, we should change it to default off. One thing we
> can do to help drive usage would be to add warning logs when it would have
> triggered. Operators who are diagnosing performance issues will naturally
> look at the logs and see indication of a potential tool for mitigation.
> cc [~andrew.purt...@gmail.com]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25895) Implement a Cluster Metrics JSON endpoint
[
https://issues.apache.org/jira/browse/HBASE-25895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505154#comment-17505154
]
Hudson commented on HBASE-25895:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Implement a Cluster Metrics JSON endpoint
> -
>
> Key: HBASE-25895
> URL: https://issues.apache.org/jira/browse/HBASE-25895
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-3
>
>
> In order to build a dynamic visualization functionality, we need access to
> the underlying dataset. I think we can do a lot by exposing the
> ClusterMetrics already available via the {{o.a.h.h.client.Admin}} interface.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26817) Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
[
https://issues.apache.org/jira/browse/HBASE-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505152#comment-17505152
]
Hudson commented on HBASE-26817:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
>
>
> Key: HBASE-26817
> URL: https://issues.apache.org/jira/browse/HBASE-26817
> Project: HBase
> Issue Type: Task
> Components: compatibility
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0, 2.4.10
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {{RpcExecutor}}, an abstract base class, is marked as {{IA.Private}}.
> However, it has several subclasses that are marked as
> {{IA.LimitedPrivate(COPROC, PHOENIX)}}. I think that the base class needs to
> match the highest exposure level of any of its subclasses.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26582) Prune use of Random and SecureRandom objects
[
https://issues.apache.org/jira/browse/HBASE-26582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505160#comment-17505160
]
Hudson commented on HBASE-26582:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Prune use of Random and SecureRandom objects
>
>
> Key: HBASE-26582
> URL: https://issues.apache.org/jira/browse/HBASE-26582
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> While looking over Findbugs/Spotbugs findings on HBASE-26555 I noticed we
> have some violations of Spotbugs rules regarding lack of Random object
> re-use, and generally have a lot of static Random and SecureRandom objects in
> our code, especially in tests, where ThreadLocalRandom would be more than
> sufficient.
> This is nitty stuff but let's clean it up. It will reduce noise on the
> analysis reports and SecureRandom is unnecessarily slow when used wherever we
> don't need it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26816) Fix CME in ReplicationSourceManager
[
https://issues.apache.org/jira/browse/HBASE-26816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505158#comment-17505158
]
Hudson commented on HBASE-26816:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Fix CME in ReplicationSourceManager
> ---
>
> Key: HBASE-26816
> URL: https://issues.apache.org/jira/browse/HBASE-26816
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Exception in thread "regionserver/hostname/ip:port"
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:832)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:162)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:155)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2623)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1175)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26715) Blocked on SyncFuture in AsyncProtobufLogWriter#write
[
https://issues.apache.org/jira/browse/HBASE-26715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505156#comment-17505156
]
Hudson commented on HBASE-26715:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Blocked on SyncFuture in AsyncProtobufLogWriter#write
> -
>
> Key: HBASE-26715
> URL: https://issues.apache.org/jira/browse/HBASE-26715
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: WAL
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Ran into an issue on hbase 2.4.6, I think related to HBASE-26679. Individual
> writes are blocking on SyncFuture, which never gets completed. Eventually
> (5m) the writes timeout and fail. But the regionserver hung on like this
> basically forever until I killed it about 14 hours later. While 26679 may fix
> the hang bug, I think we should have additional protection against such
> zombie states. In this case I think what happened is that the rollWAL was
> requested due to failed appends, but it also hung forever. See the below
> stack trace:
>
> {code:java}
> Thread 240 (regionserver/host:60020.logRoller):
> State: WAITING
> Blocked count: 38
> Waited count: 293
> Waiting on java.util.concurrent.CompletableFuture$Signaller@13342c6d
> Stack:
> java.base@11.0.5/jdk.internal.misc.Unsafe.park(Native Method)
>
> java.base@11.0.5/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796)
>
> java.base@11.0.5/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:189)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:202)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:170)
>
> app//org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:268)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:187)
> {code}
>
> The wall roller thread was stuck on this wait seemingly forever, so it was
> never able to roll the wal and get writes working again. I think we should
> add a timeout here, and abort the regionserver if a WAL cannot be rolled in a
> timely manner.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26819) Minor code cleanup in and around RpcScheduler
[
https://issues.apache.org/jira/browse/HBASE-26819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505153#comment-17505153
]
Hudson commented on HBASE-26819:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> Minor code cleanup in and around RpcScheduler
> -
>
> Key: HBASE-26819
> URL: https://issues.apache.org/jira/browse/HBASE-26819
> Project: HBase
> Issue Type: Task
> Components: IPC/RPC
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> Following the coattails of HBASE-26782.
> See detailed compatibility discussion on
> https://github.com/apache/hbase/pull/4173
> See a related concern regarding {{IA.LimitedPrivate}} annotations on class
> hierarchies on HBASE-26817.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26815) TestFanOutOneBlockAsyncDFSOutput is flakey
[
https://issues.apache.org/jira/browse/HBASE-26815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505151#comment-17505151
]
Hudson commented on HBASE-26815:
Results for branch master
[build #532 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/532//console].
> TestFanOutOneBlockAsyncDFSOutput is flakey
> --
>
> Key: HBASE-26815
> URL: https://issues.apache.org/jira/browse/HBASE-26815
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: xichaomin
>Assignee: xichaomin
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11
>
> Attachments: WX20220308-164340.png
>
>
> {code:java}
> // [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 160.408 s <<< FAILURE! - in
> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput
> [ERROR]
> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput.testConnectToDatanodeFailed
> Time elapsed: 9.631 s <<< FAILURE!
> java.lang.AssertionError: expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:633)
> at
> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput.testConnectToDatanodeFailed(TestFanOutOneBlockAsyncDFSOutput.java:220)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:38)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> I added some logs and found that the files were created with the old Datanode
> that had been restarted.
> !WX20220308-164340.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505142#comment-17505142
]
Mike Fink commented on HBASE-26042:
---
Thanks Andrew - hbase.wal.provider/meta_provider were on the default setting,
we will try out filesystem and see if it still reproduces.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled
[GitHub] [hbase] Apache-HBase commented on pull request #4181: HBASE-26810 Add dynamic configuration support for system coprocessors
Apache-HBase commented on pull request #4181: URL: https://github.com/apache/hbase/pull/4181#issuecomment-1065623520 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 56s | master passed | | +1 :green_heart: | compile | 10m 54s | master passed | | +1 :green_heart: | checkstyle | 1m 44s | master passed | | +0 :ok: | refguide | 3m 36s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 14m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 10m 52s | the patch passed | | +1 :green_heart: | javac | 10m 52s | the patch passed | | +1 :green_heart: | checkstyle | 1m 39s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 3m 17s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 17m 7s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 14m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 97m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4181 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide | | uname | Linux b04a2638f99e 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4181/7/yetus-general-check/output/branch-site/book.html | | refguide | https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4181/7/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 87 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4181/7/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065623237 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | master passed | | +1 :green_heart: | compile | 0m 40s | master passed | | +1 :green_heart: | shadedjars | 3m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 28s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 336m 6s | hbase-server in the patch passed. | | | | 353m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4207 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 07cbf4eb08a7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/testReport/ | | Max. process+thread count | 2784 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505129#comment-17505129
]
Andrew Kyle Purtell edited comment on HBASE-26042 at 3/11/22, 10:20 PM:
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
I would be curious if your experience is different with the 'filesystem'
provider or if that is in fact your current configuration.
was (Author: apurtell):
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
If you have already configured the 'filesystem' provider and are seeing issues
then the issues are in shared code instead.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path,
[jira] [Comment Edited] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505129#comment-17505129
]
Andrew Kyle Purtell edited comment on HBASE-26042 at 3/11/22, 10:17 PM:
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
If you have already configured the 'filesystem' provider then the issues are in
shared code instead.
was (Author: apurtell):
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
The thread dump snippet in the OP is in AbstractFSWAL which is shared code but
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
[jira] [Comment Edited] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505129#comment-17505129
]
Andrew Kyle Purtell edited comment on HBASE-26042 at 3/11/22, 10:17 PM:
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
If you have already configured the 'filesystem' provider and are seeing issues
then the issues are in shared code instead.
was (Author: apurtell):
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
If you have already configured the 'filesystem' provider then the issues are in
shared code instead.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
>
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505129#comment-17505129
]
Andrew Kyle Purtell commented on HBASE-26042:
-
Thanks for attaching these resources. I will block out some time next week to
look at them. Perhaps others will take a look.
The thread dump snippet in the OP is in AbstractFSWAL which is shared code but
I suspect you are using the default AsyncWAL provider. You might try this
alternate configuration:
hbase.wal.provider=filesystem
hbase.wal.meta_provider=filesystem
This will configure the WAL provider factory to use instances of FSHLog
instead, the old default implementation from HBase 1, which is backed by the
standard HDFS client. AsyncWAL in contrast uses a custom HDFS client that might
only be well tested against Hadoop 3 and in any case we are still finding edge
cases there.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
>
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505128#comment-17505128
]
Benoit Sigoure commented on HBASE-26042:
For some reason Mike can't upload files (maybe new accounts aren't immediately
allowed to upload attachments?), anyway, I just posted a heap dump along with
the thread dump that was taken at the ~same time.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> -
[jira] [Updated] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benoit Sigoure updated HBASE-26042:
---
Attachment: regionserver-heap-live.hprof.gz
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next() @bci=2, line=105
> (Interpreted frame)
> -
[jira] [Updated] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benoit Sigoure updated HBASE-26042:
---
Attachment: regionserver-threaddump.log
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next() @bci=2, line=105
> (Interpreted frame)
> - com.lmax.disruptor.RingBuffer.next() @bci=4, line=263
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505127#comment-17505127
]
Mike Fink commented on HBASE-26042:
---
Trying attaching one more time.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next() @bci=2, line=105
> (Interpreted frame)
> - com.lmax.disruptor.RingBuffer.next() @bci=4, line=263 (Compiled frame)
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505125#comment-17505125
]
Mike Fink commented on HBASE-26042:
---
Attaching a thread and heap dump from a similar system to the one [~tsuna] saw
the issue on. We have this system held indefinitely for now.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next()
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505123#comment-17505123
]
Benoit Sigoure commented on HBASE-26042:
Hi Andrew, thanks for your reply. I already attached the regionserver logs as
well as the stack trace {{/dump}} from the servlet. Mike is going to post a
heap dump soon.
We've been seeing quite a few instances of this bug lately, I think a number of
the "HBase is stuck" kinda reports I've heard about over the past year or so
were likely due to this bug. We are able to reproduce it relatively easily by
taking a cluster and killing nodes randomly.
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> -
[GitHub] [hbase] Apache-HBase commented on pull request #4176: HBASE-26531 Trace coprocessor exec endpoints
Apache-HBase commented on pull request #4176: URL: https://github.com/apache/hbase/pull/4176#issuecomment-1065549675 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 31s | master passed | | +1 :green_heart: | compile | 2m 2s | master passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 6s | the patch passed | | +1 :green_heart: | compile | 2m 2s | the patch passed | | +1 :green_heart: | javac | 2m 2s | the patch passed | | +1 :green_heart: | shadedjars | 5m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 34s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 398m 14s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 6m 27s | hbase-endpoint in the patch passed. | | | | 437m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4176 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux df1c88c0edd2 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/testReport/ | | Max. process+thread count | 2114 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-endpoint U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065547028 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 11s | branch-2 passed | | +1 :green_heart: | compile | 6m 14s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 35s | branch-2 passed | | +1 :green_heart: | spotbugs | 3m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 5m 33s | the patch passed | | +1 :green_heart: | javac | 0m 44s | hbase-http generated 0 new + 30 unchanged - 3 fixed = 30 total (was 33) | | +1 :green_heart: | javac | 4m 49s | hbase-server in the patch passed. | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 19s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 4m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 55m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 28ac5c21b3c1 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 340cc6c6f1 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065522511 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 0s | Docker command '/usr/bin/docker' is too old (1.41 < API v 1.27.0). | | Subsystem | Report/Notes | |--:|:-| | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065520604 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 19s | branch-2.5 passed | | +1 :green_heart: | compile | 0m 55s | branch-2.5 passed | | +1 :green_heart: | shadedjars | 4m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 45s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 58s | hbase-http in the patch passed. | | -1 :x: | unit | 349m 4s | hbase-server in the patch failed. | | | | 371m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4f9294984817 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / eebc2e0bbb | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/testReport/ | | Max. process+thread count | 2225 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065498144 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 56s | branch-2 passed | | +1 :green_heart: | compile | 0m 58s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 3m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 340m 25s | hbase-server in the patch passed. | | | | 360m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bdbfde0235cc 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a227d76009 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/testReport/ | | Max. process+thread count | 1835 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065497134 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 24s | branch-2.5 passed | | +1 :green_heart: | compile | 0m 59s | branch-2.5 passed | | +1 :green_heart: | shadedjars | 4m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 4m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 0s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 319m 40s | hbase-server in the patch passed. | | | | 344m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4208 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e7755e86b6a4 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / eebc2e0bbb | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/testReport/ | | Max. process+thread count | 2516 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065490641 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 32s | branch-2 passed | | +1 :green_heart: | compile | 0m 51s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 0m 51s | the patch passed | | +1 :green_heart: | javac | 0m 51s | the patch passed | | +1 :green_heart: | shadedjars | 3m 20s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-http in the patch passed. | | +1 :green_heart: | unit | 331m 33s | hbase-server in the patch passed. | | | | 349m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 71f6ccea3ba3 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a227d76009 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/testReport/ | | Max. process+thread count | 2155 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-26670) HFileLinkCleaner should be added even if snapshot is disabled
[ https://issues.apache.org/jira/browse/HBASE-26670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reassigned HBASE-26670: --- Assignee: Yi Mei > HFileLinkCleaner should be added even if snapshot is disabled > - > > Key: HBASE-26670 > URL: https://issues.apache.org/jira/browse/HBASE-26670 > Project: HBase > Issue Type: Bug > Components: snapshots >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Critical > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11 > > > We encountered a problem in our cluster: > 1. Cluster has many snapshots, cause the archive directory is too big. > 2. We delete some snapshots, but the cleaner runs slowly because this is a > race in synchronized method of SnapshotHFileCleaner. > 3. We delete all snapshots, and disable snapshot > feature(hbase.snapshot.enabled=false), so the cleaner will skip the > synchronized method in SnapshotHFileCleaner. > 4. After cleaner runs, some back reference and data files under archive > directory are deleted, but they are still used by some restored tables. This > does not meet expectations. > One solution is add HFileLinkCleaner even if snapshot is disabled. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26670) HFileLinkCleaner should be added even if snapshot is disabled
[ https://issues.apache.org/jira/browse/HBASE-26670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-26670: Fix Version/s: 2.5.0 2.6.0 3.0.0-alpha-3 2.4.11 Status: Patch Available (was: Open) > HFileLinkCleaner should be added even if snapshot is disabled > - > > Key: HBASE-26670 > URL: https://issues.apache.org/jira/browse/HBASE-26670 > Project: HBase > Issue Type: Bug > Components: snapshots >Reporter: Yi Mei >Priority: Critical > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11 > > > We encountered a problem in our cluster: > 1. Cluster has many snapshots, cause the archive directory is too big. > 2. We delete some snapshots, but the cleaner runs slowly because this is a > race in synchronized method of SnapshotHFileCleaner. > 3. We delete all snapshots, and disable snapshot > feature(hbase.snapshot.enabled=false), so the cleaner will skip the > synchronized method in SnapshotHFileCleaner. > 4. After cleaner runs, some back reference and data files under archive > directory are deleted, but they are still used by some restored tables. This > does not meet expectations. > One solution is add HFileLinkCleaner even if snapshot is disabled. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4176: HBASE-26531 Trace coprocessor exec endpoints
Apache-HBase commented on pull request #4176: URL: https://github.com/apache/hbase/pull/4176#issuecomment-1065481621 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 45s | master passed | | +1 :green_heart: | compile | 1m 3s | master passed | | +1 :green_heart: | shadedjars | 3m 55s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 24s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 3m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 333m 21s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 4m 46s | hbase-endpoint in the patch passed. | | | | 361m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4176 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0faf15327ceb 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/testReport/ | | Max. process+thread count | 3273 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-endpoint U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26807) Unify CallQueueTooBigException special pause with CallDroppedException
[ https://issues.apache.org/jira/browse/HBASE-26807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault updated HBASE-26807: -- Labels: patch-available (was: ) Status: Patch Available (was: In Progress) [https://github.com/apache/hbase/pull/4180] is ready for review with all tests passing > Unify CallQueueTooBigException special pause with CallDroppedException > -- > > Key: HBASE-26807 > URL: https://issues.apache.org/jira/browse/HBASE-26807 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > > CallQueueTooBigException and CallDroppedException crop up in very similar > circumstances – the former is thrown if the request cannot be enqueued > because the queue is full; the latter is thrown when a call is dropped from > the queue to make room for another call. > HBASE-17114 added a special pause feature, which allows pausing for a longer > period of time when CallQueueTooBigException is encountered, vs the normal > pause for other exceptions. The idea here is to help reduce load so the > server can process its queue. We should extend this feature to encompass > CallDroppedException for the same reason. > Currently the config is called "hbase.client.pause.cqtbe". We should probably > deprecate that in favor of a more generic name. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HBASE-26807) Unify CallQueueTooBigException special pause with CallDroppedException
[ https://issues.apache.org/jira/browse/HBASE-26807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26807 started by Bryan Beaudreault. - > Unify CallQueueTooBigException special pause with CallDroppedException > -- > > Key: HBASE-26807 > URL: https://issues.apache.org/jira/browse/HBASE-26807 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > CallQueueTooBigException and CallDroppedException crop up in very similar > circumstances – the former is thrown if the request cannot be enqueued > because the queue is full; the latter is thrown when a call is dropped from > the queue to make room for another call. > HBASE-17114 added a special pause feature, which allows pausing for a longer > period of time when CallQueueTooBigException is encountered, vs the normal > pause for other exceptions. The idea here is to help reduce load so the > server can process its queue. We should extend this feature to encompass > CallDroppedException for the same reason. > Currently the config is called "hbase.client.pause.cqtbe". We should probably > deprecate that in favor of a more generic name. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4180: HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException
Apache-HBase commented on pull request #4180: URL: https://github.com/apache/hbase/pull/4180#issuecomment-1065477748 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 1m 52s | master passed | | +1 :green_heart: | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 1m 51s | the patch passed | | +1 :green_heart: | javac | 1m 51s | the patch passed | | +1 :green_heart: | shadedjars | 5m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 11s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 18s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 404m 11s | hbase-server in the patch passed. | | | | 445m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4180 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e7aa5d402b64 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/testReport/ | | Max. process+thread count | 2591 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4106: HBASE-26545 Implement tracing of scan
apurtell commented on a change in pull request #4106:
URL: https://github.com/apache/hbase/pull/4106#discussion_r825049614
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableImpl.java
##
@@ -232,22 +233,29 @@ public ResultScanner getScanner(Scan scan) {
}
private void scan0(Scan scan, ScanResultConsumer consumer) {
-try (ResultScanner scanner = getScanner(scan)) {
- consumer.onScanMetricsCreated(scanner.getScanMetrics());
- for (Result result; (result = scanner.next()) != null;) {
-if (!consumer.onNext(result)) {
- break;
+Span span = null;
+try (AsyncTableResultScanner scanner = rawTable.getScanner(scan)) {
+ span = scanner.getSpan();
Review comment:
Can this be null? I guess not as long as someone is aware of
getter/setter for span in `AsyncTableResultScanner` and the expected
convention. I suppose the resulting NPE would clear enough if not.
And ditto other call sites. I might have thrown something with an explicit
message about failure to maintain the code discipline but it seems fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4106: HBASE-26545 Implement tracing of scan
apurtell commented on a change in pull request #4106:
URL: https://github.com/apache/hbase/pull/4106#discussion_r825049614
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableImpl.java
##
@@ -232,22 +233,29 @@ public ResultScanner getScanner(Scan scan) {
}
private void scan0(Scan scan, ScanResultConsumer consumer) {
-try (ResultScanner scanner = getScanner(scan)) {
- consumer.onScanMetricsCreated(scanner.getScanMetrics());
- for (Result result; (result = scanner.next()) != null;) {
-if (!consumer.onNext(result)) {
- break;
+Span span = null;
+try (AsyncTableResultScanner scanner = rawTable.getScanner(scan)) {
+ span = scanner.getSpan();
Review comment:
Can this be null? I guess not as long as someone is aware of
getter/setter for span in `AsyncTableResultScanner` and the expected
convention. I suppose the resulting NPE would clear enough if not.
And ditto other call sites, it seems fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #4106: HBASE-26545 Implement tracing of scan
apurtell commented on a change in pull request #4106:
URL: https://github.com/apache/hbase/pull/4106#discussion_r825046474
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncClientScanner.java
##
@@ -173,26 +197,41 @@ private void startScan(OpenScannerResponse resp) {
.pauseForCQTBE(pauseForCQTBENs,
TimeUnit.NANOSECONDS).maxAttempts(maxAttempts)
.startLogErrorsCnt(startLogErrorsCnt).start(resp.controller,
resp.resp),
(hasMore, error) -> {
-if (error != null) {
- consumer.onError(error);
- return;
-}
-if (hasMore) {
- openScanner();
-} else {
- consumer.onComplete();
+final Span localSpan = span;
+try (Scope ignored = localSpan.makeCurrent()) {
+ if (error != null) {
+try {
+ consumer.onError(error);
+ return;
+} finally {
+ TraceUtil.setError(localSpan, error);
+ localSpan.end();
+}
+ }
+ if (hasMore) {
+openScanner();
+ } else {
+try {
+ consumer.onComplete();
+} finally {
+ localSpan.setStatus(StatusCode.OK);
+ localSpan.end();
+}
+ }
}
});
}
private CompletableFuture openScanner(int replicaId) {
-return conn.callerFactory. single().table(tableName)
-
.row(scan.getStartRow()).replicaId(replicaId).locateType(getLocateType(scan))
- .priority(scan.getPriority())
- .rpcTimeout(rpcTimeoutNs, TimeUnit.NANOSECONDS)
- .operationTimeout(scanTimeoutNs, TimeUnit.NANOSECONDS).pause(pauseNs,
TimeUnit.NANOSECONDS)
- .pauseForCQTBE(pauseForCQTBENs,
TimeUnit.NANOSECONDS).maxAttempts(maxAttempts)
-
.startLogErrorsCnt(startLogErrorsCnt).action(this::callOpenScanner).call();
+try (Scope ignored = span.makeCurrent()) {
Review comment:
Do you want to also maintain your volatile usage discipline here and
copy 'span' to a 'localSpan' first, as you have done at other call sites?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableImpl.java
##
@@ -232,22 +233,29 @@ public ResultScanner getScanner(Scan scan) {
}
private void scan0(Scan scan, ScanResultConsumer consumer) {
-try (ResultScanner scanner = getScanner(scan)) {
- consumer.onScanMetricsCreated(scanner.getScanMetrics());
- for (Result result; (result = scanner.next()) != null;) {
-if (!consumer.onNext(result)) {
- break;
+Span span = null;
+try (AsyncTableResultScanner scanner = rawTable.getScanner(scan)) {
+ span = scanner.getSpan();
Review comment:
Can this be null? I guess not as long as someone is aware of
getter/setter for span in `AsyncTableResultScanner` and the expected
convention. I suppose the resulting NPE would clear enough if not.
##
File path:
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
##
@@ -452,6 +456,53 @@ public void testScanAll() {
assertTrace("SCAN");
}
+ @Test
+ public void testScan() throws Throwable {
Review comment:
Name: testScanTracing?
##
File path:
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
##
@@ -452,6 +456,53 @@ public void testScanAll() {
assertTrace("SCAN");
}
+ @Test
+ public void testScan() throws Throwable {
+final CountDownLatch doneSignal = new CountDownLatch(1);
+final AtomicInteger count = new AtomicInteger();
+final AtomicReference throwable = new AtomicReference<>();
+final Scan scan = new Scan().setCaching(1).setMaxResultSize(1).setLimit(1);
+table.scan(scan, new ScanResultConsumer() {
+ @Override public boolean onNext(Result result) {
+if (result.getRow() != null) {
+ count.incrementAndGet();
+}
+return true;
+ }
+
+ @Override public void onError(Throwable error) {
+throwable.set(error);
+doneSignal.countDown();
+ }
+
+ @Override public void onComplete() {
+doneSignal.countDown();
+ }
+});
+doneSignal.await();
+if (throwable.get() != null) {
+ throw throwable.get();
+}
+assertThat("user code did not run. check test setup.", count.get(),
greaterThan(0));
+assertTrace("SCAN");
+ }
+
+ @Test
+ public void testGetScanner() {
Review comment:
Name: testGetScannerTracing?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/AbstractTestAsyncTableScan.java
##
@@ -17,29 +17,95 @@
*/
package org.apache.hadoop.hbase.client;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.allOf;
+import static org.hamcrest.Matchers.endsWith;
+import static org.hamcrest.Matchers.hasItem;
+import static
[GitHub] [hbase] apurtell commented on pull request #4191: HBASE-26813 Remove javax.ws.rs-api dependency
apurtell commented on pull request #4191: URL: https://github.com/apache/hbase/pull/4191#issuecomment-1065444882 SGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26802) Backport the log4j2 changes to branch-2
[ https://issues.apache.org/jira/browse/HBASE-26802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505099#comment-17505099 ] Andrew Kyle Purtell commented on HBASE-26802: - Merged. Copied the release note from HBASE-19577 to here. Thanks [~zhangduo]! > Backport the log4j2 changes to branch-2 > --- > > Key: HBASE-26802 > URL: https://issues.apache.org/jira/browse/HBASE-26802 > Project: HBase > Issue Type: Task > Components: logging >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.5.0, 2.6.0 > > > One thing is that, we'd better make sure that users can still use the old > log4j.properties for logging. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26802) Backport the log4j2 changes to branch-2
[ https://issues.apache.org/jira/browse/HBASE-26802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell resolved HBASE-26802. - Hadoop Flags: Reviewed Release Note: Use log4j2 instead of log4j for logging. Exclude log4j dependency from hbase and transitive dependencies, use log4j-1.2-api as test dependency for bridging as hadoop still need log4j for some reasons. Copy FileAppender implementation in hbase-logging as the ContainerLogAppender for YARN NodeManager extends it. All log4j.properties files have been replaced by log4j2.xml. For log4j2, there is no 'log4j.rootLogger' config, so we need to config level and appender separately, the system properties are now 'hbase.root.logger.level' and 'hbase.root.logger.appender', for security loggers they are 'hbase.security.logger.level' and 'hbase.security.logger.appender'. But for setting them from command line, you could still use something like 'HBASE_ROOT_LOGGER=INFO,console' as we will split it and set level and appender separately. Resolution: Fixed > Backport the log4j2 changes to branch-2 > --- > > Key: HBASE-26802 > URL: https://issues.apache.org/jira/browse/HBASE-26802 > Project: HBase > Issue Type: Task > Components: logging >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.5.0, 2.6.0 > > > One thing is that, we'd better make sure that users can still use the old > log4j.properties for logging. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505092#comment-17505092
]
Lars Hofhansl commented on HBASE-26812:
---
[~comnetwork] I came to the same conclusion. So what do we do about it?
The RegionScannerImpl should be closed after the local client's Get operation
has returned. It seems we need a "fake" ServerCall for this, but that's tricky,
since we need to close only those RegionScanners involved in the local
operation not all of them. Perhaps some other API that wraps RsRPCServices in
this case.
I can think of some fragile ways of fixing this, like putting another
threadlocal marker on the current thread, but I do not like that.
As is we have a timebomb in HBase. Might be best to disable any local
optimization until we have a fix.
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] apurtell merged pull request #4166: HBASE-26802 Backport the log4j2 changes to branch-2
apurtell merged pull request #4166: URL: https://github.com/apache/hbase/pull/4166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-26820) splitRegion and mergeRegion admin RPCs are in hbase-protocol but not hbase-shaded-protocol
[ https://issues.apache.org/jira/browse/HBASE-26820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reassigned HBASE-26820: --- Assignee: (was: Andrew Kyle Purtell) > splitRegion and mergeRegion admin RPCs are in hbase-protocol but not > hbase-shaded-protocol > -- > > Key: HBASE-26820 > URL: https://issues.apache.org/jira/browse/HBASE-26820 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > On parent issue [~gjacoby] indicates that splitRegion and mergeRegion admin > RPCs are in hbase-protocol but not hbase-shaded-protocol. > It would seem functionally it is not a serious break, but is a compatibility > concern. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] (HBASE-26820) splitRegion and mergeRegion admin RPCs are in hbase-protocol but not hbase-shaded-protocol
[ https://issues.apache.org/jira/browse/HBASE-26820 ] Andrew Kyle Purtell deleted comment on HBASE-26820: - was (Author: apurtell): I will attempt to sync this, let's see what happens. > splitRegion and mergeRegion admin RPCs are in hbase-protocol but not > hbase-shaded-protocol > -- > > Key: HBASE-26820 > URL: https://issues.apache.org/jira/browse/HBASE-26820 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > On parent issue [~gjacoby] indicates that splitRegion and mergeRegion admin > RPCs are in hbase-protocol but not hbase-shaded-protocol. > It would seem functionally it is not a serious break, but is a compatibility > concern. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26820) splitRegion and mergeRegion admin RPCs are in hbase-protocol but not hbase-shaded-protocol
[ https://issues.apache.org/jira/browse/HBASE-26820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505086#comment-17505086 ] Andrew Kyle Purtell commented on HBASE-26820: - RSRpcServices doesn't implement splitRegion or mergeRegion in branch-2.x so adding these back to the proto would be the wrong thing to do. We could remove them from hbase-protocol but if we were to do that I'd want to do a resync of wider scope, more like the scope of the parent. I think the idea of hbase-protocol is it is deprecated and transitional and frozen in time in branch-2.x, and then removed in branch-3.x / master. Perhaps the only thing that is missing is what I mentioned on the parent issue... sufficient comments and annotations to the effect that hbase-protocol is all deprecated. splitRegion and mergeRegion admin RPCs direct to regionservers are not supported in hbase-2 after those functions were reimplemented using master governed Procedures. Region management admin requests all go to the master. Is there a need to bring them back, e.g. something in Phoenix maybe? [~gjacoby] > splitRegion and mergeRegion admin RPCs are in hbase-protocol but not > hbase-shaded-protocol > -- > > Key: HBASE-26820 > URL: https://issues.apache.org/jira/browse/HBASE-26820 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > On parent issue [~gjacoby] indicates that splitRegion and mergeRegion admin > RPCs are in hbase-protocol but not hbase-shaded-protocol. > It would seem functionally it is not a serious break, but is a compatibility > concern. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26817) Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
[
https://issues.apache.org/jira/browse/HBASE-26817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505083#comment-17505083
]
Hudson commented on HBASE-26817:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
>
>
> Key: HBASE-26817
> URL: https://issues.apache.org/jira/browse/HBASE-26817
> Project: HBase
> Issue Type: Task
> Components: compatibility
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0, 2.4.10
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> {{RpcExecutor}}, an abstract base class, is marked as {{IA.Private}}.
> However, it has several subclasses that are marked as
> {{IA.LimitedPrivate(COPROC, PHOENIX)}}. I think that the base class needs to
> match the highest exposure level of any of its subclasses.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26715) Blocked on SyncFuture in AsyncProtobufLogWriter#write
[
https://issues.apache.org/jira/browse/HBASE-26715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505079#comment-17505079
]
Hudson commented on HBASE-26715:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Blocked on SyncFuture in AsyncProtobufLogWriter#write
> -
>
> Key: HBASE-26715
> URL: https://issues.apache.org/jira/browse/HBASE-26715
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: WAL
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Ran into an issue on hbase 2.4.6, I think related to HBASE-26679. Individual
> writes are blocking on SyncFuture, which never gets completed. Eventually
> (5m) the writes timeout and fail. But the regionserver hung on like this
> basically forever until I killed it about 14 hours later. While 26679 may fix
> the hang bug, I think we should have additional protection against such
> zombie states. In this case I think what happened is that the rollWAL was
> requested due to failed appends, but it also hung forever. See the below
> stack trace:
>
> {code:java}
> Thread 240 (regionserver/host:60020.logRoller):
> State: WAITING
> Blocked count: 38
> Waited count: 293
> Waiting on java.util.concurrent.CompletableFuture$Signaller@13342c6d
> Stack:
> java.base@11.0.5/jdk.internal.misc.Unsafe.park(Native Method)
>
> java.base@11.0.5/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796)
>
> java.base@11.0.5/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823)
>
> java.base@11.0.5/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(AsyncProtobufLogWriter.java:189)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(AsyncProtobufLogWriter.java:202)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(AbstractProtobufLogWriter.java:170)
>
> app//org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:113)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:669)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:130)
>
> app//org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:841)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:268)
>
> app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:187)
> {code}
>
> The wall roller thread was stuck on this wait seemingly forever, so it was
> never able to roll the wal and get writes working again. I think we should
> add a timeout here, and abort the regionserver if a WAL cannot be rolled in a
> timely manner.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26824) TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
[
https://issues.apache.org/jira/browse/HBASE-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505081#comment-17505081
]
Hudson commented on HBASE-26824:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582
> --
>
> Key: HBASE-26824
> URL: https://issues.apache.org/jira/browse/HBASE-26824
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26552) Introduce retry to logroller to avoid abort
[
https://issues.apache.org/jira/browse/HBASE-26552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505078#comment-17505078
]
Hudson commented on HBASE-26552:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce retry to logroller to avoid abort
> ---
>
> Key: HBASE-26552
> URL: https://issues.apache.org/jira/browse/HBASE-26552
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-2, 2.4.10
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> When calling RollController#rollWal in AbstractWALRoller, the regionserver
> may abort when encounters exception,
> {code:java}
> ...
> } catch (FailedLogCloseException | ConnectException e) {
> abort("Failed log close in log roller", e);
> } catch (IOException ex) {
> // Abort if we get here. We probably won't recover an IOE. HBASE-1132
> abort("IOE in log roller",
> ex instanceof RemoteException ? ((RemoteException)
> ex).unwrapRemoteException() : ex);
> } catch (Exception ex) {
> LOG.error("Log rolling failed", ex);
> abort("Log rolling failed", ex);
> } {code}
> I think we should support retry of rollWal here to avoid recovering the
> service by killing regionserver. The restart of regionserver is costly and
> very not friendly to the availability.
> I find that when creating new writer for the WAL in
> FanOutOneBlockAsyncDFSOutputHelper#createOutput, it supports retry to
> addBlock by setting this config "hbase.fs.async.create.retries". The idea of
> retry to roll WAL is similar to it, they both try best to make roll WAL
> succeed.
> But the initialization of new WAL writer also includes flushing the write
> buffer flush and waiting until it is completed by
> AsyncProtobufLogWriter#writeMagicAndWALHeader, which can also fail by some
> hardware reasons. The regionserver connected to the datanodes after addBlock,
> but that not means the magic and header can be flushed successfully.
> {code:java}
> protected long writeMagicAndWALHeader(byte[] magic, WALHeader header) throws
> IOException {
> return write(future -> {
> output.write(magic);
> try {
> header.writeDelimitedTo(asyncOutputWrapper);
> } catch (IOException e) {
> // should not happen
> throw new AssertionError(e);
> }
> addListener(output.flush(false), (len, error) -> {
> if (error != null) {
> future.completeExceptionally(error);
> } else {
> future.complete(len);
> }
> });
> });
> }{code}
> We have found that in our production clusters, there exists aborting of
> regionservers that caused by "IOE in log roller". And the practice in our
> clusters is that just one more retry of rollWal can make the WAL roll
> complete and continue serving.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26782) Minor code cleanup in and around RpcExecutor
[
https://issues.apache.org/jira/browse/HBASE-26782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505075#comment-17505075
]
Hudson commented on HBASE-26782:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Minor code cleanup in and around RpcExecutor
>
>
> Key: HBASE-26782
> URL: https://issues.apache.org/jira/browse/HBASE-26782
> Project: HBase
> Issue Type: Task
> Components: IPC/RPC
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> While working on tracing, I see some minor cleanup that can be done around
> the RpcExecutor class. The implementations of the {{dispatch}} methods don't
> actually throw the exceptions defined in their signatures, and there's some
> simplification to be done and checkstyle warnings to resolve while I'm in
> there.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26582) Prune use of Random and SecureRandom objects
[
https://issues.apache.org/jira/browse/HBASE-26582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505080#comment-17505080
]
Hudson commented on HBASE-26582:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Prune use of Random and SecureRandom objects
>
>
> Key: HBASE-26582
> URL: https://issues.apache.org/jira/browse/HBASE-26582
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> While looking over Findbugs/Spotbugs findings on HBASE-26555 I noticed we
> have some violations of Spotbugs rules regarding lack of Random object
> re-use, and generally have a lot of static Random and SecureRandom objects in
> our code, especially in tests, where ThreadLocalRandom would be more than
> sufficient.
> This is nitty stuff but let's clean it up. It will reduce noise on the
> analysis reports and SecureRandom is unnecessarily slow when used wherever we
> don't need it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26804) Missing opentelemetry agent in hadoop-two-compat.xml
[
https://issues.apache.org/jira/browse/HBASE-26804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505077#comment-17505077
]
Hudson commented on HBASE-26804:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Missing opentelemetry agent in hadoop-two-compat.xml
>
>
> Key: HBASE-26804
> URL: https://issues.apache.org/jira/browse/HBASE-26804
> Project: HBase
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.5.0, 2.6.0
>Reporter: Duo Zhang
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Blocker
> Fix For: 2.5.0, 2.6.0
>
>
> We do not have hadoop-two-compat.xml on master, this is probably why we
> missed this file when backporting to branch-2.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26814) Default StoreHotnessProtector to off, with logs to guide when to turn it on
[
https://issues.apache.org/jira/browse/HBASE-26814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505076#comment-17505076
]
Hudson commented on HBASE-26814:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Default StoreHotnessProtector to off, with logs to guide when to turn it on
> ---
>
> Key: HBASE-26814
> URL: https://issues.apache.org/jira/browse/HBASE-26814
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> As discussed in HBASE-26522 and somewhat in HBASE-26575,
> StoreHotnessProtector is too conservative. One can easily trigger
> RegionTooBusyException on an otherwise idle regionserver with relatively
> small amount of traffic. The idea behind this protection is good, but it
> needs to take into account current load in order to warrant being enabled by
> default.
> Until it's improved as such, we should change it to default off. One thing we
> can do to help drive usage would be to add warning logs when it would have
> triggered. Operators who are diagnosing performance issues will naturally
> look at the logs and see indication of a potential tool for mitigation.
> cc [~andrew.purt...@gmail.com]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26761) TestMobStoreScanner (testGetMassive) can OOME
[
https://issues.apache.org/jira/browse/HBASE-26761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505082#comment-17505082
]
Hudson commented on HBASE-26761:
Results for branch branch-2.5
[build #60 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/60/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMobStoreScanner (testGetMassive) can OOME
> -
>
> Key: HBASE-26761
> URL: https://issues.apache.org/jira/browse/HBASE-26761
> Project: HBase
> Issue Type: Bug
> Components: mob, test
>Affects Versions: 2.4.10
> Environment: Apache Maven 3.6.3
> Maven home: /usr/share/maven
> Java version: 1.8.0_322, vendor: Azul Systems, Inc., runtime:
> /usr/lib/jvm/zulu-8-arm64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "5.10.0-11-arm64", arch: "aarch64", family: "unix"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.11
>
>
> Running TestMobStoreScanner, looks like it may need to go on a diet. The test
> case testGetMassive will reliably OOME for me. It might be because I am using
> an aarch64 Java 8 JVM and surefire.Xmx as currently defined in pom.xml at
> 2200m is not quite enough.
> {noformat}
> 2022-02-18T17:04:04,886 ERROR [RS:0;buildbox:45315] util.OOMEChecker(40): Run
> out of memory; RSRpcServices will abort itself immediate
> ly
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> Where the OOME is thrown is usually not germane so the rest of the stacktrace
> is not included.
> A repeatable failure for me.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4180: HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException
Apache-HBase commented on pull request #4180: URL: https://github.com/apache/hbase/pull/4180#issuecomment-1065391898 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 18s | master passed | | +1 :green_heart: | compile | 1m 52s | master passed | | +1 :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 1m 30s | the patch passed | | +1 :green_heart: | javac | 1m 30s | the patch passed | | +1 :green_heart: | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 7s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 0s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 324m 48s | hbase-server in the patch passed. | | | | 355m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4180 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 09f971980d34 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/testReport/ | | Max. process+thread count | 2484 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4180/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065346676 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 2m 45s | master passed | | +1 :green_heart: | checkstyle | 0m 44s | master passed | | +1 :green_heart: | spotbugs | 1m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 5s | the patch passed | | +1 :green_heart: | compile | 2m 40s | the patch passed | | +1 :green_heart: | javac | 2m 40s | the patch passed | | +1 :green_heart: | checkstyle | 0m 37s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 56s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 1m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 8s | The patch does not generate ASF License warnings. | | | | 36m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4207 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 688b19024257 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
ndimiduk commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065320953 > Ah, I see. This patch requires an upgrade of hbase-thirdparty to 4.0+. Nope, that's not it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4206: HBASE-26827 RegionServer JVM crash when compact mob table
Apache-HBase commented on pull request #4206: URL: https://github.com/apache/hbase/pull/4206#issuecomment-1065319196 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 9s | master passed | | +1 :green_heart: | compile | 1m 54s | master passed | | +1 :green_heart: | shadedjars | 7m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 20s | the patch passed | | +1 :green_heart: | compile | 1m 39s | the patch passed | | +1 :green_heart: | javac | 1m 39s | the patch passed | | +1 :green_heart: | shadedjars | 7m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 463m 19s | hbase-server in the patch passed. | | | | 501m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4206 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ee69bdcd2245 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/1/testReport/ | | Max. process+thread count | 1832 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26828) Increase the concurrency when running UTs in pre commit job
Duo Zhang created HBASE-26828: - Summary: Increase the concurrency when running UTs in pre commit job Key: HBASE-26828 URL: https://issues.apache.org/jira/browse/HBASE-26828 Project: HBase Issue Type: Improvement Components: jenkins, test Reporter: Duo Zhang The default forkCount is 0.25C. We used to run on a 16 cores machine so the concurrency is 4, but now our build nodes are all 8 cores, so the concurrency is 2. The build time is way more slower than before. Since we only have 2 slots on the build node, I think it is OK to increase the forkCount to 0.5C. But I'm not sure whether we should change the default value in pom, or we just set it in the pre commit job. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] ndimiduk commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
ndimiduk commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065261655 Ah, I see. This patch requires an upgrade of hbase-thirdparty to 4.0+. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4166: HBASE-26802 Backport the log4j2 changes to branch-2
Apache-HBase commented on pull request #4166: URL: https://github.com/apache/hbase/pull/4166#issuecomment-1065240422 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 39s | branch-2 passed | | +1 :green_heart: | compile | 3m 7s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 12m 11s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 3s | the patch passed | | +1 :green_heart: | compile | 2m 27s | the patch passed | | -0 :warning: | javac | 2m 27s | root generated 2 new + 37 unchanged - 0 fixed = 39 total (was 37) | | +1 :green_heart: | shadedjars | 0m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 11m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 431m 29s | root in the patch passed. | | | | 482m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4166/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4166 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 425389eb2554 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / e46fbedf56 | | Default Java | AdoptOpenJDK-11.0.10+9 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4166/9/artifact/yetus-jdk11-hadoop3-check/output/diff-compile-javac-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4166/9/testReport/ | | Max. process+thread count | 2212 (vs. ulimit of 12500) | | modules | C: hbase-logging hbase-protocol-shaded hbase-common hbase-metrics-api hbase-hadoop-compat hbase-metrics hbase-hadoop2-compat hbase-client hbase-zookeeper hbase-replication hbase-http hbase-asyncfs hbase-procedure hbase-server hbase-mapreduce hbase-testing-util hbase-thrift hbase-rsgroup hbase-shell hbase-endpoint hbase-it hbase-rest hbase-examples hbase-shaded hbase-shaded/hbase-shaded-client hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce hbase-hbtop hbase-compression/hbase-compression-aircompressor hbase-compression/hbase-compression-lz4 hbase-compression/hbase-compression-snappy hbase-compression/hbase-compression-xz hbase-compression/hbase-compression-zstd hbase-assembly hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants hbase-archetypes/hbase-client-project hbase-archetypes/hbase-shaded-client-project . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4166/9/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065230839 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 20s | master passed | | +1 :green_heart: | compile | 2m 45s | master passed | | +1 :green_heart: | checkstyle | 0m 41s | master passed | | +1 :green_heart: | spotbugs | 1m 36s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 4s | the patch passed | | +1 :green_heart: | compile | 2m 50s | the patch passed | | +1 :green_heart: | javac | 2m 50s | the patch passed | | +1 :green_heart: | checkstyle | 0m 46s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 21s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 1m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 38m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4207 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7f815c2fd458 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4208: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2.5
Apache-HBase commented on pull request #4208: URL: https://github.com/apache/hbase/pull/4208#issuecomment-1065228849 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.5 Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 14s | branch-2.5 passed | | +1 :green_heart: | compile | 3m 43s | branch-2.5 passed | | +1 :green_heart: | checkstyle | 0m 54s | branch-2.5 passed | | +1 :green_heart: | spotbugs | 2m 18s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | compile | 3m 32s | the patch passed | | +1 :green_heart: | javac | 0m 27s | hbase-http generated 0 new + 30 unchanged - 3 fixed = 30 total (was 33) | | +1 :green_heart: | javac | 3m 5s | hbase-server in the patch passed. | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 15m 52s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 48m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4208 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux fa8e727c5bf5 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / eebc2e0bbb | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4208/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4176: HBASE-26531 Trace coprocessor exec endpoints
Apache-HBase commented on pull request #4176: URL: https://github.com/apache/hbase/pull/4176#issuecomment-1065228621 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 25s | master passed | | +1 :green_heart: | compile | 7m 39s | master passed | | +1 :green_heart: | checkstyle | 2m 8s | master passed | | +1 :green_heart: | spotbugs | 5m 10s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 8m 8s | the patch passed | | -0 :warning: | javac | 5m 34s | hbase-server generated 4 new + 189 unchanged - 4 fixed = 193 total (was 193) | | -0 :warning: | javac | 0m 51s | hbase-endpoint generated 3 new + 33 unchanged - 0 fixed = 36 total (was 33) | | -0 :warning: | checkstyle | 0m 40s | hbase-client: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 22m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 6m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 77m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4176 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux 7a1a44873965 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 71ff3ec323 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-endpoint.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server hbase-endpoint U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4176/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065201163 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 1s | Docker command '/usr/bin/docker' is too old (1.41 < API v 1.27.0). | | Subsystem | Report/Notes | |--:|:-| | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4207: HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
Apache-HBase commented on pull request #4207: URL: https://github.com/apache/hbase/pull/4207#issuecomment-1065199346 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 0m 16s | Docker failed to build run-specific yetus/hbase:tp-9227}. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/4207 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4207/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4200: Backport "HBASE-25895 Implement a Cluster Metrics JSON endpoint" to branch-2
Apache-HBase commented on pull request #4200: URL: https://github.com/apache/hbase/pull/4200#issuecomment-1065193921 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 54s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 37s | branch-2 passed | | +1 :green_heart: | compile | 2m 34s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 45s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 33s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 17s | the patch passed | | +1 :green_heart: | compile | 2m 33s | the patch passed | | +1 :green_heart: | javac | 0m 17s | hbase-http generated 0 new + 30 unchanged - 3 fixed = 30 total (was 33) | | +1 :green_heart: | javac | 2m 16s | hbase-server in the patch passed. | | +1 :green_heart: | checkstyle | 0m 44s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 8m 9s | The patch causes 10 errors with Hadoop v3.2.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 26m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4200 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux e3279a17c9ed 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / a227d76009 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-http hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4200/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
