[jira] [Commented] (HBASE-27148) Move minimum hadoop 3 support version to 3.2.3
[
https://issues.apache.org/jira/browse/HBASE-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564516#comment-17564516
]
Hudson commented on HBASE-27148:
Results for branch branch-2
[build #587 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move minimum hadoop 3 support version to 3.2.3
> ---
>
> Key: HBASE-27148
> URL: https://issues.apache.org/jira/browse/HBASE-27148
> Project: HBase
> Issue Type: Task
> Components: dependencies, hadoop3, security
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> Seems the hadoop community will not make newer 3.1.x release so let's move
> the minimun hadoop 3 versions to 3.2.3, due to security issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27180) Multiple possible buffer leaks
[
https://issues.apache.org/jira/browse/HBASE-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564515#comment-17564515
]
Hudson commented on HBASE-27180:
Results for branch branch-2
[build #587 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Multiple possible buffer leaks
> --
>
> Key: HBASE-27180
> URL: https://issues.apache.org/jira/browse/HBASE-27180
> Project: HBase
> Issue Type: Bug
> Components: netty, regionserver
>Reporter: Norman Maurer
>Assignee: Norman Maurer
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> When using ByteBuf you need to be very careful about releasing it as
> otherwise you might leak data. There are various places in the code-base
> where such a leak could happen as the buffer is not correctly released
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27078) Allow configuring a separate timeout for meta scans
[
https://issues.apache.org/jira/browse/HBASE-27078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564514#comment-17564514
]
Hudson commented on HBASE-27078:
Results for branch branch-2
[build #587 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/587/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow configuring a separate timeout for meta scans
> ---
>
> Key: HBASE-27078
> URL: https://issues.apache.org/jira/browse/HBASE-27078
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> There is a {{hbase.client.meta.operation.timeout}} but it does not apply to
> meta scans, which are the primary use-case for clients (i.e. through
> RegionLocator).
> Many user-facing clients may want to have low rpc and scan timeouts. However,
> in periods of meta hotspotting, those timeouts can be way too low for the
> meta scans. The problem with low timeouts for meta scans is that without a
> populated MetaCache, user requests cannot succeed. In fact, user requests
> will continually try to re-scan meta until the MetaCache is populated. So
> having a lower rpc timeout will cause a situation where meta scans cannot
> succeed, and thus user requests cannot succeed. In this case I think it'd be
> preferable to relax the rpc timeout for meta requests so that a few long
> requests can unblock many faster requests.
> My suggestion would be to add an {{hbase.client.meta.rpc.timeout}} and ensure
> that it applies to meta scans. I also think it would be less confusing to
> have {{hbase.client.meta.operation.timeout}} apply as the scanner timeout
> period for meta scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27180) Multiple possible buffer leaks
[
https://issues.apache.org/jira/browse/HBASE-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564512#comment-17564512
]
Hudson commented on HBASE-27180:
Results for branch branch-2.4
[build #386 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/386/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/386/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/386/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/386/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/386/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Multiple possible buffer leaks
> --
>
> Key: HBASE-27180
> URL: https://issues.apache.org/jira/browse/HBASE-27180
> Project: HBase
> Issue Type: Bug
> Components: netty, regionserver
>Reporter: Norman Maurer
>Assignee: Norman Maurer
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> When using ByteBuf you need to be very careful about releasing it as
> otherwise you might leak data. There are various places in the code-base
> where such a leak could happen as the buffer is not correctly released
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564509#comment-17564509 ] Viraj Jasani commented on HBASE-27183: -- Sure [~zhangduo]. Until we get consensus on this, the PR stays blocked. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.client.consume.master.proxy.port: Clients/Regionservers can use this > config (boolean) to determine whether to connect to active master on new > proxy port that master has exposed or continue using original port of master > for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-27183: - Fix Version/s: (was: 2.6.0) (was: 2.5.1) > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.client.consume.master.proxy.port: Clients/Regionservers can use this > config (boolean) to determine whether to connect to active master on new > proxy port that master has exposed or continue using original port of master > for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-27183: - Description: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. Configs: # hbase.master.expose.proxy.port: Master can use this config (int) to expose new proxy port on active and backup master znodes. # hbase.client.consume.master.proxy.port: Clients/Regionservers can use this config (boolean) to determine whether to connect to active master on new proxy port that master has exposed or continue using original port of master for connection. was: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. Configs: # hbase.master.expose.proxy.port: Master can use this config (int) to expose new proxy port on active and backup master znodes. # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can use this config (boolean) to determine whether to connect to active master on new proxy port that master has exposed or continue using original port of master for connection. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.client.consume.master.proxy.port: Clients/Regionservers can use this > config (boolean) to determine whether to connect to active master on new > proxy port that master has exposed or continue using original port of master > for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26950) Use AsyncConnection in ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-26950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564502#comment-17564502 ] chenglei commented on HBASE-26950: -- Pushed to branch-2 and 2.5, [~bbeaudreault], thank you very much for help and review. > Use AsyncConnection in ReplicationSink > -- > > Key: HBASE-26950 > URL: https://issues.apache.org/jira/browse/HBASE-26950 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.11 >Reporter: Bryan Beaudreault >Priority: Major > Labels: branch-2, replication > Fix For: 2.5.0, 2.6.0 > > > We don't need to necessarily rewrite ReplicationSink to work fully async. I > think it would simply benefit from ConnectionFactory.createAsyncConnection > instead of ConnectionFactory.createConnection. > The reasons for this are: > * AsyncConnection is the more modern implementation, the only implementation > in master, and where most of the efforts will be going forward. > * ReplicationSink only does batch calls, and batch calls are done with > AsyncProcess. It's likely that the native AsyncTable is better than > AsyncProcess for this. > ** One specific example, AsyncProcess calls findAllLocationsOrFail > sequentially for all actions in a batch. This can take quite a while with the > default replication batch size of 5k, if actions are spread across many > regions. In AsyncTable, these calls are done in parallel -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-26950) Use AsyncConnection in ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-26950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei resolved HBASE-26950. -- Assignee: chenglei Resolution: Fixed > Use AsyncConnection in ReplicationSink > -- > > Key: HBASE-26950 > URL: https://issues.apache.org/jira/browse/HBASE-26950 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.11 >Reporter: Bryan Beaudreault >Assignee: chenglei >Priority: Major > Labels: branch-2, replication > Fix For: 2.5.0, 2.6.0 > > > We don't need to necessarily rewrite ReplicationSink to work fully async. I > think it would simply benefit from ConnectionFactory.createAsyncConnection > instead of ConnectionFactory.createConnection. > The reasons for this are: > * AsyncConnection is the more modern implementation, the only implementation > in master, and where most of the efforts will be going forward. > * ReplicationSink only does batch calls, and batch calls are done with > AsyncProcess. It's likely that the native AsyncTable is better than > AsyncProcess for this. > ** One specific example, AsyncProcess calls findAllLocationsOrFail > sequentially for all actions in a batch. This can take quite a while with the > default replication batch size of 5k, if actions are spread across many > regions. In AsyncTable, these calls are done in parallel -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-26950) Use AsyncConnection in ReplicationSink
[ https://issues.apache.org/jira/browse/HBASE-26950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenglei updated HBASE-26950: - Fix Version/s: 2.5.0 2.6.0 > Use AsyncConnection in ReplicationSink > -- > > Key: HBASE-26950 > URL: https://issues.apache.org/jira/browse/HBASE-26950 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.11 >Reporter: Bryan Beaudreault >Priority: Major > Labels: branch-2, replication > Fix For: 2.5.0, 2.6.0 > > > We don't need to necessarily rewrite ReplicationSink to work fully async. I > think it would simply benefit from ConnectionFactory.createAsyncConnection > instead of ConnectionFactory.createConnection. > The reasons for this are: > * AsyncConnection is the more modern implementation, the only implementation > in master, and where most of the efforts will be going forward. > * ReplicationSink only does batch calls, and batch calls are done with > AsyncProcess. It's likely that the native AsyncTable is better than > AsyncProcess for this. > ** One specific example, AsyncProcess calls findAllLocationsOrFail > sequentially for all actions in a batch. This can take quite a while with the > default replication batch size of 5k, if actions are spread across many > regions. In AsyncTable, these calls are done in parallel -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] comnetwork merged pull request #4607: HBASE-26950 Use AsyncConnection in ReplicationSink
comnetwork merged PR #4607: URL: https://github.com/apache/hbase/pull/4607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564500#comment-17564500 ] Duo Zhang commented on HBASE-27183: --- If the trick is done outside HBase, I think the correct way to address the remaining problem should also be done outside HBase. That means, you can implement something like a reverse proxy, by our own, and expose this reverse proxy to all the region server. The related trick logic can be included in this reverse proxy. FWIW, I do not feel like this is a general enough scenario which worth to add new stuff in HBase. For example, if masters are deployed with K8s, sometimes both the host name and port which are registered to zookeeper can not be directly connected, then we should also include a special host name field in the proto? And even on master we can not know what is the valid host name... Let's discuss more here or on the mailing list, before we land the actual changes. Thanks. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can > use this config (boolean) to determine whether to connect to active master on > new proxy port that master has exposed or continue using original port of > master for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564500#comment-17564500 ] Duo Zhang edited comment on HBASE-27183 at 7/9/22 3:14 AM: --- If the trick is done outside HBase, I think the correct way to address the remaining problem should also be done outside HBase. That means, you can implement something like a reverse proxy by your own, and expose this reverse proxy to all the region server. The related trick logic can be included in this reverse proxy. FWIW, I do not feel like this is a general enough scenario which worth to add new stuff in HBase. For example, if masters are deployed with K8s, sometimes both the host name and port which are registered to zookeeper can not be directly connected, then we should also include a special host name field in the proto? And even on master we can not know what is the valid host name... Let's discuss more here or on the mailing list, before we land the actual changes. Thanks. was (Author: apache9): If the trick is done outside HBase, I think the correct way to address the remaining problem should also be done outside HBase. That means, you can implement something like a reverse proxy, by our own, and expose this reverse proxy to all the region server. The related trick logic can be included in this reverse proxy. FWIW, I do not feel like this is a general enough scenario which worth to add new stuff in HBase. For example, if masters are deployed with K8s, sometimes both the host name and port which are registered to zookeeper can not be directly connected, then we should also include a special host name field in the proto? And even on master we can not know what is the valid host name... Let's discuss more here or on the mailing list, before we land the actual changes. Thanks. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can > use this config (boolean) to determine whether to connect to active master on > new proxy port that master has exposed or continue using original port of > master for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[
https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564499#comment-17564499
]
Viraj Jasani edited comment on HBASE-27183 at 7/9/22 3:10 AM:
--
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port in the znode but in
addition to that, also let it expose new proxy port in the znode, for any
client to connect. So masterAddress and backupAddress znodes both will have new
proto field named masterProxyPort. Now regionservers still continue using same
ServerName object retrieved/deserialized from znode but with a new config, they
can switch to using new proxy port (only if master has exposed this port on
znode in the first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port of master.
On the master host, "proxy port to original port" redirection is automatically
done, something outside the scope of HBase/Hadoop application layer.
cc [~apurtell]
was (Author: vjasani):
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znode but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port of master.
On the master host, "proxy port to original port" redirection is automatically
done, something outside the scope of HBase/Hadoop application layer.
cc [~apurtell]
> Support regionserver to connect to HMaster proxy port
> -
>
> Key: HBASE-27183
> URL: https://issues.apache.org/jira/browse/HBASE-27183
> Project: HBase
> Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4
>
>
> Regionservers get active master address from Zookeeper/Master registry and
> tries to make RPC calls to master.
> For security concerns, regionservers might require making connection to a
> different proxy port of master rather than it's original port retrieved from
> Zookeeper.
> Configs:
> # hbase.master.expose.proxy.port: Master can use this config (int) to expose
> new proxy port on active and backup master znodes.
> # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can
> use this config (boolean) to determine whether to connect to active master on
> new proxy port that master has exposed or continue using original port of
> master for connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-27183: - Fix Version/s: 2.6.0 2.5.1 > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can > use this config (boolean) to determine whether to connect to active master on > new proxy port that master has exposed or continue using original port of > master for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[
https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564499#comment-17564499
]
Viraj Jasani edited comment on HBASE-27183 at 7/9/22 3:08 AM:
--
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znode but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port of master.
On the master host, "proxy port to original port" redirection is automatically
done, something outside the scope of HBase/Hadoop application layer.
cc [~apurtell]
was (Author: vjasani):
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znodes but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port of master.
On the master host, "proxy port to original port" redirection is automatically
done, something outside the scope of HBase/Hadoop application layer.
cc [~apurtell]
> Support regionserver to connect to HMaster proxy port
> -
>
> Key: HBASE-27183
> URL: https://issues.apache.org/jira/browse/HBASE-27183
> Project: HBase
> Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> Regionservers get active master address from Zookeeper/Master registry and
> tries to make RPC calls to master.
> For security concerns, regionservers might require making connection to a
> different proxy port of master rather than it's original port retrieved from
> Zookeeper.
> Configs:
> # hbase.master.expose.proxy.port: Master can use this config (int) to expose
> new proxy port on active and backup master znodes.
> # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can
> use this config (boolean) to determine whether to connect to active master on
> new proxy port that master has exposed or continue using original port of
> master for connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4606: HBASE-27183 Support regionserver to connect to HMaster proxy port
Apache-HBase commented on PR #4606: URL: https://github.com/apache/hbase/pull/4606#issuecomment-1179467751 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 1s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 20s | master passed | | +1 :green_heart: | compile | 3m 36s | master passed | | +1 :green_heart: | checkstyle | 0m 48s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 24s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 3m 31s | the patch passed | | +1 :green_heart: | cc | 3m 31s | the patch passed | | +1 :green_heart: | javac | 3m 31s | the patch passed | | -0 :warning: | checkstyle | 0m 31s | hbase-server: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | hbaseprotoc | 1m 17s | the patch passed | | -1 :x: | spotless | 0m 18s | patch has 64 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 4m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 42m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4606/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4606 | | Optional Tests | dupname asflicense cc hbaseprotoc spotless prototool javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4dfac6595373 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4606/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4606/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 69 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-zookeeper hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4606/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[
https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564499#comment-17564499
]
Viraj Jasani edited comment on HBASE-27183 at 7/9/22 3:06 AM:
--
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znodes but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port of master.
On the master host, "proxy port to original port" redirection is automatically
done, something outside the scope of HBase/Hadoop application layer.
cc [~apurtell]
was (Author: vjasani):
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znodes but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port, and on
the master host, proxy port to original port redirection is automatically done,
something outside the scope of HBase/Hadoop application layer.
> Support regionserver to connect to HMaster proxy port
> -
>
> Key: HBASE-27183
> URL: https://issues.apache.org/jira/browse/HBASE-27183
> Project: HBase
> Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> Regionservers get active master address from Zookeeper/Master registry and
> tries to make RPC calls to master.
> For security concerns, regionservers might require making connection to a
> different proxy port of master rather than it's original port retrieved from
> Zookeeper.
> Configs:
> # hbase.master.expose.proxy.port: Master can use this config (int) to expose
> new proxy port on active and backup master znodes.
> # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can
> use this config (boolean) to determine whether to connect to active master on
> new proxy port that master has exposed or continue using original port of
> master for connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[
https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564499#comment-17564499
]
Viraj Jasani commented on HBASE-27183:
--
{quote}the better way is to change the registry data on zookeeper, instead of
letting region server to do this static configured port change?
{quote}
This is covered with PR [https://github.com/apache/hbase/pull/4606]
Basically we let master continue exposing it's binding port but in addition to
that, also let it expose new proxy port for any client to connect to. So
masterAddress and backupAddress znodes both will have new proto field named
masterProxyPort. Now regionservers still continue using same ServerName object
retrieved/deserialized from znodes but with a new config, they can switch to
using new proxy port (only if master has exposed this port on znode in the
first place). Does this sound good to you [~zhangduo]?
{quote}If there is rule to block some ports due to security, then the correct
way is to not bind the master port in this blocked range?
{quote}
Valid question, no doubt. However, we have special encryption requirement where
a service should bind on one specific port but that port is not open for secure
communication. The secure channel is established on a new proxy port, and on
the master host, proxy port to original port redirection is automatically done,
something outside the scope of HBase/Hadoop application layer.
> Support regionserver to connect to HMaster proxy port
> -
>
> Key: HBASE-27183
> URL: https://issues.apache.org/jira/browse/HBASE-27183
> Project: HBase
> Issue Type: Improvement
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> Regionservers get active master address from Zookeeper/Master registry and
> tries to make RPC calls to master.
> For security concerns, regionservers might require making connection to a
> different proxy port of master rather than it's original port retrieved from
> Zookeeper.
> Configs:
> # hbase.master.expose.proxy.port: Master can use this config (int) to expose
> new proxy port on active and backup master znodes.
> # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can
> use this config (boolean) to determine whether to connect to active master on
> new proxy port that master has exposed or continue using original port of
> master for connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564497#comment-17564497 ] Duo Zhang commented on HBASE-27183: --- I do not feel this is the correct way to solve the problem. If there is rule to block some ports due to security, then the correct way is to not bind the master port in this blocked range? And what's more, even you want to hack into HBase, the better way is to change the registry data on zookeeper, instead of letting region server to do this static configured port change? Why not just register the proxy port on zookeeper? > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can > use this config (boolean) to determine whether to connect to active master on > new proxy port that master has exposed or continue using original port of > master for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-27183: - Description: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. Configs: # hbase.master.expose.proxy.port: Master can use this config (int) to expose new proxy port on active and backup master znodes. # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can use this config (boolean) to determine whether to connect to active master on new proxy port that master has exposed or continue using original port of master for connection. was: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. Configs: # hbase.master.expose.proxy.port: Master can use this config (int) to expose new proxy port on active and backup master znodes. # hbase.regionserver.consume.master.proxy.port: Clients can use this config (boolean) to determine whether to connect to active master on new proxy port that master has exposed or continue using original port of master for connection. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients/Regionservers can > use this config (boolean) to determine whether to connect to active master on > new proxy port that master has exposed or continue using original port of > master for connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-27183: - Description: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. Configs: # hbase.master.expose.proxy.port: Master can use this config (int) to expose new proxy port on active and backup master znodes. # hbase.regionserver.consume.master.proxy.port: Clients can use this config (boolean) to determine whether to connect to active master on new proxy port that master has exposed or continue using original port of master for connection. was: Regionservers get active master address from Zookeeper/Master registry and tries to make RPC calls to master. For security concerns, regionservers might require making connection to a different proxy port of master rather than it's original port retrieved from Zookeeper. We should support this case by introducing a new config. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. > Configs: > # hbase.master.expose.proxy.port: Master can use this config (int) to expose > new proxy port on active and backup master znodes. > # hbase.regionserver.consume.master.proxy.port: Clients can use this config > (boolean) to determine whether to connect to active master on new proxy port > that master has exposed or continue using original port of master for > connection. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed'
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564493#comment-17564493
]
Danny commented on HBASE-26042:
---
Is it valid？？
> WAL lockup on 'sync failed'
> ---
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5, 2.4.8
>Reporter: Michael Stack
>Priority: Major
> Attachments: HBASE-26042-test-repro.patch, debug-dump.txt,
> hbase-cvp-regionserver-cvp328.sjc.aristanetworks.com.log, js1, js2,
> regionserver-heap-live.hprof.gz, regionserver-threaddump.log
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next() @bci=2, line=105
> (Interpreted frame)
> -
[GitHub] [hbase] comnetwork merged pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
comnetwork merged PR #4595: URL: https://github.com/apache/hbase/pull/4595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] comnetwork commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
comnetwork commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1179462342 @bbeaudreault , thank you very much for help and review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27186) Report block cache size metrics separately for L1 and L2
Bryan Beaudreault created HBASE-27186: - Summary: Report block cache size metrics separately for L1 and L2 Key: HBASE-27186 URL: https://issues.apache.org/jira/browse/HBASE-27186 Project: HBase Issue Type: Improvement Reporter: Bryan Beaudreault Currently, in terms of block cache size metrics, we report the following: blockCacheFreeSize, blockCacheCount, blockCacheSize, blockCacheEvictionCount. This isn't super useful when running with BucketCache, because in that case L1 (holding index/meta blocks) and L2 (holding data blocks) operate largely independently. Additionally, you might typically report on block cache sizes in terms of monitoring heap usage. But with BucketCache, the cache is off-heap, so it becomes a misleading metric in that case. We have metrics l1CacheHitCount/MissCount/HitRatio/MissRatio, and the same for l2. We should add l1CacheFreeSize, l1CacheSize, l1CacheCount, l1CacheEvictionCount, and the same for l2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27078) Allow configuring a separate timeout for meta scans
[
https://issues.apache.org/jira/browse/HBASE-27078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564449#comment-17564449
]
Hudson commented on HBASE-27078:
Results for branch master
[build #629 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow configuring a separate timeout for meta scans
> ---
>
> Key: HBASE-27078
> URL: https://issues.apache.org/jira/browse/HBASE-27078
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> There is a {{hbase.client.meta.operation.timeout}} but it does not apply to
> meta scans, which are the primary use-case for clients (i.e. through
> RegionLocator).
> Many user-facing clients may want to have low rpc and scan timeouts. However,
> in periods of meta hotspotting, those timeouts can be way too low for the
> meta scans. The problem with low timeouts for meta scans is that without a
> populated MetaCache, user requests cannot succeed. In fact, user requests
> will continually try to re-scan meta until the MetaCache is populated. So
> having a lower rpc timeout will cause a situation where meta scans cannot
> succeed, and thus user requests cannot succeed. In this case I think it'd be
> preferable to relax the rpc timeout for meta requests so that a few long
> requests can unblock many faster requests.
> My suggestion would be to add an {{hbase.client.meta.rpc.timeout}} and ensure
> that it applies to meta scans. I also think it would be less confusing to
> have {{hbase.client.meta.operation.timeout}} apply as the scanner timeout
> period for meta scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27180) Multiple possible buffer leaks
[
https://issues.apache.org/jira/browse/HBASE-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564451#comment-17564451
]
Hudson commented on HBASE-27180:
Results for branch master
[build #629 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Multiple possible buffer leaks
> --
>
> Key: HBASE-27180
> URL: https://issues.apache.org/jira/browse/HBASE-27180
> Project: HBase
> Issue Type: Bug
> Components: netty, regionserver
>Reporter: Norman Maurer
>Assignee: Norman Maurer
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> When using ByteBuf you need to be very careful about releasing it as
> otherwise you might leak data. There are various places in the code-base
> where such a leak could happen as the buffer is not correctly released
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-18045) Add ' -o ConnectTimeout=10' to the ssh command we use in ITBLL chaos monkeys
[
https://issues.apache.org/jira/browse/HBASE-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564450#comment-17564450
]
Hudson commented on HBASE-18045:
Results for branch master
[build #629 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/629/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add ' -o ConnectTimeout=10' to the ssh command we use in ITBLL chaos monkeys
>
>
> Key: HBASE-18045
> URL: https://issues.apache.org/jira/browse/HBASE-18045
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Michael Stack
>Assignee: Narasimha Sharma
>Priority: Trivial
> Fix For: 3.0.0-alpha-4
>
>
> Monkeys hang on me in long running tests. I've not spent too much time on it
> since it rare enough but I just went through a spate of them. When monkey
> kill ssh hangs, all killing stops which can give a false sense of victory
> when you wake up in the morning and your job 'passed'. I also see monkeys
> kill all servers in a cluster and fail to bring them back which causes job
> fail as no one is serving data. The latter may actually be another issue but
> for the former, I've had some success adding -o ConnectTimeout=10 as an
> option on ssh. You can do it easily enough via config but this issue is to
> suggest that we add it in code.
> Here is how you add it via config if interested:
>
> hbase.it.clustermanager.ssh.opts
> -o ConnectTimeout=10
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27180) Multiple possible buffer leaks
[
https://issues.apache.org/jira/browse/HBASE-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564441#comment-17564441
]
Hudson commented on HBASE-27180:
Results for branch branch-2.5
[build #160 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Multiple possible buffer leaks
> --
>
> Key: HBASE-27180
> URL: https://issues.apache.org/jira/browse/HBASE-27180
> Project: HBase
> Issue Type: Bug
> Components: netty, regionserver
>Reporter: Norman Maurer
>Assignee: Norman Maurer
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> When using ByteBuf you need to be very careful about releasing it as
> otherwise you might leak data. There are various places in the code-base
> where such a leak could happen as the buffer is not correctly released
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27078) Allow configuring a separate timeout for meta scans
[
https://issues.apache.org/jira/browse/HBASE-27078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564440#comment-17564440
]
Hudson commented on HBASE-27078:
Results for branch branch-2.5
[build #160 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/160/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Allow configuring a separate timeout for meta scans
> ---
>
> Key: HBASE-27078
> URL: https://issues.apache.org/jira/browse/HBASE-27078
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> There is a {{hbase.client.meta.operation.timeout}} but it does not apply to
> meta scans, which are the primary use-case for clients (i.e. through
> RegionLocator).
> Many user-facing clients may want to have low rpc and scan timeouts. However,
> in periods of meta hotspotting, those timeouts can be way too low for the
> meta scans. The problem with low timeouts for meta scans is that without a
> populated MetaCache, user requests cannot succeed. In fact, user requests
> will continually try to re-scan meta until the MetaCache is populated. So
> having a lower rpc timeout will cause a situation where meta scans cannot
> succeed, and thus user requests cannot succeed. In this case I think it'd be
> preferable to relax the rpc timeout for meta requests so that a few long
> requests can unblock many faster requests.
> My suggestion would be to add an {{hbase.client.meta.rpc.timeout}} and ensure
> that it applies to meta scans. I also think it would be less confusing to
> have {{hbase.client.meta.operation.timeout}} apply as the scanner timeout
> period for meta scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4206: HBASE-26827 RegionServer JVM crash when compact mob table
Apache-HBase commented on PR #4206: URL: https://github.com/apache/hbase/pull/4206#issuecomment-1179187661 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 10s | the patch passed | | +1 :green_heart: | compile | 0m 53s | the patch passed | | +1 :green_heart: | javac | 0m 53s | the patch passed | | +1 :green_heart: | shadedjars | 4m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 226m 27s | hbase-server in the patch passed. | | | | 248m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4206 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ab938921ed74 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/testReport/ | | Max. process+thread count | 2626 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564353#comment-17564353 ] Viraj Jasani commented on HBASE-27183: -- New registry is required only if we are introducing new way of storing master and meta address. Here the hostname/ip address of active master remains same but only port is a proxy. But I think the better way is for master to publish it's own port by using new config. > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. We should support this case by introducing a new config. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] virajjasani closed pull request #4605: HBASE-27183 Support regionserver to connect to HMaster proxy port
virajjasani closed pull request #4605: HBASE-27183 Support regionserver to connect to HMaster proxy port URL: https://github.com/apache/hbase/pull/4605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4206: HBASE-26827 RegionServer JVM crash when compact mob table
Apache-HBase commented on PR #4206: URL: https://github.com/apache/hbase/pull/4206#issuecomment-1179163506 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 21s | master passed | | +1 :green_heart: | compile | 0m 33s | master passed | | +1 :green_heart: | shadedjars | 3m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 33s | the patch passed | | +1 :green_heart: | javac | 0m 33s | the patch passed | | +1 :green_heart: | shadedjars | 3m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 202m 56s | hbase-server in the patch passed. | | | | 218m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4206 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 201902bf7b22 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/testReport/ | | Max. process+thread count | 2158 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
Apache-HBase commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1179145920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 14s | branch-2 passed | | +1 :green_heart: | compile | 1m 16s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 18s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 4m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 43s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 51s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 203m 7s | hbase-server in the patch passed. | | | | 228m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4595 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux abdc961adcb2 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 6dd1b58481 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/testReport/ | | Max. process+thread count | 2437 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
Apache-HBase commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1179127329 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 47s | branch-2 passed | | +1 :green_heart: | compile | 1m 25s | branch-2 passed | | +1 :green_heart: | shadedjars | 3m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | shadedjars | 3m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 49s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 183m 43s | hbase-server in the patch passed. | | | | 209m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4595 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bec94c9f42e8 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 6dd1b58481 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/testReport/ | | Max. process+thread count | 2596 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-27185) Rewrite NettyRpcServer to decode rpc request with netty handler
Duo Zhang created HBASE-27185: - Summary: Rewrite NettyRpcServer to decode rpc request with netty handler Key: HBASE-27185 URL: https://issues.apache.org/jira/browse/HBASE-27185 Project: HBase Issue Type: Improvement Components: netty, rpc Reporter: Duo Zhang This is a follow on issue of HBASE-26708. The memory leaks happens in the saslReadAndProcess method. It does not follow a typical netty style so it is more likely to introduce bugs. Since we agree that NettyRpcServer is the future and SimpleRpcServer should be deprecated and finally removed, let's first implement dedicated rpc request decoder for netty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4206: HBASE-26827 RegionServer JVM crash when compact mob table
Apache-HBase commented on PR #4206: URL: https://github.com/apache/hbase/pull/4206#issuecomment-1178975077 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 30s | master passed | | +1 :green_heart: | compile | 2m 18s | master passed | | +1 :green_heart: | checkstyle | 0m 32s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 20s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 2m 16s | the patch passed | | +1 :green_heart: | javac | 2m 16s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 34s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 32m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4206 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 74c61516b1af 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4206/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
Apache-HBase commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1178949484 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 20s | branch-2 passed | | +1 :green_heart: | compile | 3m 35s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 58s | branch-2 passed | | +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 3m 35s | the patch passed | | +1 :green_heart: | javac | 3m 35s | the patch passed | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 16s | The patch passed checkstyle in hbase-client | | +1 :green_heart: | checkstyle | 0m 31s | hbase-server: The patch generated 0 new + 4 unchanged - 3 fixed = 4 total (was 7) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 7m 54s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 36m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4595 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux a08850b5efeb 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 6dd1b58481 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 64 (vs. ulimit of 12500) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4595/11/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26708) Netty "leak detected" and OutOfDirectMemoryError due to direct memory buffering with SASL implementation
[
https://issues.apache.org/jira/browse/HBASE-26708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564264#comment-17564264
]
Hudson commented on HBASE-26708:
Results for branch branch-2.4
[build #385 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Netty "leak detected" and OutOfDirectMemoryError due to direct memory
> buffering with SASL implementation
>
>
> Key: HBASE-26708
> URL: https://issues.apache.org/jira/browse/HBASE-26708
> Project: HBase
> Issue Type: Bug
> Components: netty, rpc, sasl
>Affects Versions: 2.5.0, 2.4.6
>Reporter: Viraj Jasani
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> Under constant data ingestion, using default Netty based RpcServer and
> RpcClient implementation results in OutOfDirectMemoryError, supposedly caused
> by leaks detected by Netty's LeakDetector.
> {code:java}
> 2022-01-25 17:03:10,084 ERROR [S-EventLoopGroup-1-3]
> util.ResourceLeakDetector - java:115)
>
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.expandCumulation(ByteToMessageDecoder.java:538)
>
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder$1.cumulate(ByteToMessageDecoder.java:97)
>
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:274)
>
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
>
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
>
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
>
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
>
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
>
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795)
>
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
>
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
>
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> java.lang.Thread.run(Thread.java:748)
> {code}
> {code:java}
> 2022-01-25 17:03:14,014 ERROR [S-EventLoopGroup-1-3]
> util.ResourceLeakDetector -
> apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507)
>
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446)
>
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
>
>
[jira] [Commented] (HBASE-27175) Failure to cleanup WAL split dir log should be at INFO level
[
https://issues.apache.org/jira/browse/HBASE-27175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564263#comment-17564263
]
Hudson commented on HBASE-27175:
Results for branch branch-2.4
[build #385 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/385/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Failure to cleanup WAL split dir log should be at INFO level
>
>
> Key: HBASE-27175
> URL: https://issues.apache.org/jira/browse/HBASE-27175
> Project: HBase
> Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Ujjawal Kumar
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> As part of the SCP, after we are done splitting WALs, we try removing the
> -splitting dirs but if the dir doesn't exist, dfs#delete fails with IOE.
> Since we are aware of this case, we just log the message but we don't
> interrupt SCP because handling of the failure is gracefully done.
> Hence, failure to remove "-splitting" dir should not be mentioned as WARN
> log, let's convert this to INFO level:
> {code:java}
> LOG.warn("Remove WAL directory for {} failed, ignore...{}", serverName,
> e.getMessage()); {code}
>
> Any other genuine failure to remove the splitting dir is anyways covered by
> this log, hence we don't have worry about keeping the above mentioned log at
> WARN level, we have anyways mentioned "ignore..." in the log message.
> {code:java}
> if (!fs.delete(splitDir, false)) {
> LOG.warn("Failed delete {}, contains {}", splitDir, fs.listFiles(splitDir,
> true));
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] bbeaudreault commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
bbeaudreault commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1178909069 Thanks! I'll check in again later today once the pre-commit hooks finish -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] comnetwork commented on pull request #4595: HBASE-26950 Use AsyncConnection in ReplicationSink
comnetwork commented on PR #4595: URL: https://github.com/apache/hbase/pull/4595#issuecomment-1178908212 @bbeaudreault , thank you very much for review, ok ,I have rebase my PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4605: HBASE-27183 Support regionserver to connect to HMaster proxy port
Apache-HBase commented on PR #4605: URL: https://github.com/apache/hbase/pull/4605#issuecomment-1178836224 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 2s | master passed | | +1 :green_heart: | compile | 0m 46s | master passed | | +1 :green_heart: | shadedjars | 4m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 235m 28s | hbase-server in the patch failed. | | | | 255m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4605 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b73de6719675 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/testReport/ | | Max. process+thread count | 2712 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27148) Move minimum hadoop 3 support version to 3.2.3
[ https://issues.apache.org/jira/browse/HBASE-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27148: -- Release Note: Bump the minimum hadoop 3 dependency to 3.2.3. Also upgrade apache-avro to 1.11.0 and exclude all jackson 1.x dependencies since all jackson 1.x versions have vulnerabilities. Notice that for hadoop 2 dependency we will need to include jackson 1.x because hadoop directly depend on it. > Move minimum hadoop 3 support version to 3.2.3 > --- > > Key: HBASE-27148 > URL: https://issues.apache.org/jira/browse/HBASE-27148 > Project: HBase > Issue Type: Task > Components: dependencies, hadoop3, security >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-4 > > > Seems the hadoop community will not make newer 3.1.x release so let's move > the minimun hadoop 3 versions to 3.2.3, due to security issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27148) Move minimum hadoop 3 support version to 3.2.3
[ https://issues.apache.org/jira/browse/HBASE-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27148. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks [~sunxin] for reviewing! > Move minimum hadoop 3 support version to 3.2.3 > --- > > Key: HBASE-27148 > URL: https://issues.apache.org/jira/browse/HBASE-27148 > Project: HBase > Issue Type: Task > Components: dependencies, hadoop3, security >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-4 > > > Seems the hadoop community will not make newer 3.1.x release so let's move > the minimun hadoop 3 versions to 3.2.3, due to security issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4605: HBASE-27183 Support regionserver to connect to HMaster proxy port
Apache-HBase commented on PR #4605: URL: https://github.com/apache/hbase/pull/4605#issuecomment-1178798201 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | master passed | | +1 :green_heart: | compile | 0m 46s | master passed | | +1 :green_heart: | shadedjars | 3m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | shadedjars | 3m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 200m 19s | hbase-server in the patch passed. | | | | 217m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4605 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c25125652300 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/testReport/ | | Max. process+thread count | 2883 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27184) When Hbase uses GET to query data, some rowkeys cannot be queried，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2
[ https://issues.apache.org/jira/browse/HBASE-27184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564193#comment-17564193 ] yangzhichao commented on HBASE-27184: - The problem is solved. The main problem is that the Datanode host is stuck. You need to restart the host, and this error does not seem to be related to hbase > When Hbase uses GET to query data, some rowkeys cannot be queried，ERROR: > java.io.TOException: java.lang.NoclassDefFoundError: > org/apache/hadoop/hdfs/DFSInputStreamS2 > - > > Key: HBASE-27184 > URL: https://issues.apache.org/jira/browse/HBASE-27184 > Project: HBase > Issue Type: Bug >Reporter: yangzhichao >Priority: Major > Attachments: image-2022-07-08-16-59-52-808.png > > > !image-2022-07-08-16-59-52-808.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27184) When Hbase uses GET to query data, some rowkeys cannot be queried，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2
[ https://issues.apache.org/jira/browse/HBASE-27184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangzhichao updated HBASE-27184: Summary: When Hbase uses GET to query data, some rowkeys cannot be queried，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2 (was: Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2) > When Hbase uses GET to query data, some rowkeys cannot be queried，ERROR: > java.io.TOException: java.lang.NoclassDefFoundError: > org/apache/hadoop/hdfs/DFSInputStreamS2 > - > > Key: HBASE-27184 > URL: https://issues.apache.org/jira/browse/HBASE-27184 > Project: HBase > Issue Type: Bug >Reporter: yangzhichao >Priority: Major > Attachments: image-2022-07-08-16-59-52-808.png > > > !image-2022-07-08-16-59-52-808.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27184) Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2
[ https://issues.apache.org/jira/browse/HBASE-27184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564180#comment-17564180 ] Zhuoyue Huang commented on HBASE-27184: --- [~yangzhichao] You should submit this issue again in English so that developers from all over the world can read your description. You also need to add more information such as hbase version you are using and the steps to reproduce the problem. (您应该用英文再次提交此问题，以便来自世界各地的开发人员可以阅读您的描述。 您还需要添加更多信息，例如 hbase 版本和重现问题的步骤。) > Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: > java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2 > - > > Key: HBASE-27184 > URL: https://issues.apache.org/jira/browse/HBASE-27184 > Project: HBase > Issue Type: Bug >Reporter: yangzhichao >Priority: Major > Attachments: image-2022-07-08-16-59-52-808.png > > > !image-2022-07-08-16-59-52-808.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27184) Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2
yangzhichao created HBASE-27184: --- Summary: Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2 Key: HBASE-27184 URL: https://issues.apache.org/jira/browse/HBASE-27184 Project: HBase Issue Type: Bug Reporter: yangzhichao Attachments: image-2022-07-08-16-59-52-808.png !image-2022-07-08-16-59-52-808.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 merged pull request #4599: HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561)
Apache9 merged PR #4599: URL: https://github.com/apache/hbase/pull/4599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4599: HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561)
Apache9 commented on PR #4599: URL: https://github.com/apache/hbase/pull/4599#issuecomment-1178675880 Tried the two failed UTs locally, all passed. Let me merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4604: HBASE-27149 Server should close scanner if client times out before results are ready
Apache-HBase commented on PR #4604: URL: https://github.com/apache/hbase/pull/4604#issuecomment-1178656146 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 2s | master passed | | +1 :green_heart: | compile | 0m 51s | master passed | | +1 :green_heart: | shadedjars | 3m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 54s | the patch passed | | +1 :green_heart: | compile | 0m 50s | the patch passed | | +1 :green_heart: | javac | 0m 50s | the patch passed | | +1 :green_heart: | shadedjars | 3m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 214m 34s | hbase-server in the patch passed. | | | | 233m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4604 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6e6adc438ecf 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/testReport/ | | Max. process+thread count | 2526 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4604: HBASE-27149 Server should close scanner if client times out before results are ready
Apache-HBase commented on PR #4604: URL: https://github.com/apache/hbase/pull/4604#issuecomment-1178647670 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 26s | master passed | | +1 :green_heart: | compile | 0m 39s | master passed | | +1 :green_heart: | shadedjars | 3m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 15s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 207m 17s | hbase-server in the patch passed. | | | | 223m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4604 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 25c40a3fda46 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/testReport/ | | Max. process+thread count | 2872 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4604/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4605: HBASE-27183 Support regionserver to connect to HMaster proxy port
Apache-HBase commented on PR #4605: URL: https://github.com/apache/hbase/pull/4605#issuecomment-1178626561 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 38s | master passed | | +1 :green_heart: | compile | 2m 23s | master passed | | +1 :green_heart: | checkstyle | 0m 30s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 24s | the patch passed | | +1 :green_heart: | compile | 2m 21s | the patch passed | | +1 :green_heart: | javac | 2m 21s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotless | 0m 35s | patch has 24 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 32m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4605 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 12800344daf3 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 2197b3806b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4605/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-27169) TestSeparateClientZKCluster is flaky
[
https://issues.apache.org/jira/browse/HBASE-27169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-27169.
---
Hadoop Flags: Reviewed
Resolution: Fixed
It is much more stable than before but still have some failure runs.
https://ci-hbase.apache.org/job/HBase-Flaky-Tests/job/master/3818/testReport/junit/org.apache.hadoop.hbase.client/TestSeparateClientZKCluster/testMetaMoveDuringClientZkClusterRestart/
Will open other issues for fixing.
> TestSeparateClientZKCluster is flaky
>
>
> Key: HBASE-27169
> URL: https://issues.apache.org/jira/browse/HBASE-27169
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4, 2.4.14
>
>
> https://nightlies.apache.org/hbase/HBase-Flaky-Tests/master/3773/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestSeparateClientZKCluster-output.txt
> {noformat}
> org.apache.hadoop.hbase.exceptions.MasterStoppedException: null
> at
> org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:3177)
> ~[classes/:?]
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1954)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:743)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> ~[hbase-protocol-shaded-3.0.0-alpha-4-SNAPSHOT.jar:3.0.0-alpha-4-SNAPSHOT]
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:385)
> ~[classes/:?]
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> ~[classes/:?]
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:104)
> ~[classes/:?]
> at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:84)
> ~[classes/:?]
> {noformat}
> I think the problem is that, MasterStoppedException is a sub class of
> DoNotRetryIOException, so when hitting this issue, we will fail immediately.
> And the client zk syncer is asynchoronous, so it is possible that when we
> call admin.balance, we haven't synced the new location yet, and it will throw
> the MasterStoppedException out soon and fail the UT.
> Let me see how to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27183) Support regionserver to connect to HMaster proxy port
[ https://issues.apache.org/jira/browse/HBASE-27183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564115#comment-17564115 ] Duo Zhang commented on HBASE-27183: --- I think we can implement a special registry? > Support regionserver to connect to HMaster proxy port > - > > Key: HBASE-27183 > URL: https://issues.apache.org/jira/browse/HBASE-27183 > Project: HBase > Issue Type: Improvement >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0-alpha-4 > > > Regionservers get active master address from Zookeeper/Master registry and > tries to make RPC calls to master. > For security concerns, regionservers might require making connection to a > different proxy port of master rather than it's original port retrieved from > Zookeeper. We should support this case by introducing a new config. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4599: HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561)
Apache-HBase commented on PR #4599: URL: https://github.com/apache/hbase/pull/4599#issuecomment-1178600709 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 52s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 44s | branch-2 passed | | +1 :green_heart: | compile | 1m 50s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 41s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 24s | the patch passed | | +1 :green_heart: | compile | 1m 41s | the patch passed | | +1 :green_heart: | javac | 1m 41s | the patch passed | | +1 :green_heart: | shadedjars | 4m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 313m 35s | root in the patch failed. | | | | 340m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4599/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4599 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5e6fb27b0f2b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 92525fb869 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4599/6/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4599/6/testReport/ | | Max. process+thread count | 2140 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-testing-util . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4599/6/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
