[GitHub] [hbase] YutSean commented on pull request #3933: HBASE-26557 log4j2 has a critical RCE vulnerability
YutSean commented on pull request #3933: URL: https://github.com/apache/hbase/pull/3933#issuecomment-990665163 Added the parameter to the start script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Wang resolved HBASE-26027.
Resolution: Fixed
> The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone
> caused by ArrayStoreException
> -
>
> Key: HBASE-26027
> URL: https://issues.apache.org/jira/browse/HBASE-26027
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.7, 2.5.0, 2.3.5, 2.4.4
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 2.6.0
>
>
> The batch api of HTable contains a param named results to store result or
> exception, its type is Object[].
> If user pass an array with other type, eg:
> org.apache.hadoop.hbase.client.Result, and if we need to put an exception
> into it by some reason, then the ArrayStoreException will occur in
> AsyncRequestFutureImpl.updateResult, then the
> AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
> AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
> actionsInProgress again and again, forever.
> It is better to add an cutoff calculated by operationTimeout, instead of only
> depend on the value of actionsInProgress.
> BTW, this issue only for 2.x, since 3.x the implement has refactored.
> How to reproduce:
> 1: add sleep in RSRpcServices.multi to mock slow response
> {code:java}
> try {
> Thread.sleep(2000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> {code}
> 2: set time out in config
> {code:java}
> conf.set("hbase.rpc.timeout","2000");
> conf.set("hbase.client.operation.timeout","6000");
> {code}
> 3: call batch api
> {code:java}
> Table table = HbaseUtil.getTable("test");
> byte[] cf = Bytes.toBytes("f");
> byte[] c = Bytes.toBytes("c1");
> List gets = new ArrayList<>();
> for (int i = 0; i < 10; i++) {
> byte[] rk = Bytes.toBytes("rk-" + i);
> Get get = new Get(rk);
> get.addColumn(cf, c);
> gets.add(get);
> }
> Result[] results = new Result[gets.size()];
> table.batch(gets, results);
> {code}
> The log will looks like below:
> {code:java}
> [ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 -
> id=1 error for test processing localhost,16020,1624343786295
> java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> [INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to
> finish on table:
> [INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to
> finish on table: test
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456906#comment-17456906
]
Zheng Wang commented on HBASE-26027:
Merged to branch-2, thanks for the review. [~apurtell]
> The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone
> caused by ArrayStoreException
> -
>
> Key: HBASE-26027
> URL: https://issues.apache.org/jira/browse/HBASE-26027
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.7, 2.5.0, 2.3.5, 2.4.4
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 2.6.0
>
>
> The batch api of HTable contains a param named results to store result or
> exception, its type is Object[].
> If user pass an array with other type, eg:
> org.apache.hadoop.hbase.client.Result, and if we need to put an exception
> into it by some reason, then the ArrayStoreException will occur in
> AsyncRequestFutureImpl.updateResult, then the
> AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
> AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
> actionsInProgress again and again, forever.
> It is better to add an cutoff calculated by operationTimeout, instead of only
> depend on the value of actionsInProgress.
> BTW, this issue only for 2.x, since 3.x the implement has refactored.
> How to reproduce:
> 1: add sleep in RSRpcServices.multi to mock slow response
> {code:java}
> try {
> Thread.sleep(2000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> {code}
> 2: set time out in config
> {code:java}
> conf.set("hbase.rpc.timeout","2000");
> conf.set("hbase.client.operation.timeout","6000");
> {code}
> 3: call batch api
> {code:java}
> Table table = HbaseUtil.getTable("test");
> byte[] cf = Bytes.toBytes("f");
> byte[] c = Bytes.toBytes("c1");
> List gets = new ArrayList<>();
> for (int i = 0; i < 10; i++) {
> byte[] rk = Bytes.toBytes("rk-" + i);
> Get get = new Get(rk);
> get.addColumn(cf, c);
> gets.add(get);
> }
> Result[] results = new Result[gets.size()];
> table.batch(gets, results);
> {code}
> The log will looks like below:
> {code:java}
> [ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 -
> id=1 error for test processing localhost,16020,1624343786295
> java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> [INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to
> finish on table:
> [INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to
> finish on table: test
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] bsglz merged pull request #3925: HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
bsglz merged pull request #3925: URL: https://github.com/apache/hbase/pull/3925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-990651138 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 34s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 10m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 3s | the patch passed | | +1 :green_heart: | compile | 1m 40s | the patch passed | | +1 :green_heart: | javac | 1m 40s | the patch passed | | +1 :green_heart: | shadedjars | 10m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 225m 5s | hbase-server in the patch passed. | | | | 267m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux df4c473af150 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/testReport/ | | Max. process+thread count | 3343 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3933: HBASE-26557 log4j2 has a critical RCE vulnerability
Apache9 commented on pull request #3933: URL: https://github.com/apache/hbase/pull/3933#issuecomment-990633692 And I suggest that we add -Dlog4j2.formatMsgNoLookups=true in the start scripts to disable JNDI completely, we do not need this feature in HBase, typically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] taklwu commented on a change in pull request #3932: HBASE-26556 IT and Chaos Monkey improvements
taklwu commented on a change in pull request #3932:
URL: https://github.com/apache/hbase/pull/3932#discussion_r766348175
##
File path:
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/ConfigurableSlowDeterministicMonkeyFactory.java
##
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.chaos.factories;
+
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.chaos.actions.Action;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.Constructor;
+import java.util.function.Function;
+
+public class ConfigurableSlowDeterministicMonkeyFactory extends
SlowDeterministicMonkeyFactory {
+
+ private static final Logger LOG =
+LoggerFactory.getLogger(ConfigurableSlowDeterministicMonkeyFactory.class);
+
+ final static String HEAVY_ACTIONS = "heavy.actions";
+ final static String TABLE_PARAM = "\\$table_name";
+
+ public enum SupportedTypes {
+FLOAT(p->Float.parseFloat(p)),
+LONG(p-> Long.parseLong(p)),
+INT(p-> Integer.parseInt(p)),
+TABLENAME(p-> TableName.valueOf(p));
+
+final Function converter;
+
+SupportedTypes(Function converter){
+ this.converter = converter;
+}
+
+Object convert(String param){
+ return converter.apply(param);
+}
+ }
+
+ @Override
+ protected Action[] getHeavyWeightedActions() {
+String actions = this.properties.getProperty(HEAVY_ACTIONS);
+if( actions==null||actions.isEmpty() ){
Review comment:
[nit] `javac` and `checkstyle` seems failing, please see if you can fix
it.
##
File path:
hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
##
@@ -322,7 +323,10 @@ protected CommandProvider getCommandProvider(ServiceType
service) throws IOExcep
case ZOOKEEPER_SERVER:
return new ZookeeperShellCommandProvider(getConf());
default:
-return new HBaseShellCommandProvider(getConf());
+Class provider = getConf()
+ .getClass("hbase.it.clustermanager.hbase.command.provider",
Review comment:
[not a blocker][no action required] I found many inline property keys in
this class, but you don't need to change it
now. but at some point in the future, we may need to move all these
`hbase.it.clustermanager.*` to a single place, e.g.
`HBaseItClusterMangerConstants`
##
File path:
hbase-it/src/test/java/org/apache/hadoop/hbase/HBaseClusterManager.java
##
@@ -216,7 +217,7 @@ protected String findPidCommand(ServiceType service) {
}
public String signalCommand(ServiceType service, String signal) {
- return String.format("%s | xargs kill -s %s", findPidCommand(service),
signal);
+ return String.format("%s | xargs sudo kill -s %s",
findPidCommand(service), signal);
Review comment:
nice change, this help as long as the user is not the root user, and
normally we have the hbase user when using Chaomonkey.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3933: HBASE-26557 log4j2 has a critical RCE vulnerability
Apache-HBase commented on pull request #3933: URL: https://github.com/apache/hbase/pull/3933#issuecomment-990626527 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 57s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 29s | master passed | | +1 :green_heart: | compile | 10m 16s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 21s | the patch passed | | +1 :green_heart: | compile | 9m 26s | the patch passed | | +1 :green_heart: | javac | 9m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 21m 41s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 61m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3933/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3933 | | Optional Tests | dupname asflicense javac hadoopcheck xml compile | | uname | Linux c105ed694fdd 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3933/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26557) log4j2 has a critical RCE vulnerability
[ https://issues.apache.org/jira/browse/HBASE-26557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456886#comment-17456886 ] Yutong Xiao commented on HBASE-26557: - The vulnerability should be related with this commit: https://github.com/apache/logging-log4j2/commit/7fe72d6 > log4j2 has a critical RCE vulnerability > --- > > Key: HBASE-26557 > URL: https://issues.apache.org/jira/browse/HBASE-26557 > Project: HBase > Issue Type: Bug >Reporter: Yutong Xiao >Assignee: Yutong Xiao >Priority: Major > > Impacted log4j version: Apache Log4j 2.x <= 2.14.1 > I found that our current log4j version at master is 2.14.1. > Should upgrade the version to 2.15.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26557) log4j2 has a critical RCE vulnerability
[ https://issues.apache.org/jira/browse/HBASE-26557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456882#comment-17456882 ] Anoop Sam John commented on HBASE-26557: Can u link vulnerability details also here for completeness [~xytss123] > log4j2 has a critical RCE vulnerability > --- > > Key: HBASE-26557 > URL: https://issues.apache.org/jira/browse/HBASE-26557 > Project: HBase > Issue Type: Bug >Reporter: Yutong Xiao >Assignee: Yutong Xiao >Priority: Major > > Impacted log4j version: Apache Log4j 2.x <= 2.14.1 > I found that our current log4j version at master is 2.14.1. > Should upgrade the version to 2.15.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-990617470 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 8m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 8m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 149m 54s | hbase-server in the patch passed. | | | | 181m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 243dda4972d5 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/testReport/ | | Max. process+thread count | 5350 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] YutSean opened a new pull request #3933: HBASE-26557 log4j2 has a critical RCE vulnerability
YutSean opened a new pull request #3933: URL: https://github.com/apache/hbase/pull/3933 https://issues.apache.org/jira/browse/HBASE-26557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26557) log4j2 has a critical RCE vulnerability
[ https://issues.apache.org/jira/browse/HBASE-26557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yutong Xiao updated HBASE-26557: Description: Impacted log4j version: Apache Log4j 2.x <= 2.14.1 I found that our current log4j version at master is 2.14.1. Should upgrade the version to 2.15.0 > log4j2 has a critical RCE vulnerability > --- > > Key: HBASE-26557 > URL: https://issues.apache.org/jira/browse/HBASE-26557 > Project: HBase > Issue Type: Bug >Reporter: Yutong Xiao >Assignee: Yutong Xiao >Priority: Major > > Impacted log4j version: Apache Log4j 2.x <= 2.14.1 > I found that our current log4j version at master is 2.14.1. > Should upgrade the version to 2.15.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26557) log4j2 has a critical RCE vulnerability
Yutong Xiao created HBASE-26557: --- Summary: log4j2 has a critical RCE vulnerability Key: HBASE-26557 URL: https://issues.apache.org/jira/browse/HBASE-26557 Project: HBase Issue Type: Bug Reporter: Yutong Xiao Assignee: Yutong Xiao -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3931: HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto
Apache-HBase commented on pull request #3931: URL: https://github.com/apache/hbase/pull/3931#issuecomment-990577998 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 3s | branch-2 passed | | +1 :green_heart: | compile | 3m 21s | branch-2 passed | | +1 :green_heart: | shadedjars | 9m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 5s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | +1 :green_heart: | compile | 2m 57s | the patch passed | | +1 :green_heart: | javac | 2m 57s | the patch passed | | +1 :green_heart: | shadedjars | 8m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 54s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 3m 8s | hbase-client in the patch passed. | | -1 :x: | unit | 226m 52s | hbase-server in the patch failed. | | | | 276m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3931 | | JIRA Issue | HBASE-26537 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux adee66540a83 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / b1bc5f3a5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/testReport/ | | Max. process+thread count | 2562 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3931: HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto
Apache-HBase commented on pull request #3931: URL: https://github.com/apache/hbase/pull/3931#issuecomment-990566590 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 59s | branch-2 passed | | +1 :green_heart: | compile | 2m 42s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 44s | the patch passed | | +1 :green_heart: | compile | 2m 43s | the patch passed | | +1 :green_heart: | javac | 2m 43s | the patch passed | | +1 :green_heart: | shadedjars | 7m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 57s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 3m 6s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 208m 44s | hbase-server in the patch passed. | | | | 251m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3931 | | JIRA Issue | HBASE-26537 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d61107980e9f 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / b1bc5f3a5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/testReport/ | | Max. process+thread count | 3040 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-990565142 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 40s | master passed | | +1 :green_heart: | compile | 4m 21s | master passed | | +1 :green_heart: | checkstyle | 1m 24s | master passed | | +1 :green_heart: | spotbugs | 3m 3s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 20s | the patch passed | | +1 :green_heart: | compile | 4m 6s | the patch passed | | +1 :green_heart: | javac | 4m 6s | the patch passed | | -0 :warning: | checkstyle | 1m 21s | hbase-server: The patch generated 5 new + 20 unchanged - 1 fixed = 25 total (was 21) | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 25m 43s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 65m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux cc22869df0d1 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3916: HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto
Apache-HBase commented on pull request #3916: URL: https://github.com/apache/hbase/pull/3916#issuecomment-990561768 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 8s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 1s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 26s | branch-1 passed | | +1 :green_heart: | compile | 1m 33s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 1m 44s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 5m 27s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 13s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 21s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 2m 58s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 7m 5s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 1m 33s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | cc | 1m 33s | the patch passed | | +1 :green_heart: | javac | 1m 33s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | cc | 1m 43s | the patch passed | | +1 :green_heart: | javac | 1m 43s | the patch passed | | -1 :x: | checkstyle | 0m 37s | hbase-client: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16) | | -1 :x: | checkstyle | 1m 41s | hbase-server: The patch generated 3 new + 18 unchanged - 0 fixed = 21 total (was 18) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 26s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 30s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | -1 :x: | hbaseprotoc | 0m 11s | hbase-protocol in the patch failed. | | -1 :x: | hbaseprotoc | 0m 40s | hbase-server in the patch failed. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 22s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 7m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 29s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 2m 41s | hbase-client in the patch passed. | | -1 :x: | unit | 154m 41s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 10s | The patch does not generate ASF License warnings. | | | | 235m 46s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3916/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3916 | | JIRA Issue | HBASE-26537 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux a907f9db2b25 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3916/out/precommit/personality/provided.sh | | git revision | branch-1 / 8d96fc3614 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 |
[GitHub] [hbase] Apache-HBase commented on pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
Apache-HBase commented on pull request #3243: URL: https://github.com/apache/hbase/pull/3243#issuecomment-990560594 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 34s | master passed | | +1 :green_heart: | compile | 3m 4s | master passed | | +1 :green_heart: | shadedjars | 9m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 34s | the patch passed | | +1 :green_heart: | compile | 3m 2s | the patch passed | | +1 :green_heart: | javac | 3m 2s | the patch passed | | +1 :green_heart: | shadedjars | 9m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 399m 31s | root in the patch failed. | | | | 446m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3243 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b8c0ed3a5e7e 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/testReport/ | | Max. process+thread count | 3331 (vs. ulimit of 3) | | modules | C: hbase-http hbase-it hbase-rest . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26370) Unexpected crash makes client receive error exception
[
https://issues.apache.org/jira/browse/HBASE-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26370:
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client disabled the table {{mytable}};
3. {{mytable}} is in disabled state;
4. Client request to truncate {{mytable}};
5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
{{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
and mark the table as Enabling;
6. HMaster1 crashes;
7. HMaster2 becomes active;
8. Client gets info that the truncate operation failed due to
{{TableNotDisabledException}}, while the table has already be truncated.
In HMaster2, we got following info:
{code:java}
INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback procedure
TruncateTableProcedure (table=mytable preserveSplits=true) id=9 owner=myhbase
state=ROLLEDBACK exec-time=4.6030 sec
exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
{code}
It seems that the client receives such error message due to HMaster's recovery
process is recovering the truncate table process. But for client, it should not
get this error message caused by HMaster failover. Similarly, for create table,
disable table request, client can get java.util.concurrent.TimeoutException
caused by HMaster failover.
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client disabled the table {{mytable}};
3. {{mytable}} is in disabled state;
4. Client request to truncate {{mytable}};
5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
{{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
and mark the table as Enabling;
6. HMaster1 crashes;
7. HMaster2 becomes active;
8. Client gets info that the truncate operation failed due to
{{TableNotDisabledException}}, while the table has already be truncated.
In HMaster2, we got following info:
{code:java}
INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback procedure
TruncateTableProcedure (table=mytable preserveSplits=true) id=9 owner=myhbase
state=ROLLEDBACK exec-time=4.6030 sec
exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
{code}
It sees that the client receives such error message due to HMaster's recovery
process is recovering the truncate table process. But for client, it should not
get this error message caused by HMaster failover. Similarly, for create table,
disable table request, client can get java.util.concurrent.TimeoutException
caused by HMaster failover.
> Unexpected crash makes client receive error exception
> -
>
> Key: HBASE-26370
> URL: https://issues.apache.org/jira/browse/HBASE-26370
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.1, 2.4.8
>Reporter: May
>Priority: Major
>
> There is a table {{mytable}} in a HBase cluster with two HMasters and two
> RegionServers.
> 1. Current active HMaster is HMaster1;
> 2. Client disabled the table {{mytable}};
> 3. {{mytable}} is in disabled state;
> 4. Client request to truncate {{mytable}};
> 5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
> {{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
> and mark the table as Enabling;
> 6. HMaster1 crashes;
> 7. HMaster2 becomes active;
> 8. Client gets info that the truncate operation failed due to
> {{TableNotDisabledException}}, while the table has already be truncated.
> In HMaster2, we got following info:
> {code:java}
> INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback
> procedure TruncateTableProcedure (table=mytable preserveSplits=true) id=9
> owner=myhbase state=ROLLEDBACK exec-time=4.6030 sec
> exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
> {code}
> It seems that the client receives such error message due to HMaster's
> recovery process is recovering the truncate table process. But for client, it
> should not get this error message caused by HMaster failover. Similarly, for
> create table, disable table request, client can get
> java.util.concurrent.TimeoutException caused by HMaster failover.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26370) Unexpected crash makes client receive error exception
[
https://issues.apache.org/jira/browse/HBASE-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26370:
Description:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client disabled the table {{mytable}};
3. {{mytable}} is in disabled state;
4. Client request to truncate {{mytable}};
5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
{{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
and mark the table as Enabling;
6. HMaster1 crashes;
7. HMaster2 becomes active;
8. Client gets info that the truncate operation failed due to
{{TableNotDisabledException}}, while the table has already be truncated.
In HMaster2, we got following info:
{code:java}
INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback procedure
TruncateTableProcedure (table=mytable preserveSplits=true) id=9 owner=myhbase
state=ROLLEDBACK exec-time=4.6030 sec
exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
{code}
It sees that the client receives such error message due to HMaster's recovery
process is recovering the truncate table process. But for client, it should not
get this error message caused by HMaster failover. Similarly, for create table,
disable table request, client can get java.util.concurrent.TimeoutException
caused by HMaster failover.
was:
There is a table {{mytable}} in a HBase cluster with two HMasters and two
RegionServers.
1. Current active HMaster is HMaster1;
2. Client disabled the table {{mytable}};
3. {{mytable}} is in disabled state;
4. Client request to truncate {{mytable}};
5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
{{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
and mark the table as Enabling;
6. HMaster1 crashes;
7. HMaster2 becomes active;
8. Client gets info that the truncate operation failed due to
{{TableNotDisabledException}}, while the table has already be truncated.
In HMaster2, we got following info:
{code:java}
INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback procedure
TruncateTableProcedure (table=mytable preserveSplits=true) id=9 owner=myhbase
state=ROLLEDBACK exec-time=4.6030 sec
exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
{code}
> Unexpected crash makes client receive error exception
> -
>
> Key: HBASE-26370
> URL: https://issues.apache.org/jira/browse/HBASE-26370
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.1, 2.4.8
>Reporter: May
>Priority: Major
>
> There is a table {{mytable}} in a HBase cluster with two HMasters and two
> RegionServers.
> 1. Current active HMaster is HMaster1;
> 2. Client disabled the table {{mytable}};
> 3. {{mytable}} is in disabled state;
> 4. Client request to truncate {{mytable}};
> 5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
> {{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
> and mark the table as Enabling;
> 6. HMaster1 crashes;
> 7. HMaster2 becomes active;
> 8. Client gets info that the truncate operation failed due to
> {{TableNotDisabledException}}, while the table has already be truncated.
> In HMaster2, we got following info:
> {code:java}
> INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback
> procedure TruncateTableProcedure (table=mytable preserveSplits=true) id=9
> owner=myhbase state=ROLLEDBACK exec-time=4.6030 sec
> exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
> {code}
> It sees that the client receives such error message due to HMaster's recovery
> process is recovering the truncate table process. But for client, it should
> not get this error message caused by HMaster failover. Similarly, for create
> table, disable table request, client can get
> java.util.concurrent.TimeoutException caused by HMaster failover.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26370) Unexpected crash makes client receive error exception
[
https://issues.apache.org/jira/browse/HBASE-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
May updated HBASE-26370:
Affects Version/s: 2.4.8
> Unexpected crash makes client receive error exception
> -
>
> Key: HBASE-26370
> URL: https://issues.apache.org/jira/browse/HBASE-26370
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.1, 2.4.8
>Reporter: May
>Priority: Major
>
> There is a table {{mytable}} in a HBase cluster with two HMasters and two
> RegionServers.
> 1. Current active HMaster is HMaster1;
> 2. Client disabled the table {{mytable}};
> 3. {{mytable}} is in disabled state;
> 4. Client request to truncate {{mytable}};
> 5. HMaster1 tries to truncate {{mytable}}. For the TruncateTableState
> {{TRUNCATE_TABLE_ASSIGN_REGIONS}}, HMaster1 assigns the regions of the table
> and mark the table as Enabling;
> 6. HMaster1 crashes;
> 7. HMaster2 becomes active;
> 8. Client gets info that the truncate operation failed due to
> {{TableNotDisabledException}}, while the table has already be truncated.
> In HMaster2, we got following info:
> {code:java}
> INFO [ProcedureExecutor-12] procedure2.ProcedureExecutor: Rolledback
> procedure TruncateTableProcedure (table=mytable preserveSplits=true) id=9
> owner=myhbase state=ROLLEDBACK exec-time=4.6030 sec
> exception=org.apache.hadoop.hbase.TableNotDisabledException: mytable
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] YutSean commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
YutSean commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-990532276 Reconstructed the code and extracted the RpcHandler to avoid the redundant coding of fastpath handler. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3932: HBASE-26556 IT and Chaos Monkey improvements
Apache-HBase commented on pull request #3932: URL: https://github.com/apache/hbase/pull/3932#issuecomment-990476293 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 13s | master passed | | +1 :green_heart: | compile | 0m 49s | master passed | | +1 :green_heart: | checkstyle | 0m 22s | master passed | | +1 :green_heart: | spotbugs | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 43s | the patch passed | | +1 :green_heart: | compile | 0m 50s | the patch passed | | -0 :warning: | javac | 0m 50s | hbase-it generated 2 new + 101 unchanged - 2 fixed = 103 total (was 103) | | -0 :warning: | checkstyle | 0m 18s | hbase-it: The patch generated 3 new + 19 unchanged - 24 fixed = 22 total (was 43) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 47s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 50m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3932 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 67f2b61e44f1 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-it.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3932: HBASE-26556 IT and Chaos Monkey improvements
Apache-HBase commented on pull request #3932: URL: https://github.com/apache/hbase/pull/3932#issuecomment-990463924 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 56s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 0m 28s | master passed | | +1 :green_heart: | shadedjars | 8m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 20s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 0s | the patch passed | | +1 :green_heart: | compile | 0m 29s | the patch passed | | -0 :warning: | javac | 0m 29s | hbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shadedjars | 8m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 59s | hbase-it in the patch passed. | | | | 33m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3932 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c4b07340dcd5 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk8-hadoop3-check/output/diff-compile-javac-hbase-it.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/testReport/ | | Max. process+thread count | 689 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3932: HBASE-26556 IT and Chaos Monkey improvements
Apache-HBase commented on pull request #3932: URL: https://github.com/apache/hbase/pull/3932#issuecomment-990461645 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 44s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | shadedjars | 8m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | -0 :warning: | javac | 0m 32s | hbase-it generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 56s | hbase-it in the patch passed. | | | | 30m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3932 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 84c14a394c1a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/artifact/yetus-jdk11-hadoop3-check/output/diff-compile-javac-hbase-it.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/testReport/ | | Max. process+thread count | 698 (vs. ulimit of 3) | | modules | C: hbase-it U: hbase-it | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3932/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26548) Investigate mTLS in RPC layer
[ https://issues.apache.org/jira/browse/HBASE-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456829#comment-17456829 ] Wellington Chevreuil commented on HBASE-26548: -- [~bbeaudreault] , I had just attached the patch we mentioned over slack thread. This was originally created from some point of branch-2.2, but I don't think it will apply cleanly to any current head of branches. As already mentioned over slack, this had modified netty rpc related client/server classes to implement one way TLS in the rpc connections. It's not pluggable, so if this applied, one way TLS will be always on. I mentioned in the slack channel that it was using the test-only purpose *InsecureTrustManagerFactory* to build an SSL context, but whilst reviewing this patch today, I realised this is actually using the default JDK truststore. When testing it, I had used server self signed cert, so needed to import the server certificate into jdk default truststore, so that client can validate the server certificate. Since this is one-way, client certificate is not required. Let me know how it goes, I may be able to give you more details if you have further questions. > Investigate mTLS in RPC layer > - > > Key: HBASE-26548 > URL: https://issues.apache.org/jira/browse/HBASE-26548 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Priority: Major > Attachments: 0001-One-way-TLS-on-Netty-RPC-Implementation.patch > > > Current authentication options are heavily based on SASL and Kerberos. For > organizations that don't already deploy Kerberos or other token provider, > this is a heavy lift. Another very common way of authenticating in the > industry is mTLS, which makes use of SSL certifications and can solve both > wire encryption and auth. For those already deploying trusted certificates in > their infra, mTLS may be much easier to integrate. > It isn't necessarily easy to implement this, but I do think we could use > existing Netty SSL support in the NettyRpcClient and NettyRpcServer. I know > it's easy to add SSL to non-blocking IO through a > hadoop.rpc.socket.factory.class.default which returns SSLSockets, but that > doesn't touch on the certification verification at all. > Much more investigation is needed, but logging this due to some interest > encountered on slack. > Slack thread: > https://apache-hbase.slack.com/archives/C13K8NVAM/p1638980520110600 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26548) Investigate mTLS in RPC layer
[ https://issues.apache.org/jira/browse/HBASE-26548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-26548: - Attachment: 0001-One-way-TLS-on-Netty-RPC-Implementation.patch > Investigate mTLS in RPC layer > - > > Key: HBASE-26548 > URL: https://issues.apache.org/jira/browse/HBASE-26548 > Project: HBase > Issue Type: New Feature >Reporter: Bryan Beaudreault >Priority: Major > Attachments: 0001-One-way-TLS-on-Netty-RPC-Implementation.patch > > > Current authentication options are heavily based on SASL and Kerberos. For > organizations that don't already deploy Kerberos or other token provider, > this is a heavy lift. Another very common way of authenticating in the > industry is mTLS, which makes use of SSL certifications and can solve both > wire encryption and auth. For those already deploying trusted certificates in > their infra, mTLS may be much easier to integrate. > It isn't necessarily easy to implement this, but I do think we could use > existing Netty SSL support in the NettyRpcClient and NettyRpcServer. I know > it's easy to add SSL to non-blocking IO through a > hadoop.rpc.socket.factory.class.default which returns SSLSockets, but that > doesn't touch on the certification verification at all. > Much more investigation is needed, but logging this due to some interest > encountered on slack. > Slack thread: > https://apache-hbase.slack.com/archives/C13K8NVAM/p1638980520110600 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
Apache-HBase commented on pull request #3243: URL: https://github.com/apache/hbase/pull/3243#issuecomment-990437496 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 11s | master passed | | +1 :green_heart: | compile | 4m 1s | master passed | | +1 :green_heart: | shadedjars | 10m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 42s | the patch passed | | +1 :green_heart: | compile | 3m 54s | the patch passed | | +1 :green_heart: | javac | 3m 54s | the patch passed | | +1 :green_heart: | shadedjars | 10m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 38s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 241m 12s | root in the patch failed. | | | | 295m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3243 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 23247d66e084 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/testReport/ | | Max. process+thread count | 3566 (vs. ulimit of 3) | | modules | C: hbase-http hbase-it hbase-rest . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26556) IT and Chaos Monkey improvements
Wellington Chevreuil created HBASE-26556: Summary: IT and Chaos Monkey improvements Key: HBASE-26556 URL: https://issues.apache.org/jira/browse/HBASE-26556 Project: HBase Issue Type: Improvement Components: integration tests Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil While running the IntegrationTestIngest with slowDeterministic chaos monkey in our environment, we faced some issues with some of the chaos actions that required run some linux shell commands, due to the way we set hbase environment variables and start/stop scripts. This patch allows for defining extensions of HBaseShellCommandProvider in the configuration, so that custom commands can be used. It also extends SlowDeterministicMonkeyFactory to provide the ability to configure the destructive actions to be executed. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase-filesystem] wchevreuil merged pull request #31: HBASE-26513 [HBOSS] Update readme to reflect dependencies change intr…
wchevreuil merged pull request #31: URL: https://github.com/apache/hbase-filesystem/pull/31 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26521) Name RPC spans as `$package.$service/$method`
[ https://issues.apache.org/jira/browse/HBASE-26521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-26521: - Status: Patch Available (was: Open) > Name RPC spans as `$package.$service/$method` > - > > Key: HBASE-26521 > URL: https://issues.apache.org/jira/browse/HBASE-26521 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > As per > https://github.com/open-telemetry/opentelemetry-specification/blob/3e380e2/specification/trace/semantic_conventions/rpc.md#span-name -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3931: HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto
Apache-HBase commented on pull request #3931: URL: https://github.com/apache/hbase/pull/3931#issuecomment-990413664 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 56s | branch-2 passed | | +1 :green_heart: | compile | 5m 57s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 0s | branch-2 passed | | +1 :green_heart: | spotbugs | 6m 29s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 5m 34s | the patch passed | | +1 :green_heart: | cc | 5m 34s | the patch passed | | +1 :green_heart: | javac | 5m 34s | the patch passed | | -0 :warning: | checkstyle | 0m 38s | hbase-client: The patch generated 1 new + 19 unchanged - 0 fixed = 20 total (was 19) | | -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 3 new + 13 unchanged - 0 fixed = 16 total (was 13) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 3s | the patch passed | | +1 :green_heart: | spotbugs | 7m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. | | | | 62m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3931 | | JIRA Issue | HBASE-26537 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux a7c275c9ba8f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / b1bc5f3a5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3931/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456812#comment-17456812
]
Nick Dimiduk commented on HBASE-25865:
--
bq. Thinking of the Jersey and JAX-RS related changes in this regard, their
potential consequences for anyone who may have implemented custom servlets.
It's true that people who are relying on these ancient versions of jersey and
the JSRs via transitive dependency will be affected. In a [slightly different
context|https://hbase.apache.org/book.html#hbase.versioning], we say that a
client application MAY require recompilation when upgrading to a new minor
version, which means they have opportunity to discover the issue at compile
time.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456809#comment-17456809 ] Bryan Beaudreault commented on HBASE-26537: --- Sounds good. Since I know you're working on 2.5.0 now, I went ahead and ported to branch-2. PR is here: https://github.com/apache/hbase/pull/3931 > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 2.5.0, 1.8.0, 2.6.0 > > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26555) Fix findbugs/spotbugs findings
[ https://issues.apache.org/jira/browse/HBASE-26555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reassigned HBASE-26555: --- Assignee: Andrew Kyle Purtell > Fix findbugs/spotbugs findings > -- > > Key: HBASE-26555 > URL: https://issues.apache.org/jira/browse/HBASE-26555 > Project: HBase > Issue Type: Task >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.8, 2.4.9, 2.6.0 > > > Clean up findbugs/spotbugs findings on branch HEADs (master, branch-2, > branch-2.5, branch-2.4, branch-2.3) > For example, from recent precommits: > Possible null pointer dereference in > org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) > due to return value of called method Dereferenced at > MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) > due to return value of called method Dereferenced at > MetaTableMetrics.java:[line 230] > Possible null pointer dereference in > org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) > due to return value of called method Dereferenced at > MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) > due to return value of called method Dereferenced at > MetaTableMetrics.java:[line 210] > Possible null pointer dereference in > org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due > to return value of called method Method invoked at > LruCachedBlockQueue.java:org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) > due to return value of called method Method invoked at > LruCachedBlockQueue.java:[line 73] > Possible null pointer dereference in > org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due > to return value of called method Dereferenced at > CachedEntryQueue.java:org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) > due to return value of called method Dereferenced at > CachedEntryQueue.java:[line 79] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26555) Fix findbugs/spotbugs findings
Andrew Kyle Purtell created HBASE-26555: --- Summary: Fix findbugs/spotbugs findings Key: HBASE-26555 URL: https://issues.apache.org/jira/browse/HBASE-26555 Project: HBase Issue Type: Task Reporter: Andrew Kyle Purtell Fix For: 2.5.0, 3.0.0-alpha-2, 2.3.8, 2.4.9, 2.6.0 Clean up findbugs/spotbugs findings on branch HEADs (master, branch-2, branch-2.5, branch-2.4, branch-2.3) For example, from recent precommits: Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 230] Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 210] Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:[line 73] Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:[line 79] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26537) FuzzyRowFilter backwards compatibility
[
https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456710#comment-17456710
]
Andrew Kyle Purtell edited comment on HBASE-26537 at 12/9/21, 10:06 PM:
I see, it's as you said to Duo
{quote}
The implementation of FuzzyRowFilter in 1.1.5, 1.2.2, 1.3.0, 2.0.0 is not
broken – just the ability to upgrade from a previous release to one of those
releases.
{quote}
however asking someone to upgrade from 1.2.0 to 1.8.0 is going to have other
implications, it is such a large jump. Would just opt for an upgrade to 2.x,
personally. It is still worth making this change, though, I will concede that.
was (Author: apurtell):
I see, it's as you said to Duo
{quote}
The implementation of FuzzyRowFilter in 1.1.5, 1.2.2, 1.3.0, 2.0.0 is not
broken – just the ability to upgrade from a previous release to one of those
releases.
{quote}
however asking someone to upgrade from 1.2.0 to 1.8.0 is going to have other
implications, it is such a large jump. It is still worth making this change,
though, I will concede that.
> FuzzyRowFilter backwards compatibility
> --
>
> Key: HBASE-26537
> URL: https://issues.apache.org/jira/browse/HBASE-26537
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.13, 1.2.12, 1.3.6
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.0, 1.8.0, 2.6.0
>
>
> HBASE-15676 introduced a backwards incompatible change which makes it
> impossible to upgrade in our designated upgraded order (server first, then
> client) without potential bad results. Worse, the failure mode is silent – a
> pre-HBASE-15676 client would incorrectly receive 0 results from a
> post-HBASE-15676 server.
> I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding
> a new proto field to switch between the two implementations. I'm submitting
> this Jira to capture and potentially backport that fix for anyone else who
> encounters it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456757#comment-17456757 ] Andrew Kyle Purtell edited comment on HBASE-26537 at 12/9/21, 10:05 PM: bq. If you guys think I should commit this to branch-2 as well I think we do, if the intent is to support server-first upgrade scenarios with this filter in use. There are going to be other issues and errata going from older 1.x to 2.5.0+ but this won't be one. bq. I think we can all agree to leave it out of master. Agreed. Set fixVersions accordingly. was (Author: apurtell): bq. If you guys think I should commit this to branch-2 as well I think we do, if the intent is to support server-first upgrade scenarios with this filter in use. bq. I think we can all agree to leave it out of master. Agreed. Set fixVersions accordingly. > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 2.5.0, 1.8.0, 2.6.0 > > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456757#comment-17456757 ] Andrew Kyle Purtell commented on HBASE-26537: - bq. If you guys think I should commit this to branch-2 as well I think we do, if the intent is to support server-first upgrade scenarios with this filter in use. bq. I think we can all agree to leave it out of master. Agreed. Set fixVersions accordingly. > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 2.5.0, 1.8.0, 2.6.0 > > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-26537: Fix Version/s: 2.5.0 1.8.0 2.6.0 > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 2.5.0, 1.8.0, 2.6.0 > > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26122) Limit max result size of individual Gets
[ https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456754#comment-17456754 ] Andrew Kyle Purtell commented on HBASE-26122: - Thanks for the follow up [~bbeaudreault], most appreciated. > Limit max result size of individual Gets > > > Key: HBASE-26122 > URL: https://issues.apache.org/jira/browse/HBASE-26122 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > Scans have the ability to have a configured max result size, which causes > them to return a partial result once the limit has been reached. MultiGets > also can throw MultiActionResultTooLarge if the response size is over a > configured quota. Neither of these really accounts for a single Get of a > too-large row. Such too-large Gets can cause substantial GC pressure or worse > if sent at volume. > Currently one can work around this by converting their Get to a single row > Scan, but this requires a developer to proactively know about and prepare for > the issue by using a Scan upfront or wait for the RegionServer to choke on a > large request and only then rewrite the Get for future requests. > We should implement the same response size limits for for Get as for Scan, > whereby the server returns a partial result to the client for handling. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456751#comment-17456751
]
Andrew Kyle Purtell edited comment on HBASE-25865 at 12/9/21, 10:00 PM:
bq. do you intend to update the hbase-thirdparty dependency in hbase-2.5.0 to a
new 4.0.x release?
[~ndimiduk] I don't have a strong opinion either way, although the minor
increment is a good opportunity to move from 3.x to 4.x thirdparty given more
tolerance for incompatible changes per our guidelines. Thinking of the Jersey
and JAX-RS related changes in this regard, their potential consequences for
anyone who may have implemented custom servlets.
was (Author: apurtell):
bq. do you intend to update the hbase-thirdparty dependency in hbase-2.5.0 to a
new 4.0.x release?
[~ndimiduk] I don't have a strong opinion either way, although the minor
increment is a good opportunity to move from 3.x to 4.x thirdparty given more
tolerance for incompatible changes per our guidelines.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26122) Limit max result size of individual Gets
[ https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456750#comment-17456750 ] Bryan Beaudreault commented on HBASE-26122: --- [~apurtell] I just did another verification. This was only merged into branch-2, where I have confirmed it was reverted. So this has been reverted everywhere. I'm actually going to close this issue as we decided to utilize hbase.table.max.rowsize instead. We can re-open if there is interest in this feature again. > Limit max result size of individual Gets > > > Key: HBASE-26122 > URL: https://issues.apache.org/jira/browse/HBASE-26122 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 3.0.0-alpha-2, 2.6.0 > > > Scans have the ability to have a configured max result size, which causes > them to return a partial result once the limit has been reached. MultiGets > also can throw MultiActionResultTooLarge if the response size is over a > configured quota. Neither of these really accounts for a single Get of a > too-large row. Such too-large Gets can cause substantial GC pressure or worse > if sent at volume. > Currently one can work around this by converting their Get to a single row > Scan, but this requires a developer to proactively know about and prepare for > the issue by using a Scan upfront or wait for the RegionServer to choke on a > large request and only then rewrite the Get for future requests. > We should implement the same response size limits for for Get as for Scan, > whereby the server returns a partial result to the client for handling. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26122) Limit max result size of individual Gets
[
https://issues.apache.org/jira/browse/HBASE-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault resolved HBASE-26122.
---
Fix Version/s: (was: 3.0.0-alpha-2)
(was: 2.6.0)
Release Note: (was: Can now call Get.setMaxResultSize(). When set to a
positive value, the server will return the results when that threshold is met.
This may result in partial results for large rows, so the caller is expected to
handle the case where Result#mayHaveMoreCellsInRow() is true when
setMaxResultSize is used. Possible options include paginating using PageFilter,
reducing the returned data set using other filters, converting the Get to a
Scan (which can take advantage of partial response stitching), or throwing a
non-retryable exception if using this as a guardrail. See below for example
usage in shell:
Create table
hbase:005:0> create 't1', 'f1'
Created table t1
Took 1.1306 seconds
Insert test data
hbase:012:0> put 't1', 'r1', 'f1:c1', 'a'
Took 0.0416 seconds
hbase:014:0> put 't1', 'r1', 'f1:c2', 'b'
Took 0.0059 seconds
hbase:015:0> put 't1', 'r1', 'f1:c3', 'c'
Took 0.0097 seconds
Get without setMaxResultSize, returns full row and mayHaveMoreCellsInRow = false
hbase:037:0> g = Get.new('r1'.to_s.to_java_bytes)
=> #
hbase:038:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)
=> #
hbase:039:0> result.mayHaveMoreCellsInRow
=> false
hbase:040:0> result.toString
=> "keyvalues={r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0,
r1/f1:c3/1627498280413/Put/vlen=1/seqid=0}"
Get with setMaxResultSize, returns first two columns and mayHaveMoreCellsInRow
= true
hbase:059:0> g = Get.new('r1'.to_s.to_java_bytes).setMaxResultSize(100)
=> #
hbase:060:0> result = @hbase.table('t1',
@shell).instance_variable_get(:@table).get(g)
=> #
hbase:061:0> result.mayHaveMoreCellsInRow
=> true
hbase:062:0> result.toString
=> "keyvalues={r1/f1:c1/1627498270850/Put/vlen=1/seqid=0,
r1/f1:c2/1627498276326/Put/vlen=1/seqid=0}")
Resolution: Won't Fix
> Limit max result size of individual Gets
>
>
> Key: HBASE-26122
> URL: https://issues.apache.org/jira/browse/HBASE-26122
> Project: HBase
> Issue Type: New Feature
> Components: Client, regionserver
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> Scans have the ability to have a configured max result size, which causes
> them to return a partial result once the limit has been reached. MultiGets
> also can throw MultiActionResultTooLarge if the response size is over a
> configured quota. Neither of these really accounts for a single Get of a
> too-large row. Such too-large Gets can cause substantial GC pressure or worse
> if sent at volume.
> Currently one can work around this by converting their Get to a single row
> Scan, but this requires a developer to proactively know about and prepare for
> the issue by using a Scan upfront or wait for the RegionServer to choke on a
> large request and only then rewrite the Get for future requests.
> We should implement the same response size limits for for Get as for Scan,
> whereby the server returns a partial result to the client for handling.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456751#comment-17456751
]
Andrew Kyle Purtell commented on HBASE-25865:
-
bq. do you intend to update the hbase-thirdparty dependency in hbase-2.5.0 to a
new 4.0.x release?
[~ndimiduk] I don't have a strong opinion either way, although the minor
increment is a good opportunity to move from 3.x to 4.x thirdparty given more
tolerance for incompatible changes per our guidelines.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456740#comment-17456740
]
Nick Dimiduk commented on HBASE-25865:
--
Not entirely. I was delayed waiting for the hbase-thirdparty 4.0.0 release, and
then I shifted my attention to the OpenTelemetry verification. Now that
thirdparty is nearly here, I'd like to return to this and land the one chart
that I have.
[~apurtell] do you intend to update the hbase-thirdparty dependency in
hbase-2.5.0 to a new 4.0.x release? If no, we can drop this from 2.5.0.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
> Attachments: image1.png, image2.png
>
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
Apache-HBase commented on pull request #3243: URL: https://github.com/apache/hbase/pull/3243#issuecomment-990242364 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 9m 0s | master passed | | +1 :green_heart: | checkstyle | 2m 1s | master passed | | +1 :green_heart: | spotbugs | 11m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 8m 52s | the patch passed | | +1 :green_heart: | javac | 8m 52s | the patch passed | | +1 :green_heart: | checkstyle | 2m 2s | root: The patch generated 0 new + 202 unchanged - 16 fixed = 202 total (was 218) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 6s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 21m 31s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | -1 :x: | spotbugs | 10m 24s | root generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | | | | 87m 54s | | | Reason | Tests | |---:|:--| | FindBugs | module:root | | | Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 230] | | | Possible null pointer dereference in org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.opWithClientMeterName(Object) due to return value of called method Dereferenced at MetaTableMetrics.java:[line 210] | | | Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:org.apache.hadoop.hbase.io.hfile.LruCachedBlockQueue.add(LruCachedBlock) due to return value of called method Method invoked at LruCachedBlockQueue.java:[line 73] | | | Possible null pointer dereference in org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:org.apache.hadoop.hbase.io.hfile.bucket.CachedEntryQueue.add(Map$Entry) due to return value of called method Dereferenced at CachedEntryQueue.java:[line 79] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3243 | | Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle | | uname | Linux ae7da42e3ad4 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/artifact/yetus-general-check/output/new-spotbugs-root.html | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: hbase-http hbase-it hbase-rest . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456715#comment-17456715 ] Bryan Beaudreault commented on HBASE-26537: --- Yea I agree – I wouldn't do it myself, instead I'd go to 2.x. We're in a weird spot because these affected releases are so old. If you guys think I should commit this to branch-2 as well, I can give it a shot. It seems a little unfortunate, but not the end of the world. I think we can all agree to leave it out of master. The other thing to note is, I made the most expedient change here, because I needed to quickly get past this issue and onto other things. There may be a better way. Unfortunately I can't commit to looking into that in the near future, but wanted to provide this in case it helped anyone. We could leave this Jira open for a bit with the attached patch and if I have time in the future I can try a different, cleaner approach. If someone else wants to take it over that's also ok. > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[
https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456710#comment-17456710
]
Andrew Kyle Purtell commented on HBASE-26537:
-
I see, it's as you said to Duo
{quote}
The implementation of FuzzyRowFilter in 1.1.5, 1.2.2, 1.3.0, 2.0.0 is not
broken – just the ability to upgrade from a previous release to one of those
releases.
{quote}
however asking someone to upgrade from 1.2.0 to 1.8.0 is going to have other
implications, it is such a large jump. It is still worth making this change,
though, I will concede that.
> FuzzyRowFilter backwards compatibility
> --
>
> Key: HBASE-26537
> URL: https://issues.apache.org/jira/browse/HBASE-26537
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.13, 1.2.12, 1.3.6
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> HBASE-15676 introduced a backwards incompatible change which makes it
> impossible to upgrade in our designated upgraded order (server first, then
> client) without potential bad results. Worse, the failure mode is silent – a
> pre-HBASE-15676 client would incorrectly receive 0 results from a
> post-HBASE-15676 server.
> I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding
> a new proto field to switch between the two implementations. I'm submitting
> this Jira to capture and potentially backport that fix for anyone else who
> encounters it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reassigned HBASE-26537: --- Assignee: Bryan Beaudreault > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456709#comment-17456709 ] Bryan Beaudreault edited comment on HBASE-26537 at 12/9/21, 8:03 PM: - [~apurtell] Sorry I missed your comment at first. I think upgrading from 1.2.0 to 1.2.2 would have been (and still is) a breaking change. You can't follow the usual path of upgrading server first, then client, because the 0 values sent by the 1.2.0 client would be mis-interpreted by the 1.2.2 server. Similar is true if you tried upgrading client first, the 1.2.0 server would not know what to do with the new '2' values sent by 1.2.2 client. I think if we had caught this post-merge back in 2016 I'm guessing we would have pushed a 1.2.3 or 1.2.2.1 and tried to dissuade upgrading to 1.2.2 was (Author: bbeaudreault): [~apurtell] Sorry I missed your comment at first. I think upgrading from 1.2.0 to 1.2.2 would have been breaking change. You can't follow the usual path of upgrading server first, then client, because the 0 values sent by the 1.2.0 client would be mis-interpreted by the 1.2.2 server. Similar is true if you tried upgrading client first, the 1.2.0 server would not know what to do with the new '2' values sent by 1.2.2 client. I think if we had caught this post-merge back in 2016 I'm guessing we would have pushed a 1.2.3 or 1.2.2.1 and tried to dissuade upgrading to 1.2.2 > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456709#comment-17456709 ] Bryan Beaudreault edited comment on HBASE-26537 at 12/9/21, 8:02 PM: - [~apurtell] Sorry I missed your comment at first. I think upgrading from 1.2.0 to 1.2.2 would have been breaking change. You can't follow the usual path of upgrading server first, then client, because the 0 values sent by the 1.2.0 client would be mis-interpreted by the 1.2.2 server. Similar is true if you tried upgrading client first, the 1.2.0 server would not know what to do with the new '2' values sent by 1.2.2 client. I think if we had caught this post-merge back in 2016 I'm guessing we would have pushed a 1.2.3 or 1.2.2.1 and tried to dissuade upgrading to 1.2.2 was (Author: bbeaudreault): [~apurtell] Sorry I missed your comment at first. I think upgrading from 1.2.0 to 1.2.2 would have been breaking change. You can't follow the usual path of upgrading server first, then client, because the 0 values sent by the client would be mis-interpreted by the server. Similar is true if you tried upgrading client first, the server would not know what to do with the new '2' values sent by client. I think if we had caught this post-merge back in 2016 I'm guessing we would have pushed a 1.2.3 or 1.2.2.1 and tried to dissuade upgrading to 1.2.2 > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456709#comment-17456709 ] Bryan Beaudreault commented on HBASE-26537: --- [~apurtell] Sorry I missed your comment at first. I think upgrading from 1.2.0 to 1.2.2 would have been breaking change. You can't follow the usual path of upgrading server first, then client, because the 0 values sent by the client would be mis-interpreted by the server. Similar is true if you tried upgrading client first, the server would not know what to do with the new '2' values sent by client. I think if we had caught this post-merge back in 2016 I'm guessing we would have pushed a 1.2.3 or 1.2.2.1 and tried to dissuade upgrading to 1.2.2 > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456707#comment-17456707 ] Bryan Beaudreault commented on HBASE-26537: --- [~zhangduo] the reason I decided to only commit this to branch-1 is as follows: * HBASE-15676 has been integrated for almost 6 years, and landed in the 2.0.0 release * As such, anyone running 2.0.0 or higher will not be affected by this bug, only those running a 1.x prior to that Jira – so they must be running a minor version over 5 years old. * The implementation of FuzzyRowFilter in 1.1.5, 1.2.2, 1.3.0, 2.0.0 is not broken – just the ability to upgrade from a previous release to one of those releases. * 1.x line is EOL I believe (or almost EOL), but definitely the 1.x releases implicated in the bug are EOL. I.E. < 1.1.5 and < 1.2.2 * This fix is sort of ugly – it will immortalize the old mask format from pre-15676 and a new proto field to switch between them. * Since all releases in our currently supported versions already use the new mask format, it seems best not to pollute these branches with this compatibility shim Correct me if I'm wrong on this, but let's rewind back to April 2016. If we had caught this bug before merging, we would have required the shim for branch-1 but allowed the cleaner breaking change in branch-2. So the intent of submitting this patch is to make it possible for those users running very old EOL releases to have an upgrade path. The idea is they'd do this: * Upgrade to a version with this patch – either backport it to themselves, or upgrade to HEAD of branch-1 or 1.8.0 if that gets released. ** Upgrade server first, then client * From there they can upgrade to any 2.x release or 3.x release. Does this sound reasonable? > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26522) Improve documentation of hbase 1.x to 2.x potential incompatibilities
[ https://issues.apache.org/jira/browse/HBASE-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456706#comment-17456706 ] Andrew Kyle Purtell commented on HBASE-26522: - Not sure HBASE-26537 is a real issue, see that JIRA for discussion. > Improve documentation of hbase 1.x to 2.x potential incompatibilities > - > > Key: HBASE-26522 > URL: https://issues.apache.org/jira/browse/HBASE-26522 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > > We're working on a major upgrade of almost 900 tables across 100 production > clusters (and corresponding QA environment clusters). We've upgraded about > 25% of our QA environment and run into a series of incompatibilities along > the way. Most of them have been easy to get around, but I wanted to create > this Jira to collect them so that we can make an update to the docs for > future upgraders. > My plan is to periodically edit this description to add to the list. If > anyone else has anything to contribute, feel free to edit as well or add a > comment. > Incompatibilities to document: > - HBASE-15676 changed the serialized byte string used for the fuzzy mask. > FuzzyRowFilters created by older clients will not match any rows in an hbase2 > cluster. This was fixed in HBASE-26537 but should be documented in our > upgrade guide. > - CDH5 try/catches bad HTableDescriptor.getDurability calls and returns > USE_DEFAULT. In hbase2, if someone creates a table with a bad durability > (i.e. DEFAULT instead of USE_DEFAULT), it results in a failure which causes > the CreateTableProcedure to infinitely retries with no backoff. This rapid > retry caused a bunch of pain on the cluster that encountered it, backing up > datanode's ability to keep up with the millions of calls to create and delete > .regioninfo files. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456704#comment-17456704 ] Andrew Kyle Purtell edited comment on HBASE-26537 at 12/9/21, 7:56 PM: --- HBASE-15676 was committed to 1.2.2. Would it have been possible to solve this by upgrading from 1.2.0 to 1.2.2 or later 1.2.x i.e if you had kept up to date with the patch releases of the version you were on there would have been no issue? Shouldn't that be the recommended path? was (Author: apurtell): HBASE-15676 was committed to 1.2.2. Would it have been possible to solve this by upgrading from 1.2.0 to 1.2.2 or later? Shouldn't that be the recommended path? > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26537) FuzzyRowFilter backwards compatibility
[ https://issues.apache.org/jira/browse/HBASE-26537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456704#comment-17456704 ] Andrew Kyle Purtell commented on HBASE-26537: - HBASE-15676 was committed to 1.2.2. Would it have been possible to solve this by upgrading from 1.2.0 to 1.2.2 or later? Shouldn't that be the recommended path? > FuzzyRowFilter backwards compatibility > -- > > Key: HBASE-26537 > URL: https://issues.apache.org/jira/browse/HBASE-26537 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.13, 1.2.12, 1.3.6 >Reporter: Bryan Beaudreault >Priority: Major > > HBASE-15676 introduced a backwards incompatible change which makes it > impossible to upgrade in our designated upgraded order (server first, then > client) without potential bad results. Worse, the failure mode is silent – a > pre-HBASE-15676 client would incorrectly receive 0 results from a > post-HBASE-15676 server. > I solved this internally as part of our upgrade from 1.2.0 to 2.4.6 by adding > a new proto field to switch between the two implementations. I'm submitting > this Jira to capture and potentially backport that fix for anyone else who > encounters it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26469) HBase shell has changed exit behavior
[
https://issues.apache.org/jira/browse/HBASE-26469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456680#comment-17456680
]
Sean Busbey commented on HBASE-26469:
-
are you good with this approach?
* Match current patch results on master, branch-2, branch-2.5 (presuming before
2.5.0 goes out)
* update branch-2.4 to match branch-2.3 behavior
if so I'll get updated patches put up. I think testing branches other than 2.4
will be blocked on HBASE-26543
> HBase shell has changed exit behavior
> -
>
> Key: HBASE-26469
> URL: https://issues.apache.org/jira/browse/HBASE-26469
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.8
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.5.0, 2.4.9, 2.6.0
>
> Attachments: hbase-1.4.14-exit-behavior.log,
> hbase-1.7.1-exit-behavior.log, hbase-2.0.6-exit-behavior.log,
> hbase-2.1.9-exit-behavior.log, hbase-2.2.7-exit-behavior.log,
> hbase-2.3.7-exit-behavior.log, hbase-2.4.8-exit-behavior.log,
> hbase-3.0.0-alpha-2-exit-behavior.log
>
>
> The HBase shell has changed behavior in a way that breaks being able to exit
> properly.
> Two example scripts to act as stand ins for hbase shell scripts to "do
> something simple then exit":
> {code}
> tmp % echo "list\nexit" > clean_exit.rb
> tmp % echo "list\nexit 1" > error_exit.rb
> {code}
> Giving these two scripts is possible:
> * passed as a cli argument
> * via redirected stdin
> Additionally the shell invocation can be:
> * in the default compatibility mode
> * with the "non interactive" flag that gives an exit code that reflects
> runtime errors
> I'll post logs of the details as attachments but here are some tables of the
> exit codes.
> The {{clean_exit.rb}} invocations ought to exit with success, exit code 0.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8
> || master ||
> | cli, default |0 |0 |0 |0 |0 |0 |1 |
>1* |
> | cli, -n | 0 |0 |0 |0 |0 |0 |1 |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
>0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
>1* |
> The {{error_exit.rb}} invocation should return a non-zero exit code, unless
> we're specifically trying to match a normal hbase shell session.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 ||
> master ||
> | cli, default | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> | cli, -n |1 |1 |1 |1 |1 |1 |1* |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
> 0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> In cases marked with * the error details are different.
> The biggest concern are the new-to-2.4 non-zero exit code when we should have
> a success and the hanging.
> The former looks like this:
> {code}
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> {code}
> The change in error details for the error exit script also shows this same
> detail.
> This behavior appears to be a side effect of HBASE-11686. As far as I can
> tell, the IRB handling of 'exit' calls fail because we implement our own
> handling of sessoins rather than rely on the intended session interface. We
> never set a current session, and IRB's exit implementation presumes there
> will be one.
> Running in debug shows this in a stacktrace:
> {code}
> Took 0.4563 seconds
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> NoMethodError: private method `exit' called for nil:NilClass
> irb_exit at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb:30
> evaluate at stdin:2
> eval at org/jruby/RubyKernel.java:1048
> evaluate at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85
> eval_io at uri:classloader:/shell.rb:327
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:246
> loop at org/jruby/RubyKernel.java:1442
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:232
> catch at org/jruby/RubyKernel.java:1189
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:231
> eval_io at uri:classloader:/shell.rb:326
> classpath:/jar-bootstrap.rb at
[jira] [Commented] (HBASE-26522) Improve documentation of hbase 1.x to 2.x potential incompatibilities
[ https://issues.apache.org/jira/browse/HBASE-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456678#comment-17456678 ] Viraj Jasani commented on HBASE-26522: -- Thanks for starting this [~bbeaudreault]. FYI [~apurtell] > Improve documentation of hbase 1.x to 2.x potential incompatibilities > - > > Key: HBASE-26522 > URL: https://issues.apache.org/jira/browse/HBASE-26522 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > > We're working on a major upgrade of almost 900 tables across 100 production > clusters (and corresponding QA environment clusters). We've upgraded about > 25% of our QA environment and run into a series of incompatibilities along > the way. Most of them have been easy to get around, but I wanted to create > this Jira to collect them so that we can make an update to the docs for > future upgraders. > My plan is to periodically edit this description to add to the list. If > anyone else has anything to contribute, feel free to edit as well or add a > comment. > Incompatibilities to document: > - HBASE-15676 changed the serialized byte string used for the fuzzy mask. > FuzzyRowFilters created by older clients will not match any rows in an hbase2 > cluster. This was fixed in HBASE-26537 but should be documented in our > upgrade guide. > - CDH5 try/catches bad HTableDescriptor.getDurability calls and returns > USE_DEFAULT. In hbase2, if someone creates a table with a bad durability > (i.e. DEFAULT instead of USE_DEFAULT), it results in a failure which causes > the CreateTableProcedure to infinitely retries with no backoff. This rapid > retry caused a bunch of pain on the cluster that encountered it, backing up > datanode's ability to keep up with the millions of calls to create and delete > .regioninfo files. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456672#comment-17456672
]
Andrew Kyle Purtell commented on HBASE-26027:
-
Thank you [~filtertip]. +1
> The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone
> caused by ArrayStoreException
> -
>
> Key: HBASE-26027
> URL: https://issues.apache.org/jira/browse/HBASE-26027
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.7, 2.5.0, 2.3.5, 2.4.4
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 2.6.0
>
>
> The batch api of HTable contains a param named results to store result or
> exception, its type is Object[].
> If user pass an array with other type, eg:
> org.apache.hadoop.hbase.client.Result, and if we need to put an exception
> into it by some reason, then the ArrayStoreException will occur in
> AsyncRequestFutureImpl.updateResult, then the
> AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
> AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
> actionsInProgress again and again, forever.
> It is better to add an cutoff calculated by operationTimeout, instead of only
> depend on the value of actionsInProgress.
> BTW, this issue only for 2.x, since 3.x the implement has refactored.
> How to reproduce:
> 1: add sleep in RSRpcServices.multi to mock slow response
> {code:java}
> try {
> Thread.sleep(2000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> {code}
> 2: set time out in config
> {code:java}
> conf.set("hbase.rpc.timeout","2000");
> conf.set("hbase.client.operation.timeout","6000");
> {code}
> 3: call batch api
> {code:java}
> Table table = HbaseUtil.getTable("test");
> byte[] cf = Bytes.toBytes("f");
> byte[] c = Bytes.toBytes("c1");
> List gets = new ArrayList<>();
> for (int i = 0; i < 10; i++) {
> byte[] rk = Bytes.toBytes("rk-" + i);
> Get get = new Get(rk);
> get.addColumn(cf, c);
> gets.add(get);
> }
> Result[] results = new Result[gets.size()];
> table.batch(gets, results);
> {code}
> The log will looks like below:
> {code:java}
> [ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 -
> id=1 error for test processing localhost,16020,1624343786295
> java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> [INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to
> finish on table:
> [INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to
> finish on table: test
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-26027:
Fix Version/s: (was: 2.5.0)
> The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone
> caused by ArrayStoreException
> -
>
> Key: HBASE-26027
> URL: https://issues.apache.org/jira/browse/HBASE-26027
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.7, 2.5.0, 2.3.5, 2.4.4
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 2.6.0
>
>
> The batch api of HTable contains a param named results to store result or
> exception, its type is Object[].
> If user pass an array with other type, eg:
> org.apache.hadoop.hbase.client.Result, and if we need to put an exception
> into it by some reason, then the ArrayStoreException will occur in
> AsyncRequestFutureImpl.updateResult, then the
> AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
> AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
> actionsInProgress again and again, forever.
> It is better to add an cutoff calculated by operationTimeout, instead of only
> depend on the value of actionsInProgress.
> BTW, this issue only for 2.x, since 3.x the implement has refactored.
> How to reproduce:
> 1: add sleep in RSRpcServices.multi to mock slow response
> {code:java}
> try {
> Thread.sleep(2000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> {code}
> 2: set time out in config
> {code:java}
> conf.set("hbase.rpc.timeout","2000");
> conf.set("hbase.client.operation.timeout","6000");
> {code}
> 3: call batch api
> {code:java}
> Table table = HbaseUtil.getTable("test");
> byte[] cf = Bytes.toBytes("f");
> byte[] c = Bytes.toBytes("c1");
> List gets = new ArrayList<>();
> for (int i = 0; i < 10; i++) {
> byte[] rk = Bytes.toBytes("rk-" + i);
> Get get = new Get(rk);
> get.addColumn(cf, c);
> gets.add(get);
> }
> Result[] results = new Result[gets.size()];
> table.batch(gets, results);
> {code}
> The log will looks like below:
> {code:java}
> [ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 -
> id=1 error for test processing localhost,16020,1624343786295
> java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> [INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to
> finish on table:
> [INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to
> finish on table: test
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26027) The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
[
https://issues.apache.org/jira/browse/HBASE-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-26027:
Fix Version/s: 2.5.0
> The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone
> caused by ArrayStoreException
> -
>
> Key: HBASE-26027
> URL: https://issues.apache.org/jira/browse/HBASE-26027
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.2.7, 2.5.0, 2.3.5, 2.4.4
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 2.5.0, 2.6.0
>
>
> The batch api of HTable contains a param named results to store result or
> exception, its type is Object[].
> If user pass an array with other type, eg:
> org.apache.hadoop.hbase.client.Result, and if we need to put an exception
> into it by some reason, then the ArrayStoreException will occur in
> AsyncRequestFutureImpl.updateResult, then the
> AsyncRequestFutureImpl.decActionCounter will be skipped, then in the
> AsyncRequestFutureImpl.waitUntilDone we will stuck at here checking the
> actionsInProgress again and again, forever.
> It is better to add an cutoff calculated by operationTimeout, instead of only
> depend on the value of actionsInProgress.
> BTW, this issue only for 2.x, since 3.x the implement has refactored.
> How to reproduce:
> 1: add sleep in RSRpcServices.multi to mock slow response
> {code:java}
> try {
> Thread.sleep(2000);
> } catch (InterruptedException e) {
> e.printStackTrace();
> }
> {code}
> 2: set time out in config
> {code:java}
> conf.set("hbase.rpc.timeout","2000");
> conf.set("hbase.client.operation.timeout","6000");
> {code}
> 3: call batch api
> {code:java}
> Table table = HbaseUtil.getTable("test");
> byte[] cf = Bytes.toBytes("f");
> byte[] c = Bytes.toBytes("c1");
> List gets = new ArrayList<>();
> for (int i = 0; i < 10; i++) {
> byte[] rk = Bytes.toBytes("rk-" + i);
> Get get = new Get(rk);
> get.addColumn(cf, c);
> gets.add(get);
> }
> Result[] results = new Result[gets.size()];
> table.batch(gets, results);
> {code}
> The log will looks like below:
> {code:java}
> [ERROR] [2021/06/22 23:23:00,676] hconnection-0x6b927fb-shared-pool3-t1 -
> id=1 error for test processing localhost,16020,1624343786295
> java.lang.ArrayStoreException: org.apache.hadoop.hbase.DoNotRetryIOException
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.updateResult(AsyncRequestFutureImpl.java:1242)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.trySetResultSimple(AsyncRequestFutureImpl.java:1087)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.setError(AsyncRequestFutureImpl.java:1021)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.manageError(AsyncRequestFutureImpl.java:683)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.receiveGlobalFailure(AsyncRequestFutureImpl.java:716)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.access$1500(AsyncRequestFutureImpl.java:69)
> at
> org.apache.hadoop.hbase.client.AsyncRequestFutureImpl$SingleServerRequestRunnable.run(AsyncRequestFutureImpl.java:219)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> at java.util.concurrent.FutureTask.run(FutureTask.java)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> [INFO ] [2021/06/22 23:23:10,375] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:20,378] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:30,384] main - #1, waiting for 10 actions to
> finish on table:
> [INFO ] [2021/06/22 23:23:40,387] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:23:50,397] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:00,400] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:10,408] main - #1, waiting for 10 actions to
> finish on table: test
> [INFO ] [2021/06/22 23:24:20,413] main - #1, waiting for 10 actions to
> finish on table: test
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26541) hbase-protocol-shaded not buildable on M1 MacOSX
[
https://issues.apache.org/jira/browse/HBASE-26541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456668#comment-17456668
]
Hudson commented on HBASE-26541:
Results for branch branch-2
[build #413 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/413//console].
> hbase-protocol-shaded not buildable on M1 MacOSX
>
>
> Key: HBASE-26541
> URL: https://issues.apache.org/jira/browse/HBASE-26541
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.4.8
> Environment: Apache Maven 3.8.3
> (ff8e977a158738155dc465c6a97ffaf31982d739)
> Java version: 1.8.0_312, vendor: Azul Systems, Inc., runtime:
> /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
> OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
>
> I needed to make some changes to get branch-2's hbase-protocol-shaded
> building on an M1 mac.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] ndimiduk commented on pull request #3924: HBASE-26542 Apply a `package` to test protobuf files
ndimiduk commented on pull request #3924: URL: https://github.com/apache/hbase/pull/3924#issuecomment-990101767 The primary change on this PR (HBASE-26542) is a change to the test protoc files. pre-commit did not handle the change properly and errored out (see the initial jenkins build). I pushed a second commit here, the same commit as is on #3927 / HBASE-26549, which tweaks the `hbaseprotoc` plugin shipped in our personality file, in a way that I thought might fix the initial build failure. This PR/HBASE-26542 is need in order to test inclusion of protoc service package names in HBASE-26521 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-24356) Move hbase-connectors to hbase-thirdparty-3.3.0
[ https://issues.apache.org/jira/browse/HBASE-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak-Lon (Stephen) Wu updated HBASE-24356: - Fix Version/s: hbase-connectors-1.1.0 (was: hbase-connectors-1.0.1) > Move hbase-connectors to hbase-thirdparty-3.3.0 > --- > > Key: HBASE-24356 > URL: https://issues.apache.org/jira/browse/HBASE-24356 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Affects Versions: hbase-connectors-1.0.1 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: hbase-connectors-1.1.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-24356) Move hbase-connectors to hbase-thirdparty-3.3.0
[ https://issues.apache.org/jira/browse/HBASE-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456585#comment-17456585 ] Peter Somogyi commented on HBASE-24356: --- This ticket was planned for the next hbase-connectors release. At that point, it was not decided if it will be 1.0.1 or 1.1.0. The best would be if connectors can use the latest hbase-thirdparty but is compatible with the still active hbase-2 releases. I haven't tested if it could cause any classpath issues if a different third-party version is used in hbase and connectors. > Move hbase-connectors to hbase-thirdparty-3.3.0 > --- > > Key: HBASE-24356 > URL: https://issues.apache.org/jira/browse/HBASE-24356 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Affects Versions: hbase-connectors-1.0.1 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: hbase-connectors-1.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-24356) Move hbase-connectors to hbase-thirdparty-3.3.0
[ https://issues.apache.org/jira/browse/HBASE-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456571#comment-17456571 ] Tak-Lon (Stephen) Wu commented on HBASE-24356: -- [~psomogyi] sorry for pinging here, but what is the plan for 1.0.1 ? do we need to confirm if after upgrading hbase-thirdparty to 3.3.0 and hbase-connectors works across some hbase version? > Move hbase-connectors to hbase-thirdparty-3.3.0 > --- > > Key: HBASE-24356 > URL: https://issues.apache.org/jira/browse/HBASE-24356 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Affects Versions: hbase-connectors-1.0.1 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: hbase-connectors-1.0.1 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
[
https://issues.apache.org/jira/browse/HBASE-26553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andor Molnar updated HBASE-26553:
-
Description:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because that's the
primary use case of this new feature.
* New client example is added to the hbase-examples project to showcase the
feature.
* It's important that this Jira does not cover the solution for obtaining a
token from Knox. The assumption is that the client already has a valid token in
base64 encoded string and we only provide a helper method for adding it to
user's credentials.
* Renewing expired tokens is also the responsibility of the client. We don't
provide a mechanism for that in this Jira, but it's planned to be covered in a
follow-up ticket.
The following new parameters are introduced in hbase-site.xml:
* hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set.
(required if URL not specified)
* hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required
if File not specified)
* hbase.security.oauth.jwt.audience - Required audience, "aud" claim of the
JWT. (optional)
* hbase.security.oauth.jwt.issuer - Required issuer, "iss" claim of the JWT.
(optional)
The feature will be behind feature-flag. No code part is executed unless the
following configuration is set in hbase-site.xml:
{noformat}
hbase.client.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslClientAuthenticationProvider
hbase.server.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslServerAuthenticationProvider
hbase.client.sasl.provider.class
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslProviderSelector
{noformat}
Example of Knox provided JWKS file:
{noformat}
{
"keys":
[{
"kty": "RSA",
"e": "",
"use": "sig",
"kid": "",
"alg": "RS256",
"n": ""
}]
}{noformat}
Example of Knox issued JWT header:
{noformat}
{
"jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
"kid": "",
"alg": "RS256"
}{noformat}
And payload:
{noformat}
{
"sub": "user_andor",
"aud": "knox-proxy-token",
"jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
"kid": "",
"iss": "KNOXSSO",
"exp": 1636644029,
"managed.token": "true",
"knox.id": ""
}{noformat}
was:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because
[jira] [Commented] (HBASE-22338) LICENSE file only contains Apache 2.0
[ https://issues.apache.org/jira/browse/HBASE-22338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456570#comment-17456570 ] Tak-Lon (Stephen) Wu commented on HBASE-22338: -- will this block 1.0.1 release ? if so, I (or @rabi-kumar if he is still around) may pick this up. > LICENSE file only contains Apache 2.0 > - > > Key: HBASE-22338 > URL: https://issues.apache.org/jira/browse/HBASE-22338 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Affects Versions: connector-1.0.0 >Reporter: Peter Somogyi >Assignee: Rabi Kumar K C >Priority: Critical > Fix For: hbase-connectors-1.0.1 > > Attachments: hbase-connectors-dependency.html > > > LICENSE.md file has only Apache 2.0 licenses but we package dependencies that > use different ones. For example jcodings uses MIT. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase-connectors] taklwu commented on pull request #41: HBASE-22338 LICENSE file only contains Apache 2.0.
taklwu commented on pull request #41: URL: https://github.com/apache/hbase-connectors/pull/41#issuecomment-990049044 will this block 1.0.1 release ? if so, I (or @rabi-kumar if he is still around) may pick this up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26469) HBase shell has changed exit behavior
[
https://issues.apache.org/jira/browse/HBASE-26469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456568#comment-17456568
]
Mike Drob commented on HBASE-26469:
---
Ok, thank you. Then I'm -1 on the current patch because it doesn't do what we
claim. :)
> HBase shell has changed exit behavior
> -
>
> Key: HBASE-26469
> URL: https://issues.apache.org/jira/browse/HBASE-26469
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.8
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.5.0, 2.4.9, 2.6.0
>
> Attachments: hbase-1.4.14-exit-behavior.log,
> hbase-1.7.1-exit-behavior.log, hbase-2.0.6-exit-behavior.log,
> hbase-2.1.9-exit-behavior.log, hbase-2.2.7-exit-behavior.log,
> hbase-2.3.7-exit-behavior.log, hbase-2.4.8-exit-behavior.log,
> hbase-3.0.0-alpha-2-exit-behavior.log
>
>
> The HBase shell has changed behavior in a way that breaks being able to exit
> properly.
> Two example scripts to act as stand ins for hbase shell scripts to "do
> something simple then exit":
> {code}
> tmp % echo "list\nexit" > clean_exit.rb
> tmp % echo "list\nexit 1" > error_exit.rb
> {code}
> Giving these two scripts is possible:
> * passed as a cli argument
> * via redirected stdin
> Additionally the shell invocation can be:
> * in the default compatibility mode
> * with the "non interactive" flag that gives an exit code that reflects
> runtime errors
> I'll post logs of the details as attachments but here are some tables of the
> exit codes.
> The {{clean_exit.rb}} invocations ought to exit with success, exit code 0.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8
> || master ||
> | cli, default |0 |0 |0 |0 |0 |0 |1 |
>1* |
> | cli, -n | 0 |0 |0 |0 |0 |0 |1 |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
>0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
>1* |
> The {{error_exit.rb}} invocation should return a non-zero exit code, unless
> we're specifically trying to match a normal hbase shell session.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 ||
> master ||
> | cli, default | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> | cli, -n |1 |1 |1 |1 |1 |1 |1* |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
> 0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> In cases marked with * the error details are different.
> The biggest concern are the new-to-2.4 non-zero exit code when we should have
> a success and the hanging.
> The former looks like this:
> {code}
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> {code}
> The change in error details for the error exit script also shows this same
> detail.
> This behavior appears to be a side effect of HBASE-11686. As far as I can
> tell, the IRB handling of 'exit' calls fail because we implement our own
> handling of sessoins rather than rely on the intended session interface. We
> never set a current session, and IRB's exit implementation presumes there
> will be one.
> Running in debug shows this in a stacktrace:
> {code}
> Took 0.4563 seconds
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> NoMethodError: private method `exit' called for nil:NilClass
> irb_exit at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb:30
> evaluate at stdin:2
> eval at org/jruby/RubyKernel.java:1048
> evaluate at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85
> eval_io at uri:classloader:/shell.rb:327
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:246
> loop at org/jruby/RubyKernel.java:1442
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:232
> catch at org/jruby/RubyKernel.java:1189
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:231
> eval_io at uri:classloader:/shell.rb:326
> classpath:/jar-bootstrap.rb at classpath:/jar-bootstrap.rb:194
> exception_handler at uri:classloader:/shell.rb:339
> at classpath:/jar-bootstrap.rb:194
> {code}
> And in our version of IRB (0.9.6) [line 30 for
>
[jira] [Commented] (HBASE-26469) HBase shell has changed exit behavior
[
https://issues.apache.org/jira/browse/HBASE-26469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456558#comment-17456558
]
Sean Busbey commented on HBASE-26469:
-
{quote}1. For HBase 2.4.9, clean_exit.rb passed as a cli argument should return
0. This is different from 2.4.8 but the same as 2.3.x
{quote}
yes, I agree with this statement.
{quote}2. For HBase 2.4.9, clean_exit.rb passed via stdin with non-interactive
mode should return 1. This is different from what the current proposed PR does
and will require special logic to detect this for keeping backward
compatibility.
{quote}
yes, I agree with this statement.
{quote}3. For HBase 2.5.0, clean_exit.rb passed via stdin with non-interactive
mode should return 0. This is different from 2.3.x and 2.4.x
{quote}
yes, I agree with this statement.
> HBase shell has changed exit behavior
> -
>
> Key: HBASE-26469
> URL: https://issues.apache.org/jira/browse/HBASE-26469
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.8
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.5.0, 2.4.9, 2.6.0
>
> Attachments: hbase-1.4.14-exit-behavior.log,
> hbase-1.7.1-exit-behavior.log, hbase-2.0.6-exit-behavior.log,
> hbase-2.1.9-exit-behavior.log, hbase-2.2.7-exit-behavior.log,
> hbase-2.3.7-exit-behavior.log, hbase-2.4.8-exit-behavior.log,
> hbase-3.0.0-alpha-2-exit-behavior.log
>
>
> The HBase shell has changed behavior in a way that breaks being able to exit
> properly.
> Two example scripts to act as stand ins for hbase shell scripts to "do
> something simple then exit":
> {code}
> tmp % echo "list\nexit" > clean_exit.rb
> tmp % echo "list\nexit 1" > error_exit.rb
> {code}
> Giving these two scripts is possible:
> * passed as a cli argument
> * via redirected stdin
> Additionally the shell invocation can be:
> * in the default compatibility mode
> * with the "non interactive" flag that gives an exit code that reflects
> runtime errors
> I'll post logs of the details as attachments but here are some tables of the
> exit codes.
> The {{clean_exit.rb}} invocations ought to exit with success, exit code 0.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8
> || master ||
> | cli, default |0 |0 |0 |0 |0 |0 |1 |
>1* |
> | cli, -n | 0 |0 |0 |0 |0 |0 |1 |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
>0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
>1* |
> The {{error_exit.rb}} invocation should return a non-zero exit code, unless
> we're specifically trying to match a normal hbase shell session.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 ||
> master ||
> | cli, default | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> | cli, -n |1 |1 |1 |1 |1 |1 |1* |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
> 0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> In cases marked with * the error details are different.
> The biggest concern are the new-to-2.4 non-zero exit code when we should have
> a success and the hanging.
> The former looks like this:
> {code}
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> {code}
> The change in error details for the error exit script also shows this same
> detail.
> This behavior appears to be a side effect of HBASE-11686. As far as I can
> tell, the IRB handling of 'exit' calls fail because we implement our own
> handling of sessoins rather than rely on the intended session interface. We
> never set a current session, and IRB's exit implementation presumes there
> will be one.
> Running in debug shows this in a stacktrace:
> {code}
> Took 0.4563 seconds
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> NoMethodError: private method `exit' called for nil:NilClass
> irb_exit at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb:30
> evaluate at stdin:2
> eval at org/jruby/RubyKernel.java:1048
> evaluate at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85
> eval_io at uri:classloader:/shell.rb:327
> each_top_level_statement at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:246
> loop at org/jruby/RubyKernel.java:1442
> each_top_level_statement at
>
[GitHub] [hbase] Apache-HBase commented on pull request #3930: HBASE-26554 Introduce a new parameter in jmx servlet to exclude the s…
Apache-HBase commented on pull request #3930: URL: https://github.com/apache/hbase/pull/3930#issuecomment-990014567 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 0m 35s | master passed | | +1 :green_heart: | checkstyle | 0m 17s | master passed | | +1 :green_heart: | spotbugs | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | checkstyle | 0m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 39m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3930 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 76d5797c4186 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3930: HBASE-26554 Introduce a new parameter in jmx servlet to exclude the s…
Apache-HBase commented on pull request #3930: URL: https://github.com/apache/hbase/pull/3930#issuecomment-990008793 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 21s | master passed | | +1 :green_heart: | compile | 0m 21s | master passed | | +1 :green_heart: | shadedjars | 9m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 59s | hbase-http in the patch passed. | | | | 33m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3930 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 91b15608de04 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/testReport/ | | Max. process+thread count | 380 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3930: HBASE-26554 Introduce a new parameter in jmx servlet to exclude the s…
Apache-HBase commented on pull request #3930: URL: https://github.com/apache/hbase/pull/3930#issuecomment-990007252 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 32s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | shadedjars | 9m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | shadedjars | 9m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 54s | hbase-http in the patch passed. | | | | 31m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3930 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 827496e32403 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/testReport/ | | Max. process+thread count | 410 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3930/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on pull request #3928: HBASE-26550 Make sure the master is running normally before accepting…
joshelser commented on pull request #3928: URL: https://github.com/apache/hbase/pull/3928#issuecomment-990004738 > Should this be called in MasterRpcServicces method or here? I can not fully recall the suggested pattern... Heh, I had to ask myself the same question! I saw other instances (like `mergeRegions()` https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L2045-L2051) which did this in HMaster rather than in MasterRpcServices. I think this change is in the correct place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26469) HBase shell has changed exit behavior
[
https://issues.apache.org/jira/browse/HBASE-26469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456537#comment-17456537
]
Mike Drob commented on HBASE-26469:
---
bq. I am hesitant to change exit codes in patch releases. I think the
additional error indicating exit codes that happen on 2.4.y are disruptive
enough for those migrating from 2.3 and earlier that branch-2.4 should be
updated to match the behavior in 2.3. (edited for clarity, thanks Mike)
Still somewhat confused. Please confirm each of the following statements
individually:
1. For HBase 2.4.9, clean_exit.rb passed as a cli argument should return 0.
This is different from 2.4.8 but the same as 2.3.x
2. For HBase 2.4.9, clean_exit.rb passed via stdin with non-interactive mode
should return 1. This is different from what the current proposed PR does and
will require special logic to detect this for keeping backward compatibility.
3. For HBase 2.5.0, clean_exit.rb passed via stdin with non-interactive mode
should return 0. This is different from 2.3.x and 2.4.x
> HBase shell has changed exit behavior
> -
>
> Key: HBASE-26469
> URL: https://issues.apache.org/jira/browse/HBASE-26469
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.4.8
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.5.0, 2.4.9, 2.6.0
>
> Attachments: hbase-1.4.14-exit-behavior.log,
> hbase-1.7.1-exit-behavior.log, hbase-2.0.6-exit-behavior.log,
> hbase-2.1.9-exit-behavior.log, hbase-2.2.7-exit-behavior.log,
> hbase-2.3.7-exit-behavior.log, hbase-2.4.8-exit-behavior.log,
> hbase-3.0.0-alpha-2-exit-behavior.log
>
>
> The HBase shell has changed behavior in a way that breaks being able to exit
> properly.
> Two example scripts to act as stand ins for hbase shell scripts to "do
> something simple then exit":
> {code}
> tmp % echo "list\nexit" > clean_exit.rb
> tmp % echo "list\nexit 1" > error_exit.rb
> {code}
> Giving these two scripts is possible:
> * passed as a cli argument
> * via redirected stdin
> Additionally the shell invocation can be:
> * in the default compatibility mode
> * with the "non interactive" flag that gives an exit code that reflects
> runtime errors
> I'll post logs of the details as attachments but here are some tables of the
> exit codes.
> The {{clean_exit.rb}} invocations ought to exit with success, exit code 0.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8
> || master ||
> | cli, default |0 |0 |0 |0 |0 |0 |1 |
>1* |
> | cli, -n | 0 |0 |0 |0 |0 |0 |1 |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
>0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
>1* |
> The {{error_exit.rb}} invocation should return a non-zero exit code, unless
> we're specifically trying to match a normal hbase shell session.
> || || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 ||
> master ||
> | cli, default | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> | cli, -n |1 |1 |1 |1 |1 |1 |1* |
> hang |
> | stdin, default | 0 |0 |0 |0 |0 |0 |0 |
> 0|
> | stdin, -n | 1 |1 |1 |1 |1 |1 |1* |
> 1* |
> In cases marked with * the error details are different.
> The biggest concern are the new-to-2.4 non-zero exit code when we should have
> a success and the hanging.
> The former looks like this:
> {code}
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> {code}
> The change in error details for the error exit script also shows this same
> detail.
> This behavior appears to be a side effect of HBASE-11686. As far as I can
> tell, the IRB handling of 'exit' calls fail because we implement our own
> handling of sessoins rather than rely on the intended session interface. We
> never set a current session, and IRB's exit implementation presumes there
> will be one.
> Running in debug shows this in a stacktrace:
> {code}
> Took 0.4563 seconds
> ERROR NoMethodError: private method `exit' called for nil:NilClass
> NoMethodError: private method `exit' called for nil:NilClass
> irb_exit at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb:30
> evaluate at stdin:2
> eval at org/jruby/RubyKernel.java:1048
> evaluate at
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85
> eval_io at
[GitHub] [hbase] frostruan opened a new pull request #3930: HBASE-26554 Introduce a new parameter in jmx servlet to exclude the s…
frostruan opened a new pull request #3930: URL: https://github.com/apache/hbase/pull/3930 …pecific mbean -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26554) Introduce a new parameter in jmx servlet to exclude the specific mbean
[ https://issues.apache.org/jira/browse/HBASE-26554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ruanhui updated HBASE-26554: Description: There are many regionservers serving over a thousand regions in our company, and the metric load is pretty big. I tried to exclude some huge mbean like 'Hadoop:service=HBase,name=RegionServer,sub=Regions' with regex, but didn't succeed. So I want to propose a new parameter 'excl' in jmx servlet to exclude the splecific bean or beans. was: There are many regionservers serving over a thousand regions, and the metric load is pretty big. I tried to exclude some huge mbean like 'Hadoop:service=HBase,name=RegionServer,sub=Regions' with regex, but didn't succeed. So I want to propose a new parameter 'excl' in jmx servlet to exclude the splecific bean or beans. > Introduce a new parameter in jmx servlet to exclude the specific mbean > -- > > Key: HBASE-26554 > URL: https://issues.apache.org/jira/browse/HBASE-26554 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.4.8 >Reporter: ruanhui >Assignee: ruanhui >Priority: Minor > Fix For: 3.0.0-alpha-2 > > > There are many regionservers serving over a thousand regions in our company, > and the metric load is pretty big. > I tried to exclude some huge mbean like > 'Hadoop:service=HBase,name=RegionServer,sub=Regions' with regex, but didn't > succeed. > So I want to propose a new parameter 'excl' in jmx servlet to exclude the > splecific bean or beans. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26554) Introduce a new parameter in jmx servlet to exclude the specific mbean
ruanhui created HBASE-26554: --- Summary: Introduce a new parameter in jmx servlet to exclude the specific mbean Key: HBASE-26554 URL: https://issues.apache.org/jira/browse/HBASE-26554 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 2.4.8 Reporter: ruanhui Assignee: ruanhui Fix For: 3.0.0-alpha-2 There are many regionservers serving over a thousand regions, and the metric load is pretty big. I tried to exclude some huge mbean like 'Hadoop:service=HBase,name=RegionServer,sub=Regions' with regex, but didn't succeed. So I want to propose a new parameter 'excl' in jmx servlet to exclude the splecific bean or beans. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
[
https://issues.apache.org/jira/browse/HBASE-26553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456497#comment-17456497
]
Josh Elser commented on HBASE-26553:
Hooray! I've been talking with Andor in the background about implementing this
functionality. I'm very excited to see this issue and the incoming PR.
One of the things we've been trying to figure out is how to build an
appropriate "some batteries included" version of this for others to pick up in
Apache. If anyone has input/motivation to also get involved, happy to chat. We
can also discuss on email (probably a better way than here).
> OAuth Bearer authentication mech plugin for SASL
>
>
> Key: HBASE-26553
> URL: https://issues.apache.org/jira/browse/HBASE-26553
> Project: HBase
> Issue Type: New Feature
> Components: security
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
>
> Implementation of a new SASL plugin to add support for OAuth Bearer token
> authentication for HBase client RPC.
> * The plugin supports secured (cryptographically signed) JSON Web Token
> authentication as defined in
> [RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format
> in [RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
> * The implementation is inspired by [Apache Kafka's OAuth Bearer
> token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
> support with the important difference that HBase version is intended for
> production usage. The two main differences are that Kafka supports unsecured
> tokens only and it issues the tokens for itself which breaks the principle of
> OAuth token authentication.
> * We use the [Nimbus JOSE +
> JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java
> library for signature verification and token processing and we add it as a
> new dependency to HBase.
> * We add secure JWT support and verification of digital signatures with
> multiple algorithms as supported by Nimbus. Json-formatted JWK set is
> required for the signature verification as defined in
> [RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
> * The impl is verified with Apache Knox issued tokens, because that's the
> primary use case of this new feature.
> * New client example is added to the hbase-examples project to showcase the
> feature.
> * It's important that this Jira does not cover the solution for obtaining a
> token from Knox. The assumption is that the client already has a valid token
> in base64 encoded string and we only provide a helper method for adding it to
> user's credentials.
> * Renewing expired tokens is also the responsibility of the client. We don't
> provide a mechanism for that in this Jira, but it's planned to be covered in
> a follow-up ticket.
> The following new parameters are introduced in hbase-site.xml:
> * hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set.
> (required if URL not specified)
> * hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required
> if File not specified)
> * hbase.security.oauth.jwt.requiredaudience - Required audience of the JWT.
> (optional)
> The feature will be behind feature-flag. No code part is executed unless the
> following configuration is set in hbase-site.xml:
> {noformat}
>
> hbase.client.sasl.provider.extras
>
> org.apache.hadoop.hbase.security.provider.OAuthBearerSaslClientAuthenticationProvider
>
>
> hbase.server.sasl.provider.extras
>
> org.apache.hadoop.hbase.security.provider.OAuthBearerSaslServerAuthenticationProvider
>
>
> hbase.client.sasl.provider.class
>
> org.apache.hadoop.hbase.security.provider.OAuthBearerSaslProviderSelector
>
> {noformat}
> Example of Knox provided JWKS file:
> {noformat}
> {
> "keys":
> [{
> "kty": "RSA",
> "e": "",
> "use": "sig",
> "kid": "",
> "alg": "RS256",
> "n": ""
> }]
> }{noformat}
> Example of Knox issued JWT header:
> {noformat}
> {
> "jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
> "kid": "",
> "alg": "RS256"
> }{noformat}
> And payload:
> {noformat}
> {
> "sub": "user_andor",
> "aud": "knox-proxy-token",
> "jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
> "kid": "",
> "iss": "KNOXSSO",
> "exp": 1636644029,
> "managed.token": "true",
> "knox.id": ""
> }{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
[
https://issues.apache.org/jira/browse/HBASE-26553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andor Molnar updated HBASE-26553:
-
Description:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because that's the
primary use case of this new feature.
* New client example is added to the hbase-examples project to showcase the
feature.
* It's important that this Jira does not cover the solution for obtaining a
token from Knox. The assumption is that the client already has a valid token in
base64 encoded string and we only provide a helper method for adding it to
user's credentials.
* Renewing expired tokens is also the responsibility of the client. We don't
provide a mechanism for that in this Jira, but it's planned to be covered in a
follow-up ticket.
The following new parameters are introduced in hbase-site.xml:
* hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set.
(required if URL not specified)
* hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required
if File not specified)
* hbase.security.oauth.jwt.requiredaudience - Required audience of the JWT.
(optional)
The feature will be behind feature-flag. No code part is executed unless the
following configuration is set in hbase-site.xml:
{noformat}
hbase.client.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslClientAuthenticationProvider
hbase.server.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslServerAuthenticationProvider
hbase.client.sasl.provider.class
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslProviderSelector
{noformat}
Example of Knox provided JWKS file:
{noformat}
{
"keys":
[{
"kty": "RSA",
"e": "",
"use": "sig",
"kid": "",
"alg": "RS256",
"n": ""
}]
}{noformat}
Example of Knox issued JWT header:
{noformat}
{
"jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
"kid": "",
"alg": "RS256"
}{noformat}
And payload:
{noformat}
{
"sub": "user_andor",
"aud": "knox-proxy-token",
"jku": "https://path/to/homepage/knoxtoken/api/v1/jwks.json;,
"kid": "",
"iss": "KNOXSSO",
"exp": 1636644029,
"managed.token": "true",
"knox.id": ""
}{noformat}
was:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because that's the
primary use case of this new feature.
* New client example is added to the
[jira] [Updated] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
[
https://issues.apache.org/jira/browse/HBASE-26553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andor Molnar updated HBASE-26553:
-
Description:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because that's the
primary use case of this new feature.
* New client example is added to the hbase-examples project to showcase the
feature.
* It's important that this Jira does not cover the solution for obtaining a
token from Knox. The assumption is that the client already has a valid token in
base64 encoded string and we only provide a helper method for adding it to
user's credentials.
* Renewing expired tokens is also the responsibility of the client. We don't
provide a mechanism for that in this Jira, but it's planned to be covered in a
follow-up ticket.
The following new parameters are introduced in hbase-site.xml:
* hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set.
(required if URL not specified)
* hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required
if File not specified)
* hbase.security.oauth.jwt.requiredaudience - Required audience of the JWT.
(optional)
The feature will be behind feature-flag. No code part is executed unless the
following configuration is set in hbase-site.xml:
{noformat}
hbase.client.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslClientAuthenticationProvider
hbase.server.sasl.provider.extras
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslServerAuthenticationProvider
hbase.client.sasl.provider.class
org.apache.hadoop.hbase.security.provider.OAuthBearerSaslProviderSelector
{noformat}
was:
Implementation of a new SASL plugin to add support for OAuth Bearer token
authentication for HBase client RPC.
* The plugin supports secured (cryptographically signed) JSON Web Token
authentication as defined in
[RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in
[RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] .
* The implementation is inspired by [Apache Kafka's OAuth Bearer
token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html]
support with the important difference that HBase version is intended for
production usage. The two main differences are that Kafka supports unsecured
tokens only and it issues the tokens for itself which breaks the principle of
OAuth token authentication.
* We use the [Nimbus JOSE +
JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library
for signature verification and token processing and we add it as a new
dependency to HBase.
* We add secure JWT support and verification of digital signatures with
multiple algorithms as supported by Nimbus. Json-formatted JWK set is required
for the signature verification as defined in
[RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517].
* The impl is verified with Apache Knox issued tokens, because that's the
primary use case of this new feature.
* New client example is added to the hbase-examples project to showcase the
feature.
* It's important that this Jira does not cover the solution for obtaining a
token from Knox. The assumption is that the client already has a valid token in
base64 encoded string and we only provide a helper method for adding it to
user's credentials.
* Renewing expired tokens is also the responsibility of the client. We don't
provide a mechanism for that in this Jira, but it's planned to be covered in a
follow-up ticket.
The following new parameters are introduced in hbase-site.xml:
* hbase.security.oauth.jwt.jwks.file
[jira] [Commented] (HBASE-26541) hbase-protocol-shaded not buildable on M1 MacOSX
[
https://issues.apache.org/jira/browse/HBASE-26541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456444#comment-17456444
]
Hudson commented on HBASE-26541:
Results for branch master
[build #464 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/464/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/464/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/464/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/464/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> hbase-protocol-shaded not buildable on M1 MacOSX
>
>
> Key: HBASE-26541
> URL: https://issues.apache.org/jira/browse/HBASE-26541
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.0, 2.4.8
> Environment: Apache Maven 3.8.3
> (ff8e977a158738155dc465c6a97ffaf31982d739)
> Java version: 1.8.0_312, vendor: Azul Systems, Inc., runtime:
> /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
> OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.6.0
>
>
> I needed to make some changes to get branch-2's hbase-protocol-shaded
> building on an M1 mac.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work started] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
[ https://issues.apache.org/jira/browse/HBASE-26553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26553 started by Andor Molnar. > OAuth Bearer authentication mech plugin for SASL > > > Key: HBASE-26553 > URL: https://issues.apache.org/jira/browse/HBASE-26553 > Project: HBase > Issue Type: New Feature > Components: security >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > > Implementation of a new SASL plugin to add support for OAuth Bearer token > authentication for HBase client RPC. > * The plugin supports secured (cryptographically signed) JSON Web Token > authentication as defined in > [RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format > in [RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] . > * The implementation is inspired by [Apache Kafka's OAuth Bearer > token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html] > support with the important difference that HBase version is intended for > production usage. The two main differences are that Kafka supports unsecured > tokens only and it issues the tokens for itself which breaks the principle of > OAuth token authentication. > * We use the [Nimbus JOSE + > JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java > library for signature verification and token processing and we add it as a > new dependency to HBase. > * We add secure JWT support and verification of digital signatures with > multiple algorithms as supported by Nimbus. Json-formatted JWK set is > required for the signature verification as defined in > [RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517]. > * The impl is verified with Apache Knox issued tokens, because that's the > primary use case of this new feature. > * New client example is added to the hbase-examples project to showcase the > feature. > * It's important that this Jira does not cover the solution for obtaining a > token from Knox. The assumption is that the client already has a valid token > in base64 encoded string and we only provide a helper method for adding it to > user's credentials. > * Renewing expired tokens is also the responsibility of the client. We don't > provide a mechanism for that in this Jira, but it's planned to be covered in > a follow-up ticket. > The following new parameters are introduced in hbase-site.xml: > * hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set. > (required if URL not specified) > * hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required > if File not specified) > * hbase.security.oauth.jwt.requiredaudience - Required audience of the JWT. > (optional) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26553) OAuth Bearer authentication mech plugin for SASL
Andor Molnar created HBASE-26553: Summary: OAuth Bearer authentication mech plugin for SASL Key: HBASE-26553 URL: https://issues.apache.org/jira/browse/HBASE-26553 Project: HBase Issue Type: New Feature Components: security Reporter: Andor Molnar Assignee: Andor Molnar Implementation of a new SASL plugin to add support for OAuth Bearer token authentication for HBase client RPC. * The plugin supports secured (cryptographically signed) JSON Web Token authentication as defined in [RFC-7628|https://datatracker.ietf.org/doc/html/rfc7628] and the JWT format in [RFC-7519|https://datatracker.ietf.org/doc/html/rfc7519] . * The implementation is inspired by [Apache Kafka's OAuth Bearer token|https://docs.confluent.io/platform/current/kafka/authentication_sasl/authentication_sasl_oauth.html] support with the important difference that HBase version is intended for production usage. The two main differences are that Kafka supports unsecured tokens only and it issues the tokens for itself which breaks the principle of OAuth token authentication. * We use the [Nimbus JOSE + JWT|https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/] Java library for signature verification and token processing and we add it as a new dependency to HBase. * We add secure JWT support and verification of digital signatures with multiple algorithms as supported by Nimbus. Json-formatted JWK set is required for the signature verification as defined in [RFC-7517|https://datatracker.ietf.org/doc/html/rfc7517]. * The impl is verified with Apache Knox issued tokens, because that's the primary use case of this new feature. * New client example is added to the hbase-examples project to showcase the feature. * It's important that this Jira does not cover the solution for obtaining a token from Knox. The assumption is that the client already has a valid token in base64 encoded string and we only provide a helper method for adding it to user's credentials. * Renewing expired tokens is also the responsibility of the client. We don't provide a mechanism for that in this Jira, but it's planned to be covered in a follow-up ticket. The following new parameters are introduced in hbase-site.xml: * hbase.security.oauth.jwt.jwks.file - Path of a local file for JWK set. (required if URL not specified) * hbase.security.oauth.jwt.jwks.url - URL to download the JWK set. (required if File not specified) * hbase.security.oauth.jwt.requiredaudience - Required audience of the JWT. (optional) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-989753278 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 22s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 9m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 9m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 203m 51s | hbase-server in the patch passed. | | | | 239m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c8ffa088ef85 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/testReport/ | | Max. process+thread count | 3851 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-989707110 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 8m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 0s | hbase-server in the patch passed. | | | | 181m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f2374933c2e4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/testReport/ | | Max. process+thread count | 4045 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3929: HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor
Apache-HBase commented on pull request #3929: URL: https://github.com/apache/hbase/pull/3929#issuecomment-989602800 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | checkstyle | 1m 3s | master passed | | +1 :green_heart: | spotbugs | 2m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | +1 :green_heart: | compile | 3m 8s | the patch passed | | +1 :green_heart: | javac | 3m 8s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 33s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 49m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3929 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux f762e594ad1d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b5cf3cdc5c | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3929/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
