Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1859062441 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 4s | master passed | | +1 :green_heart: | compile | 2m 33s | master passed | | +1 :green_heart: | shadedjars | 6m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 10s | master passed | | -0 :warning: | patch | 7m 52s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 2m 36s | the patch passed | | +1 :green_heart: | javac | 2m 36s | the patch passed | | +1 :green_heart: | shadedjars | 6m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 50s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 222m 50s | hbase-server in the patch passed. | | -1 :x: | unit | 7m 38s | hbase-shell in the patch failed. | | | | 266m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 382dd4731302 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-shell.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/testReport/ | | Max. process+thread count | 4398 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1859061539 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 31s | master passed | | +1 :green_heart: | compile | 1m 31s | master passed | | +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | | -0 :warning: | patch | 6m 32s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 1m 32s | the patch passed | | +1 :green_heart: | javac | 1m 32s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. | | -1 :x: | unit | 226m 56s | hbase-server in the patch failed. | | -1 :x: | unit | 6m 59s | hbase-shell in the patch failed. | | | | 261m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f0bb87ab0baa 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-shell.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/testReport/ | | Max. process+thread count | 4687 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-20528) Revise collections copying from iteration to built-in function
[
https://issues.apache.org/jira/browse/HBASE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797893#comment-17797893
]
Hudson commented on HBASE-20528:
Results for branch branch-2
[build #945 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Revise collections copying from iteration to built-in function
> --
>
> Key: HBASE-20528
> URL: https://issues.apache.org/jira/browse/HBASE-20528
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1
>Reporter: Hua-Yi Ho
>Assignee: Hua-Yi Ho
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-20528.master.001.patch
>
>
> Some collection codes in file
> StochasticLoadBalancer.java, AbstractHBaseTool.java, HFileInputFormat.java,
> Result.java, and WalPlayer.java, using iterations to copy whole data in
> collections. The iterations can just replace by just Colletions.addAll and
> Arrays.copyOf.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-21243) Correct java-doc for the method RpcServer.getRemoteAddress()
[
https://issues.apache.org/jira/browse/HBASE-21243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797892#comment-17797892
]
Hudson commented on HBASE-21243:
Results for branch branch-2
[build #945 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Correct java-doc for the method RpcServer.getRemoteAddress()
>
>
> Key: HBASE-21243
> URL: https://issues.apache.org/jira/browse/HBASE-21243
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Trivial
> Labels: beginner, beginners, documentaion
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-21243.master.001.patch,
> HBASE-21243.master.002.patch
>
>
> Correct the java-doc for the method {{RpcServer.getRemoteAddress()}}.
> Currently it look like as below:
> {code:java}
> /**
>* @return Address of remote client if a request is ongoing, else null
>*/
> public static Optional getRemoteAddress() {
> return getCurrentCall().map(RpcCall::getRemoteAddress);
> }
> {code}
> Contrary to the doc the method will never return null. Rather it may return
> an empty Optional.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28180) Review the usage of RegionStates.getOrCreateServer
[
https://issues.apache.org/jira/browse/HBASE-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797891#comment-17797891
]
Hudson commented on HBASE-28180:
Results for branch branch-2
[build #945 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/945/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Review the usage of RegionStates.getOrCreateServer
> --
>
> Key: HBASE-28180
> URL: https://issues.apache.org/jira/browse/HBASE-28180
> Project: HBase
> Issue Type: Bug
> Components: amv2, master
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
> Attachments:
> org.apache.hadoop.hbase.master.TestClusterRestartFailover-output.txt
>
>
> It failed two times in this PR.
> https://github.com/apache/hbase/pull/5475
> Filed an issue to track this problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1859050441 Hi @bbeaudreault thanks for the PR, overall looks good. Also have posted a few questions/reviews. Please let me know of your opinion on same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429032871
##
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestManageTableErasureCodingPolicy.java:
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.procedure;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.greaterThan;
+import static org.hamcrest.Matchers.notNullValue;
+import static org.hamcrest.Matchers.nullValue;
+import static org.junit.Assert.assertThrows;
+
+import java.io.IOException;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseIOException;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.hfile.HFile;
+import org.apache.hadoop.hbase.regionserver.CompactedHFilesDischarger;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.CommonFSUtils;
+import org.apache.hadoop.hbase.util.JVMClusterUtil;
+import org.apache.hadoop.hdfs.DistributedFileSystem;
+import org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({ MasterTests.class, MediumTests.class })
+public class TestManageTableErasureCodingPolicy {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestManageTableErasureCodingPolicy.class);
+
+ private static final HBaseTestingUtil UTIL = new HBaseTestingUtil();
+ private static final byte[] FAMILY = Bytes.toBytes("a");
+ private static final TableName NON_EC_TABLE = TableName.valueOf("foo");
+ private static final TableDescriptor NON_EC_TABLE_DESC =
TableDescriptorBuilder
+
.newBuilder(NON_EC_TABLE).setColumnFamily(ColumnFamilyDescriptorBuilder.of(FAMILY)).build();
+ private static final TableName EC_TABLE = TableName.valueOf("bar");
+ private static final TableDescriptor EC_TABLE_DESC =
+
TableDescriptorBuilder.newBuilder(EC_TABLE).setErasureCodingPolicy("RS-6-3-1024k")
+ .setColumnFamily(ColumnFamilyDescriptorBuilder.of(FAMILY)).build();
+
+ @BeforeClass
+ public static void beforeClass() throws Exception {
+UTIL.startMiniDFSCluster(6); // 6 necessary for RS-6-3-1024k
Review Comment:
Comment not related to test:
Just wondering, what happens if some one mistakenly sets tis policy on a
cluster with less than 6 nodes? Can the table/system be recovered/fixed?
Also, do we have any checks to ensure ECP cannot be configured if this
condition is not satisfied? Is it worth adding?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429032871
##
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestManageTableErasureCodingPolicy.java:
##
@@ -0,0 +1,231 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.procedure;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.greaterThan;
+import static org.hamcrest.Matchers.notNullValue;
+import static org.hamcrest.Matchers.nullValue;
+import static org.junit.Assert.assertThrows;
+
+import java.io.IOException;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseIOException;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.io.hfile.HFile;
+import org.apache.hadoop.hbase.regionserver.CompactedHFilesDischarger;
+import org.apache.hadoop.hbase.regionserver.HRegion;
+import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.util.CommonFSUtils;
+import org.apache.hadoop.hbase.util.JVMClusterUtil;
+import org.apache.hadoop.hdfs.DistributedFileSystem;
+import org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+@Category({ MasterTests.class, MediumTests.class })
+public class TestManageTableErasureCodingPolicy {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestManageTableErasureCodingPolicy.class);
+
+ private static final HBaseTestingUtil UTIL = new HBaseTestingUtil();
+ private static final byte[] FAMILY = Bytes.toBytes("a");
+ private static final TableName NON_EC_TABLE = TableName.valueOf("foo");
+ private static final TableDescriptor NON_EC_TABLE_DESC =
TableDescriptorBuilder
+
.newBuilder(NON_EC_TABLE).setColumnFamily(ColumnFamilyDescriptorBuilder.of(FAMILY)).build();
+ private static final TableName EC_TABLE = TableName.valueOf("bar");
+ private static final TableDescriptor EC_TABLE_DESC =
+
TableDescriptorBuilder.newBuilder(EC_TABLE).setErasureCodingPolicy("RS-6-3-1024k")
+ .setColumnFamily(ColumnFamilyDescriptorBuilder.of(FAMILY)).build();
+
+ @BeforeClass
+ public static void beforeClass() throws Exception {
+UTIL.startMiniDFSCluster(6); // 6 necessary for RS-6-3-1024k
Review Comment:
Just wondering, what happens if some one mistakenly sets tis policy on a
cluster with less than 6 nodes? Can the table/system be recovered/fixed?
Also, do we have any checks to ensure ECP cannot be configured if this
condition is not satisfied? Is it worth adding?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429029998
##
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactedHFilesDischarger.java:
##
@@ -70,7 +70,7 @@ public CompactedHFilesDischarger(final int period, final
Stoppable stopper,
* no-executor before you call run.
* @return The old setting for useExecutor
*/
- boolean setUseExecutor(final boolean useExecutor) {
+ public boolean setUseExecutor(final boolean useExecutor) {
Review Comment:
I see this is used only in tests and hence made public, can we see if we can
add @ RestrictedApi and mention its usage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429028180
##
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java:
##
@@ -143,6 +143,14 @@ public class TableDescriptorBuilder {
private static final Bytes REGION_MEMSTORE_REPLICATION_KEY =
new Bytes(Bytes.toBytes(REGION_MEMSTORE_REPLICATION));
+ /**
+ * If non-null, the HDFS erasure coding policy to set on the data dir of the
table
+ */
+ public static final String ERASURE_CODING_POLICY = "ERASURE_CODING_POLICY";
Review Comment:
Could we please add a sanity check at
`hbase-server/src/main/java/org/apache/hadoop/hbase/util/TableDescriptorChecker.java`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429026847
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -512,6 +527,23 @@ private void
closeExcessReplicasIfNeeded(MasterProcedureEnv env) {
.createUnassignProceduresForClosingExcessRegionReplicas(getTableName(),
newReplicaCount));
}
+ private void syncErasureCodingPolicy(MasterProcedureEnv env) throws
IOException {
+String oldPolicy = unmodifiedTableDescriptor.getErasureCodingPolicy();
Review Comment:
The first 4 lines can be refactored into needSyncErasureCodingPolicy and
same can be used as if condition while deciding next state in the procedure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429026847
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -512,6 +527,23 @@ private void
closeExcessReplicasIfNeeded(MasterProcedureEnv env) {
.createUnassignProceduresForClosingExcessRegionReplicas(getTableName(),
newReplicaCount));
}
+ private void syncErasureCodingPolicy(MasterProcedureEnv env) throws
IOException {
+String oldPolicy = unmodifiedTableDescriptor.getErasureCodingPolicy();
Review Comment:
The first 4 lines can be refactored into neeSynccErasureCodingPolicy and
same can be used as if condition while deciding next state in the procedure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429026506
##
hbase-server/src/main/java/org/apache/hadoop/hbase/fs/ErasureCodingUtils.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.fs;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.stream.Collectors;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.HBaseIOException;
+import org.apache.hadoop.hdfs.DistributedFileSystem;
+import org.apache.hadoop.hdfs.protocol.ErasureCodingPolicyInfo;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@InterfaceAudience.Private
+public final class ErasureCodingUtils {
+
+ private ErasureCodingUtils() {
+ }
+
+ private static final Logger LOG =
LoggerFactory.getLogger(ErasureCodingUtils.class);
+
+ /**
+ * Runs checks against the FileSystem, verifying that HDFS is supported and
the policy is
+ * available and enabled.
+ */
+ public static void checkAvailable(FileSystem fs, String policy) throws
HBaseIOException {
+DistributedFileSystem dfs = getDfs(fs);
+
+Collection policies;
+try {
+ policies = dfs.getAllErasureCodingPolicies();
Review Comment:
Should we provide a utility or api to get a list of supported and available
policies which can be used before some one invokes set ECP? Or is it overkill?
Please let me know what do you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
NihalJain commented on code in PR #5579:
URL: https://github.com/apache/hbase/pull/5579#discussion_r1429025305
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java:
##
@@ -98,6 +99,15 @@ protected Flow executeFromState(final MasterProcedureEnv
env, final CreateTableS
DeleteTableProcedure.deleteFromFs(env, getTableName(), newRegions,
true);
newRegions = createFsLayout(env, tableDescriptor, newRegions);
env.getMasterServices().getTableDescriptors().update(tableDescriptor, true);
+
setNextState(CreateTableState.CREATE_TABLE_SYNC_ERASURE_CODING_POLICY);
+ break;
+case CREATE_TABLE_SYNC_ERASURE_CODING_POLICY:
+ if (tableDescriptor.getErasureCodingPolicy() != null) {
Review Comment:
The if condition should be moved to previous step where we check for this if
and set the next step to be either `CREATE_TABLE_SYNC_ERASURE_CODING_POLICY` or
`CREATE_TABLE_ADD_TO_META`. This will also ensure the extra step is never
called if ECP is not configured to the table.
We can do same in other procedures too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-20528) Revise collections copying from iteration to built-in function
[
https://issues.apache.org/jira/browse/HBASE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797874#comment-17797874
]
Hudson commented on HBASE-20528:
Results for branch branch-2.4
[build #668 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Revise collections copying from iteration to built-in function
> --
>
> Key: HBASE-20528
> URL: https://issues.apache.org/jira/browse/HBASE-20528
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1
>Reporter: Hua-Yi Ho
>Assignee: Hua-Yi Ho
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-20528.master.001.patch
>
>
> Some collection codes in file
> StochasticLoadBalancer.java, AbstractHBaseTool.java, HFileInputFormat.java,
> Result.java, and WalPlayer.java, using iterations to copy whole data in
> collections. The iterations can just replace by just Colletions.addAll and
> Arrays.copyOf.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-21243) Correct java-doc for the method RpcServer.getRemoteAddress()
[
https://issues.apache.org/jira/browse/HBASE-21243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797873#comment-17797873
]
Hudson commented on HBASE-21243:
Results for branch branch-2.4
[build #668 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/668/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Correct java-doc for the method RpcServer.getRemoteAddress()
>
>
> Key: HBASE-21243
> URL: https://issues.apache.org/jira/browse/HBASE-21243
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Trivial
> Labels: beginner, beginners, documentaion
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-21243.master.001.patch,
> HBASE-21243.master.002.patch
>
>
> Correct the java-doc for the method {{RpcServer.getRemoteAddress()}}.
> Currently it look like as below:
> {code:java}
> /**
>* @return Address of remote client if a request is ongoing, else null
>*/
> public static Optional getRemoteAddress() {
> return getCurrentCall().map(RpcCall::getRemoteAddress);
> }
> {code}
> Contrary to the doc the method will never return null. Rather it may return
> an empty Optional.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-20528) Revise collections copying from iteration to built-in function
[
https://issues.apache.org/jira/browse/HBASE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797872#comment-17797872
]
Hudson commented on HBASE-20528:
Results for branch branch-2.6
[build #13 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Revise collections copying from iteration to built-in function
> --
>
> Key: HBASE-20528
> URL: https://issues.apache.org/jira/browse/HBASE-20528
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1
>Reporter: Hua-Yi Ho
>Assignee: Hua-Yi Ho
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-20528.master.001.patch
>
>
> Some collection codes in file
> StochasticLoadBalancer.java, AbstractHBaseTool.java, HFileInputFormat.java,
> Result.java, and WalPlayer.java, using iterations to copy whole data in
> collections. The iterations can just replace by just Colletions.addAll and
> Arrays.copyOf.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-21243) Correct java-doc for the method RpcServer.getRemoteAddress()
[
https://issues.apache.org/jira/browse/HBASE-21243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797871#comment-17797871
]
Hudson commented on HBASE-21243:
Results for branch branch-2.6
[build #13 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Correct java-doc for the method RpcServer.getRemoteAddress()
>
>
> Key: HBASE-21243
> URL: https://issues.apache.org/jira/browse/HBASE-21243
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Trivial
> Labels: beginner, beginners, documentaion
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-21243.master.001.patch,
> HBASE-21243.master.002.patch
>
>
> Correct the java-doc for the method {{RpcServer.getRemoteAddress()}}.
> Currently it look like as below:
> {code:java}
> /**
>* @return Address of remote client if a request is ongoing, else null
>*/
> public static Optional getRemoteAddress() {
> return getCurrentCall().map(RpcCall::getRemoteAddress);
> }
> {code}
> Contrary to the doc the method will never return null. Rather it may return
> an empty Optional.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28180) Review the usage of RegionStates.getOrCreateServer
[
https://issues.apache.org/jira/browse/HBASE-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797870#comment-17797870
]
Hudson commented on HBASE-28180:
Results for branch branch-2.6
[build #13 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/13/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Review the usage of RegionStates.getOrCreateServer
> --
>
> Key: HBASE-28180
> URL: https://issues.apache.org/jira/browse/HBASE-28180
> Project: HBase
> Issue Type: Bug
> Components: amv2, master
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
> Attachments:
> org.apache.hadoop.hbase.master.TestClusterRestartFailover-output.txt
>
>
> It failed two times in this PR.
> https://github.com/apache/hbase/pull/5475
> Filed an issue to track this problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1859033192 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 45s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 51s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 5m 4s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotless | 0m 50s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 5m 54s | master passed | | -0 :warning: | patch | 2m 26s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 4m 22s | the patch passed | | +1 :green_heart: | cc | 4m 22s | the patch passed | | -0 :warning: | javac | 2m 42s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | +1 :green_heart: | checkstyle | 1m 10s | the patch passed | | -0 :warning: | rubocop | 0m 7s | The patch generated 4 new + 409 unchanged - 333 fixed = 413 total (was 742) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 15m 18s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | hbaseprotoc | 2m 6s | the patch passed | | +1 :green_heart: | spotless | 0m 58s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 7m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. | | | | 63m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 495a7cb4afbc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | rubocop | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 81 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 rubocop=1.37.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858945926 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 0m 31s | master passed | | +1 :green_heart: | checkstyle | 0m 12s | master passed | | +1 :green_heart: | spotless | 0m 54s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 33s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 8s | The patch does not generate ASF License warnings. | | | | 29m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 194b1714ac33 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 77 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858944173 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 0m 14s | master passed | | +1 :green_heart: | shadedjars | 5m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 0m 14s | the patch passed | | +1 :green_heart: | javac | 0m 14s | the patch passed | | +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 49s | hbase-http in the patch passed. | | | | 18m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d895613670d5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/testReport/ | | Max. process+thread count | 265 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858943916 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 19s | master passed | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 4m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 4m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-http in the patch passed. | | | | 16m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 569d75ebd360 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/testReport/ | | Max. process+thread count | 365 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
NihalJain commented on code in PR #5586:
URL: https://github.com/apache/hbase/pull/5586#discussion_r1428947843
##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -770,6 +766,37 @@ protected void addDefaultApps(ContextHandlerCollection
parent, final String appD
defaultContexts.put(staticContext, true);
}
+ /**
+ * This method configures the alias checks for the given
ServletContextHandler based on the
+ * provided value of shouldServeAlias.
+ * If the configuration allows serving aliases, it checks if
SymlinkAllowedResourceAliasChecker is
+ * already a part of the alias check list.
+ * If not, it adds it to the list. If it is already a part of the list, no
changes are made.
+ * If the configuration does not allow serving aliases, it clears all alias
checks from the
+ * ServletContextHandler.
+ * .
+ * @param context The ServletContextHandler whose alias checks are
to be configured
+ * @param shouldServeAlias Whether aliases should be allowed or not
+ */
+ private void configureAliasChecks(ServletContextHandler context, boolean
shouldServeAlias) {
+if (shouldServeAlias) {
+ Class aliasCheckerClass = SymlinkAllowedResourceAliasChecker.class;
+ // check if SymlinkAllowedResourceAliasChecker is already part of alias
check list
+ // NOTE: we are doing this because by default this is already present in
the context
+ if
(!context.getAliasChecks().stream().anyMatch(aliasCheckerClass::isInstance)) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
NihalJain commented on code in PR #5586:
URL: https://github.com/apache/hbase/pull/5586#discussion_r1428946667
##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:
##
@@ -770,6 +766,37 @@ protected void addDefaultApps(ContextHandlerCollection
parent, final String appD
defaultContexts.put(staticContext, true);
}
+ /**
+ * This method configures the alias checks for the given
ServletContextHandler based on the
+ * provided value of shouldServeAlias.
+ * If the configuration allows serving aliases, it checks if
SymlinkAllowedResourceAliasChecker is
+ * already a part of the alias check list.
+ * If not, it adds it to the list. If it is already a part of the list, no
changes are made.
+ * If the configuration does not allow serving aliases, it clears all alias
checks from the
+ * ServletContextHandler.
+ * .
+ * @param context The ServletContextHandler whose alias checks are
to be configured
+ * @param shouldServeAlias Whether aliases should be allowed or not
+ */
+ private void configureAliasChecks(ServletContextHandler context, boolean
shouldServeAlias) {
+if (shouldServeAlias) {
+ Class aliasCheckerClass = SymlinkAllowedResourceAliasChecker.class;
+ // check if SymlinkAllowedResourceAliasChecker is already part of alias
check list
+ // NOTE: we are doing this because by default this is already present in
the context
+ if
(!context.getAliasChecks().stream().anyMatch(aliasCheckerClass::isInstance)) {
Review Comment:
simplify and switch IF/ELSE conditions check by removing NOT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858938139 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 10s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 8s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 48s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | checkstyle | 0m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 38s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 29m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 3e5d68fe8173 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858936643 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 48s | master passed | | +1 :green_heart: | compile | 0m 14s | master passed | | +1 :green_heart: | shadedjars | 5m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 0m 14s | the patch passed | | +1 :green_heart: | javac | 0m 14s | the patch passed | | +1 :green_heart: | shadedjars | 5m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 49s | hbase-http in the patch passed. | | | | 20m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a3824748be38 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/testReport/ | | Max. process+thread count | 343 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
Apache-HBase commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858936163 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 35s | master passed | | +1 :green_heart: | compile | 0m 14s | master passed | | +1 :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 16s | the patch passed | | +1 :green_heart: | javac | 0m 16s | the patch passed | | +1 :green_heart: | shadedjars | 4m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-http in the patch passed. | | | | 17m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5586 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 60d2c420eba2 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/testReport/ | | Max. process+thread count | 357 (vs. ulimit of 3) | | modules | C: hbase-http U: hbase-http | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5586/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-21500) "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version
[
https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797501#comment-17797501
]
Nihal Jain commented on HBASE-21500:
Copy pasted RCA from github PR:
Changing the property name to {{org.eclipse.jetty.servlet.Default.aliases}}
does not solve the problem.
Upon investigating found that in
[Jetty#ContextHandler|https://github.com/jetty/jetty.project/blob/d49f298c7f5361161cd71264aabd50dbfc8e4c59/jetty-server/src/main/java/org/eclipse/jetty/server/handler/ContextHandler.java],
{{SymlinkAllowedResourceAliasChecker}} is added by default, irrespective of
whether {{hbase.jetty.logs.serve.aliases}} is set to {{true}} or {{{}false{}}},
allowing aliases for {{/logs}} always. Hence issue is revealed only if the
value is set to {{{}false{}}}.
See
[serving-aliased-files|https://github.com/jetty/jetty.project/blob/jetty-9.4.53.v20231009/jetty-documentation/src/main/asciidoc/configuring/security/serving-aliased-files.adoc]
for how this feature works.
Based on the docs, I have created a patch with fix.
To test the fix following steps were followed:
# Disallowed flow
* Build code with {{assembly:single}}
* Untar the tarball
* Configure {{hbase.jetty.logs.serve.aliases}} to {{false}} and start hbase in
local mode
hbase.jetty.logs.serve.aliases
false
* Goto logs directory and run following:
** {{touch /tmp/test.txt test.txt}}
** {{ln -s /tmp/test.txt test.txt}}
* Goto {{http://localhost:16010/logs/test.txt}} and a 404 error should be
thrown. See
[!https://private-user-images.githubusercontent.com/3429351/291016234-12ad9b92-3808-40b0-a9b3-d135ce156623.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDI3NjMxNzMsIm5iZiI6MTcwMjc2Mjg3MywicGF0aCI6Ii8zNDI5MzUxLzI5MTAxNjIzNC0xMmFkOWI5Mi0zODA4LTQwYjAtYTliMy1kMTM1Y2UxNTY2MjMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQUlXTkpZQVg0Q1NWRUg1M0ElMkYyMDIzMTIxNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyMzEyMTZUMjE0MTEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ODgzODY2NjljNGNiYzk4YTVjMjY2ODMzN2EyMmI3ZTQ0ZDdmMGI1YWRiMGQ3NDQ5MzMxNDcxOWUxYzI3MjBmNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.biyQngECGTa9eHMuY46JhVF6dP3cd_lfSK7v-stgjdo|width=556!|https://private-user-images.githubusercontent.com/3429351/291016234-12ad9b92-3808-40b0-a9b3-d135ce156623.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDI3NjMxNzMsIm5iZiI6MTcwMjc2Mjg3MywicGF0aCI6Ii8zNDI5MzUxLzI5MTAxNjIzNC0xMmFkOWI5Mi0zODA4LTQwYjAtYTliMy1kMTM1Y2UxNTY2MjMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQUlXTkpZQVg0Q1NWRUg1M0ElMkYyMDIzMTIxNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyMzEyMTZUMjE0MTEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ODgzODY2NjljNGNiYzk4YTVjMjY2ODMzN2EyMmI3ZTQ0ZDdmMGI1YWRiMGQ3NDQ5MzMxNDcxOWUxYzI3MjBmNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.biyQngECGTa9eHMuY46JhVF6dP3cd_lfSK7v-stgjdo]
# Allowed flow, which is current default behavior and works even without the
patch
* Repeat above steps {{hbase.jetty.logs.serve.aliases}} to {{true}} or just
remove it.
* We should be able to access symlinked file test.txt
> "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version
> -
>
> Key: HBASE-21500
> URL: https://issues.apache.org/jira/browse/HBASE-21500
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0, 2.0.1, 2.1.1
>Reporter: Bhupendra Kumar Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-21500.master.001.patch
>
>
> Noticed that Jetty aliases parameter in HttpServer.java
> "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and
> need to change as per jetty 9.x new version after the HBASE-12894
> Refer
> https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647
> It should be *"org.eclipse.jetty.servlet.Default.aliases"*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-21500) "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-21500: --- Status: Patch Available (was: Open) > "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version > - > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.1, 2.0.1, 2.0.0, 2.1.0, 3.0.0-alpha-1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
NihalJain commented on PR #5586: URL: https://github.com/apache/hbase/pull/5586#issuecomment-1858935387 Changing the property name to `org.eclipse.jetty.servlet.Default.aliases` does not solve the problem. Upon investigating found that in [Jetty#ContextHandler](https://github.com/jetty/jetty.project/blob/d49f298c7f5361161cd71264aabd50dbfc8e4c59/jetty-server/src/main/java/org/eclipse/jetty/server/handler/ContextHandler.java), `SymlinkAllowedResourceAliasChecker` is added by default, irrespective of whether `hbase.jetty.logs.serve.aliases` is set to true or false, allowing aliases for /logs always. Hence issue is revealed only if the value is set to `false`. See [serving-aliased-files](https://github.com/jetty/jetty.project/blob/jetty-9.4.53.v20231009/jetty-documentation/src/main/asciidoc/configuring/security/serving-aliased-files.adoc) for how this feature works. Based on the docs, I have created a patch with fix. To test the fix following steps were followed: 1. Disallowed flow - Build code with `assembly:single` - Untar the tarball - Configure `hbase.jetty.logs.serve.aliases` to `false` and start hbase in local mode ``` hbase.jetty.logs.serve.aliases false ``` - Goto logs directory and run following: - `touch /tmp/test.txt test.txt` - `ln -s /tmp/test.txt test.txt` - Goto `http://localhost:16010/logs/test.txt` and a 404 error should be thrown. See https://github.com/apache/hbase/assets/3429351/12ad9b92-3808-40b0-a9b3-d135ce156623;> 2. Allowed flow, which is current default behavior and works even without the patch - Repeat above steps `hbase.jetty.logs.serve.aliases` to `true` or just remove it. - We should be able to access symlinked file test.txt CC: @Apache9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] HBASE-21500 "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x… [hbase]
NihalJain opened a new pull request, #5586: URL: https://github.com/apache/hbase/pull/5586 … version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-21500) "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-21500: --- Summary: "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version (was: Jetty aliases parameter need to change as per jetty 9.x version) > "hbase.jetty.logs.serve.aliases" is broken with jetty 9.x version > - > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0, 2.0.1, 2.1.1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28031) TestClusterScopeQuotaThrottle is still failing with broken WAL writer
[
https://issues.apache.org/jira/browse/HBASE-28031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797489#comment-17797489
]
Hudson commented on HBASE-28031:
Results for branch branch-3
[build #104 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/104/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/104/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/104/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/104/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestClusterScopeQuotaThrottle is still failing with broken WAL writer
> -
>
> Key: HBASE-28031
> URL: https://issues.apache.org/jira/browse/HBASE-28031
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
>
> {noformat}
> 2023-08-17T10:47:31,026 WARN [regionserver/jenkins-hbase19:0.logRoller {}]
> asyncfs.FanOutOneBlockAsyncDFSOutputHelper(515): create fan-out dfs output
> /user/jenkins/test-data/bb8017fa-92f5-92c9-2f1d-aa9b90cf4b80/WALs/jenkins-hbase19.apache.org,43363,1692269230784/jenkins-hbase19.apache.org%2C43363%2C1692269230784.meta.1692433272886.meta
> failed, retry = 0
> org.apache.hadoop.ipc.RemoteException: File
> /user/jenkins/test-data/bb8017fa-92f5-92c9-2f1d-aa9b90cf4b80/WALs/jenkins-hbase19.apache.org,43363,1692269230784/jenkins-hbase19.apache.org%2C43363%2C1692269230784.meta.1692433272886.meta
> could only be written to 0 of the 1 minReplication nodes. There are 2
> datanode(s) running and 2 node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2276)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:294)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2820)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:910)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:577)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:549)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:518)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1086)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1035)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2960)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1612)
> ~[hadoop-common-3.2.4.jar:?]
> at org.apache.hadoop.ipc.Client.call(Client.java:1558)
> ~[hadoop-common-3.2.4.jar:?]
> at org.apache.hadoop.ipc.Client.call(Client.java:1455)
> ~[hadoop-common-3.2.4.jar:?]
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
> ~[hadoop-common-3.2.4.jar:?]
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
> ~[hadoop-common-3.2.4.jar:?]
> at com.sun.proxy.$Proxy41.addBlock(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:520)
> ~[hadoop-hdfs-client-3.2.4.jar:?]
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_362]
> at java.lang.reflect.Method.invoke(Method.java:498)
[jira] [Commented] (HBASE-28180) Review the usage of RegionStates.getOrCreateServer
[
https://issues.apache.org/jira/browse/HBASE-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797488#comment-17797488
]
Hudson commented on HBASE-28180:
Results for branch master
[build #965 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/965/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/965/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/965/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/965/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Review the usage of RegionStates.getOrCreateServer
> --
>
> Key: HBASE-28180
> URL: https://issues.apache.org/jira/browse/HBASE-28180
> Project: HBase
> Issue Type: Bug
> Components: amv2, master
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
> Attachments:
> org.apache.hadoop.hbase.master.TestClusterRestartFailover-output.txt
>
>
> It failed two times in this PR.
> https://github.com/apache/hbase/pull/5475
> Filed an issue to track this problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28267) create-release should run spotless
Andrew Kyle Purtell created HBASE-28267: --- Summary: create-release should run spotless Key: HBASE-28267 URL: https://issues.apache.org/jira/browse/HBASE-28267 Project: HBase Issue Type: Task Reporter: Andrew Kyle Purtell Before committing generated files like CHANGES.md and RELEASENOTES.md we should run 'mvn spotless:apply' first to ensure what is committed is formatted per our rules and will not be modified when someone invokes spotless later. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28262) Fix spotless error on branch-2.5
[ https://issues.apache.org/jira/browse/HBASE-28262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-28262: Fix Version/s: 2.5.8 Assignee: Duo Zhang Status: Patch Available (was: Open) > Fix spotless error on branch-2.5 > > > Key: HBASE-28262 > URL: https://issues.apache.org/jira/browse/HBASE-28262 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.5.8 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28262) Fix spotless error on branch-2.5
[ https://issues.apache.org/jira/browse/HBASE-28262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797468#comment-17797468 ] Andrew Kyle Purtell edited comment on HBASE-28262 at 12/16/23 6:33 PM: --- Yes. Filed HBASE-28267 was (Author: apurtell): Yes. > Fix spotless error on branch-2.5 > > > Key: HBASE-28262 > URL: https://issues.apache.org/jira/browse/HBASE-28262 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28262) Fix spotless error on branch-2.5
[ https://issues.apache.org/jira/browse/HBASE-28262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797468#comment-17797468 ] Andrew Kyle Purtell commented on HBASE-28262: - Yes. > Fix spotless error on branch-2.5 > > > Key: HBASE-28262 > URL: https://issues.apache.org/jira/browse/HBASE-28262 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27943 Rework kuttl image as a yetus precommit base [hbase-kustomize]
Apache-HBase commented on PR #5:
URL: https://github.com/apache/hbase-kustomize/pull/5#issuecomment-1858851480
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ main Compile Tests _ |
_ Patch Compile Tests _ |
| +1 :green_heart: | codespell | 0m 0s | | No new issues. |
| +1 :green_heart: | detsecrets | 0m 2s | | No new issues. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 0s | | No new issues. |
| +1 :green_heart: | markdownlint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shelldocs | 0m 0s | | No new issues. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 1s | | The patch does not
generate ASF License warnings. |
| | | 0m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| GITHUB PR | https://github.com/apache/hbase-kustomize/pull/5 |
| Optional Tests | dupname asflicense codespell detsecrets shellcheck
shelldocs hadolint markdownlint |
| uname | Linux 8777e688f4bf 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality | /workdir/src/.yetus/personality.sh |
| git revision | main / daa11d3a9a00c94463cc9aef2cf6371497e5d3ad |
| modules | C: . U: . |
| versions | git=2.25.1 codespell=2.2.1 detsecrets=1.2.0 markdownlint=0.31.1
shellcheck=0.8.0 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-27943) Rework kuttl image as a yetus precommit base
[ https://issues.apache.org/jira/browse/HBASE-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-27943: - Status: Patch Available (was: In Progress) > Rework kuttl image as a yetus precommit base > > > Key: HBASE-27943 > URL: https://issues.apache.org/jira/browse/HBASE-27943 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Due to permissions issues on build workers (INFRA-24621), I think we'll > benefit by shifting the perspective a little on how the kuttl image is used > in CI. The current PRs establish the kuttl image as a (relatively) small, > self-contained utility image. When it is invoked in CI, we run yetus with > docker-in-docker support, and launch tests by calling `docker container run > ... kuttl ...`. So, docker is invoked from within the Yetus pre-commit docker > container. Rather, I want to implement the kuttl image as extending from the > yetus image (or yetus-base, I'm not sure yet). That way we don't need to run > pre-commit with docker-in-docker mode and the precommit process can invoke > kuttl directly. > The existing kuttl image build is too sophisticated to be invoked by Yetus as > a provided docker image (it makes use of build-args). It may also be too > sophisticated to be run in a Jenkins worker (it makes use of buildx, > INFRA-24704). So I guess we need to publish the image someplace and use the > published tag via the Yetus `--docker-tag` flag. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
Apache-HBase commented on PR #5585: URL: https://github.com/apache/hbase/pull/5585#issuecomment-1858843172 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | javadoc | 0m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 6m 53s | hbase-shell in the patch passed. | | | | 16m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5585 | | Optional Tests | javac javadoc unit | | uname | Linux cc1bc9bd7a55 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/testReport/ | | Max. process+thread count | 1525 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
Apache-HBase commented on PR #5585: URL: https://github.com/apache/hbase/pull/5585#issuecomment-1858843083 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 59s | master passed | | +1 :green_heart: | javadoc | 0m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | javadoc | 0m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 6m 44s | hbase-shell in the patch passed. | | | | 15m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5585 | | Optional Tests | javac javadoc unit | | uname | Linux 1a1668a49eb5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/testReport/ | | Max. process+thread count | 1585 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
Apache-HBase commented on PR #5585: URL: https://github.com/apache/hbase/pull/5585#issuecomment-1858842243 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 5s | master passed | | +1 :green_heart: | spotless | 0m 45s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 52s | the patch passed | | -0 :warning: | rubocop | 0m 19s | The patch generated 200 new + 395 unchanged - 12 fixed = 595 total (was 407) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 11m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5585 | | Optional Tests | dupname asflicense javac spotless rubocop | | uname | Linux 0cb042f9f2ce 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b7c3f8c6e3 | | Default Java | Eclipse Adoptium-11.0.17+8 | | rubocop | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5585/1/console | | versions | git=2.34.1 maven=3.8.6 rubocop=1.37.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
NihalJain commented on code in PR #5585:
URL: https://github.com/apache/hbase/pull/5585#discussion_r1428834651
##
hbase-shell/src/main/ruby/shell/commands/put.rb:
##
@@ -37,16 +37,50 @@ def help
t to table 't1', the corresponding command would be:
hbase> t.put 'r1', 'c1', 'value', ts1, {ATTRIBUTES=>{'mykey'=>'myvalue'}}
+
+Alternately, we can put cell values for multiple columns at specified
table/row and
+optionally timestamp coordinates.
+
+ hbase> put 'ns1:t1', 'r1', {'c1'=>'value1', 'c2'=>'value2'}
+ hbase> put 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'}
+ hbase> put 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'}, ts1
+ hbase> put 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'},
{ATTRIBUTES=>{'mykey'=>'myvalue'}}
+ hbase> put 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'}, ts1,
{ATTRIBUTES=>{'mykey'=>'myvalue'}}
+ hbase> put 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'}, ts1,
{VISIBILITY=>'PRIVATE|SECRET'}
+
+The same commands also can be run on a table reference.
+
+ hbase> t.put 'r1', {'c1'=>'value1', 'c2'=>'value2'}, ts1,
{ATTRIBUTES=>{'mykey'=>'myvalue'}}
EOF
end
- def command(table, row, column, value, timestamp = nil, args = {})
-put table(table), row, column, value, timestamp, args
+ def command(table, row, column, value = value_omitted = {}, timestamp =
nil, args = args_omitted = {})
+# Conditional block to omit passing optional arguments explicitly
Review Comment:
improve docs here as this is very cryptic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
NihalJain commented on code in PR #5585:
URL: https://github.com/apache/hbase/pull/5585#discussion_r1428834524
##
hbase-shell/src/main/ruby/shell/commands/append.rb:
##
@@ -32,18 +32,49 @@ def help
hbase> t.append 'r1', 'c1', 'value', ATTRIBUTES=>{'mykey'=>'myvalue'}
hbase> t.append 'r1', 'c1', 'value', {VISIBILITY=>'PRIVATE|SECRET'}
+
+Alternately, we can run the following commands for appending cell values for
+multiple columns at specified table/row coordinates.
+
+ hbase> append 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'},
{ATTRIBUTES=>{'mykey'=>'myvalue'}}
+ hbase> append 't1', 'r1', {'c1'=>'value1', 'c2'=>'value2'},
{VISIBILITY=>'PRIVATE|SECRET'}
+
+The same commands also can be run on a table reference.
+
+ hbase> t.append 'r1', {'c1'=>'value1', 'c2'=>'value2'},
{ATTRIBUTES=>{'mykey'=>'myvalue'}}
+ hbase> t.append 'r1', {'c1'=>'value1', 'c2'=>'value2'},
{VISIBILITY=>'PRIVATE|SECRET'}
+
EOF
end
- def command(table_name, row, column, value, args = {})
+ def command(table_name, row, column, value = value_omitted = {}, args =
args_omitted = {})
table = table(table_name)
@start_time = Time.now
-append(table, row, column, value, args)
+# Conditional block to omit passing optional arguments explicitly
Review Comment:
improve docs here as this is very cryptic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
NihalJain commented on PR #5585: URL: https://github.com/apache/hbase/pull/5585#issuecomment-1858838669 Draft PR with changes from the last submitted patch in HBASE-18999. I have run hbase build with this change along with the shell test cases. Pending: * Will see if code can be improved. * And also test out using a shell instance. CC: @Apache9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] HBASE-18999 Shell support for multiple columns for DML commands [hbase]
NihalJain opened a new pull request, #5585: URL: https://github.com/apache/hbase/pull/5585 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-27943 Rework kuttl image as a yetus precommit base [hbase-kustomize]
Apache-HBase commented on PR #5:
URL: https://github.com/apache/hbase-kustomize/pull/5#issuecomment-1858837710
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ main Compile Tests _ |
| -0 :warning: | patch | 0m 0s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | codespell | 0m 0s | | No new issues. |
| +1 :green_heart: | detsecrets | 0m 2s | | No new issues. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 0s | | No new issues. |
| +1 :green_heart: | markdownlint | 0m 1s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shelldocs | 0m 0s | | No new issues. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 1s | | The patch does not
generate ASF License warnings. |
| | | 0m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| GITHUB PR | https://github.com/apache/hbase-kustomize/pull/5 |
| Optional Tests | dupname asflicense codespell detsecrets shellcheck
shelldocs hadolint markdownlint |
| uname | Linux 37c5cca7a83d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | nobuild |
| Personality | /workdir/src/.yetus/personality.sh |
| git revision | main / f54044e8b561d68664d7d3fed07e184d514960c1 |
| modules | C: . U: . |
| versions | git=2.25.1 codespell=2.2.1 detsecrets=1.2.0 markdownlint=0.31.1
shellcheck=0.8.0 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-21500) Jetty aliases parameter need to change as per jetty 9.x version
[
https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797447#comment-17797447
]
Nihal Jain commented on HBASE-21500:
Sure [~zhangduo] let me do it. AFAIR, we were using this property in one of my
previous orgs to allow creating symlinks for resources on hbase ui, for
examples logs can point to a different folder via symlink only if this property
is enabled.
Also this is what chatgpt says about the property:
{quote}The {{org.eclipse.jetty.servlet.Default.aliases}} is a configuration
setting in Jetty, a Java HTTP (Web) server and Java Servlet container. This
setting controls whether Jetty allows the use of aliases in its handling of
static content.
If you set {{org.eclipse.jetty.servlet.Default.aliases}} to {{{}true{}}}, Jetty
will allow the use of symbolic links and aliases when serving static content.
This can be useful if your static content is spread across different
directories or even different systems, and you want to use symbolic links to
make it appear as if all the content is in one place.
If you set {{org.eclipse.jetty.servlet.Default.aliases}} to {{{}false{}}},
Jetty will not allow the use of symbolic links and aliases. This can be more
secure, as it ensures that Jetty can only serve content from the directories
you have explicitly specified.
By default, this setting is {{{}false{}}}.
{quote}
Let me put up a patch and also test out if this is how the property works.
> Jetty aliases parameter need to change as per jetty 9.x version
> ---
>
> Key: HBASE-21500
> URL: https://issues.apache.org/jira/browse/HBASE-21500
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0, 2.0.1, 2.1.1
>Reporter: Bhupendra Kumar Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-21500.master.001.patch
>
>
> Noticed that Jetty aliases parameter in HttpServer.java
> "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and
> need to change as per jetty 9.x new version after the HBASE-12894
> Refer
> https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647
> It should be *"org.eclipse.jetty.servlet.Default.aliases"*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20693: --- Status: Open (was: Patch Available) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-20693.master.001.patch, rest_home_after_patch.png, > thrift_home_after_patch.png, thrift_log_level_after_patch.png, > thrift_log_level_before_patch.png > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' from them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-18999) Shell support for multiple columns for DML commands
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nihal Jain updated HBASE-18999:
---
Status: Open (was: Patch Available)
> Shell support for multiple columns for DML commands
>
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 2.0.0, 3.0.0-alpha-1, 1.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-21500) Jetty aliases parameter need to change as per jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-21500: --- Status: Open (was: Patch Available) > Jetty aliases parameter need to change as per jetty 9.x version > --- > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.1, 2.0.1, 2.0.0, 2.1.0, 3.0.0-alpha-1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27635) Shutdown zookeeper logs coming via ReadOnlyZKClient when hbase shell started
[
https://issues.apache.org/jira/browse/HBASE-27635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27635:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Shutdown zookeeper logs coming via ReadOnlyZKClient when hbase shell started
>
>
> Key: HBASE-27635
> URL: https://issues.apache.org/jira/browse/HBASE-27635
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 2.6.0, 2.5.8, 3.0.0-beta-2
>
>
> When hbase shell with HBase 2.5.2 started there is too much logging of zk
> connection realated, classpaths etc. Even though we enabled ERROR log level
> for zookeeper package.
> {noformat}
> 2023-02-10 17:34:25,211 INFO
> [ReadOnlyZKClient-host1:2181,host2:2181,host3:2181@0x15c16f19]
> zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.5.9-5-a433770fc7b303332f10174221799495a26bbca2,
> built on 02/07/2023 13:02 GMT
> 2023-02-10 17:34:25,212 INFO
> [ReadOnlyZKClient-host1:2181,host2:2181,host3:2181@0x15c16f19]
> zookeeper.ZooKeeper: Client environment:host.name=host1
> 2023-02-10 17:34:25,212 INFO
> [ReadOnlyZKClient-host1:2181,host2:2181,host3:2181:2181@0x15c16f19]
> zookeeper.ZooKeeper: Client environment:java.version=1.8.0_352
> 2023-02-10 17:34:25,212 INFO
> [ReadOnlyZKClient-host1:2181,host2:2181,host3:2181@0x15c16f19]
> zookeeper.ZooKeeper: Client environment:java.vendor=Red Hat, Inc.
> 2023-02-10 17:34:25,212 INFO
> [ReadOnlyZKClient-host1:2181,host2:2181,host3:2181@0x15c16f19]
> zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.352.b08-2.el7_9.x86_64/jre
> {noformat}
> Better to change the org.apache.hadoop.hbase.zookeeper package log level to
> error.
> {noformat}
> # Set logging level to avoid verboseness
> org.apache.logging.log4j.core.config.Configurator.setAllLevels('org.apache.zookeeper',
> log_level)
> org.apache.logging.log4j.core.config.Configurator.setAllLevels('org.apache.hadoop',
> log_level)
> org.apache.logging.log4j.core.config.Configurator.setAllLevels('org.apache.hadoop.hbase.zookeeper',
> log_level)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26322) Add rsgroup support for Backup
[ https://issues.apache.org/jira/browse/HBASE-26322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797446#comment-17797446 ] Duo Zhang commented on HBASE-26322: --- [~bbeaudreault] FYI, another backup/restore improvement. > Add rsgroup support for Backup > -- > > Key: HBASE-26322 > URL: https://issues.apache.org/jira/browse/HBASE-26322 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Minor > Fix For: 3.0.0-beta-2 > > > There are some places where backup needs some changes with respect to > rsgroup. Some of them being addressed here are > # Incremental backup wal backup should happen only for servers which are > part of a particular rsgroup under which namespace is configured for table > backup under consideration > # BackupLogCleaner should keep references only from those servers which are > part of a particular rsgroup under which namesapce is configured for table > backup under consideration -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-26322) Add rsgroup support for Backup
[ https://issues.apache.org/jira/browse/HBASE-26322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26322: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Add rsgroup support for Backup > -- > > Key: HBASE-26322 > URL: https://issues.apache.org/jira/browse/HBASE-26322 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Minor > Fix For: 3.0.0-beta-2 > > > There are some places where backup needs some changes with respect to > rsgroup. Some of them being addressed here are > # Incremental backup wal backup should happen only for servers which are > part of a particular rsgroup under which namespace is configured for table > backup under consideration > # BackupLogCleaner should keep references only from those servers which are > part of a particular rsgroup under which namesapce is configured for table > backup under consideration -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-26192:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-beta-2
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797445#comment-17797445
]
Duo Zhang commented on HBASE-26192:
---
Any updates here?
Thanks.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-beta-1
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-25540) ArrayIndexOutOfBoundsException thrown when table CF name is "#"
[
https://issues.apache.org/jira/browse/HBASE-25540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-25540:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> ArrayIndexOutOfBoundsException thrown when table CF name is "#"
> ---
>
> Key: HBASE-25540
> URL: https://issues.apache.org/jira/browse/HBASE-25540
> Project: HBase
> Issue Type: Bug
> Components: metrics
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Minor
> Fix For: 3.0.0-beta-2
>
>
> ArrayIndexOutOfBoundsException will be thrown when CF name is "#",
> https://github.com/apache/hbase/blob/a04ea7ea4493f5bc583b4d08a2a6a88e7c6b8c54/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSourceImpl.java#L340
> {noformat}
> 2021-01-30 00:11:14,172 | ERROR | HBase-Metrics2-1 | Error getting metrics
> from source RegionServer,sub=Tables |
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:202)
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.addGauge(MetricsTableSourceImpl.java:336)
> at
> org.apache.hadoop.hbase.regionserver.MetricsTableSourceImpl.snapshot(MetricsTableSourceImpl.java:321)
> at
> org.apache.hadoop.hbase.regionserver.MetricsTableAggregateSourceImpl.getMetrics(MetricsTableAggregateSourceImpl.java:98)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:200)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:183)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:156)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:333)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:319)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
> at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:67)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:222)
> at
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:101)
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:268)
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:239)
> at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:324)
> at com.sun.proxy.$Proxy7.postStart(Unknown Source)
> at
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:193)
> at
> org.apache.hadoop.metrics2.impl.JmxCacheBuster$JmxCacheBusterRunnable.run(JmxCacheBuster.java:109)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-25878) Fix various formatting issues in the refrence guide book
[ https://issues.apache.org/jira/browse/HBASE-25878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25878: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Fix various formatting issues in the refrence guide book > > > Key: HBASE-25878 > URL: https://issues.apache.org/jira/browse/HBASE-25878 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Che Xun >Assignee: Che Xun >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: image-2021-05-11-22-55-13-810.png > > > This jira is used to collect and fix various formatting issues in the > refrence guide book. > For example,in the first paragraph of the chapter 8.1，the bulleted list is > not displayed correctly，as show below： > !image-2021-05-11-22-55-13-810.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-25540 ArrayIndexOutOfBoundsException thrown when table CF name is "#" [hbase]
Apache9 commented on PR #3005: URL: https://github.com/apache/hbase/pull/3005#issuecomment-1858829613 Any updates here? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25402) HBase UI: sorting order by start key or end key is not considering empty start key/end key
[ https://issues.apache.org/jira/browse/HBASE-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25402: -- Fix Version/s: 3.0.0-alpha-1 (was: 3.0.0-alpha-4) > HBase UI: sorting order by start key or end key is not considering empty > start key/end key > -- > > Key: HBASE-25402 > URL: https://issues.apache.org/jira/browse/HBASE-25402 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.2.3 >Reporter: Ajeet Rai >Assignee: Akshay Sudheer >Priority: Minor > Fix For: 3.0.0-alpha-1 > > Attachments: HBASE-25402 Fix.png, HBASE-25402 Issue.png > > > HBase UI: sorting order by start key or end key is not considering empty > start key/end key. > Steps: > 1: Open any table page having multiple regions > 2: click on sort button of start key/end key > 3: Observed that sorting doesn't consider start and end key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-25402) HBase UI: sorting order by start key or end key is not considering empty start key/end key
[ https://issues.apache.org/jira/browse/HBASE-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25402: -- Fix Version/s: 3.0.0-alpha-4 (was: 3.0.0-beta-1) > HBase UI: sorting order by start key or end key is not considering empty > start key/end key > -- > > Key: HBASE-25402 > URL: https://issues.apache.org/jira/browse/HBASE-25402 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.2.3 >Reporter: Ajeet Rai >Assignee: Akshay Sudheer >Priority: Minor > Fix For: 3.0.0-alpha-4 > > Attachments: HBASE-25402 Fix.png, HBASE-25402 Issue.png > > > HBase UI: sorting order by start key or end key is not considering empty > start key/end key. > Steps: > 1: Open any table page having multiple regions > 2: click on sort button of start key/end key > 3: Observed that sorting doesn't consider start and end key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-25402) HBase UI: sorting order by start key or end key is not considering empty start key/end key
[ https://issues.apache.org/jira/browse/HBASE-25402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25402: -- Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Please file a new issue for backporting as this has been merged to master branch long ago. > HBase UI: sorting order by start key or end key is not considering empty > start key/end key > -- > > Key: HBASE-25402 > URL: https://issues.apache.org/jira/browse/HBASE-25402 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 2.2.3 >Reporter: Ajeet Rai >Assignee: Akshay Sudheer >Priority: Minor > Fix For: 3.0.0-beta-1 > > Attachments: HBASE-25402 Fix.png, HBASE-25402 Issue.png > > > HBase UI: sorting order by start key or end key is not considering empty > start key/end key. > Steps: > 1: Open any table page having multiple regions > 2: click on sort button of start key/end key > 3: Observed that sorting doesn't consider start and end key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-23656) [MERGETOOL] HBASE Support Merge region by pattern
[ https://issues.apache.org/jira/browse/HBASE-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23656: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > [MERGETOOL] HBASE Support Merge region by pattern > - > > Key: HBASE-23656 > URL: https://issues.apache.org/jira/browse/HBASE-23656 > Project: HBase > Issue Type: New Feature > Components: master >Reporter: zhengsicheng >Assignee: zhengsicheng >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-23656.branch-1.4.v1.patch, > HBASE-23656.master.v1.patch > > > Design Objective: > # Merge empty region > # Neat region > # merge expired region > Usage: bin/hbase onlinemerge [--tableName=] [--startRegion=] [--stopRegion=] > [--maxRegionSize=] [--maxRegionCreateTime=] [--numMaxMergePlans=] > [--targetRegionCount=] [--printExecutionPlan=] [--configMergePauseTime=] > Options: > --h or --h print help > --tableName table name must be not null > --startRegion start region > --stopRegion stop region > --maxRegionSize max region size Unit GB > --maxRegionCreateTime max Region Create Time /MM/dd HH:mm:ss > --numMaxMergePlans num MaxMerge Plans > --targetRegionCount target Region Count > --configMergePauseTime config Merge Pause Time In milliseconds > --printExecutionPlan Value default is true print execution plans false is > execution merge > Examples: > bin/hbase onlinemerge --tableName=test:test1 > --startRegion=test:test1,,1576835912332.01d0d6c2b41e204104524d9aec6074fb. > --stopRegion=test:test1,,1573044786980.0c9b5bd93f3b19eb9bd1a1011ddff66f. > --maxRegionSize=0 --maxRegionCreateTime=/MM/dd HH:mm:ss > --numMaxMergePlans=2 --targetRegionCount=4 --printExecutionPlan=false -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-21692) Chapter 36 "On the number of column families" may have some problems
[ https://issues.apache.org/jira/browse/HBASE-21692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21692: -- Fix Version/s: 4.0.0-alpha-1 (was: 3.0.0-beta-1) > Chapter 36 "On the number of column families" may have some problems > > > Key: HBASE-21692 > URL: https://issues.apache.org/jira/browse/HBASE-21692 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 1.2.0, 1.3.0, 2.0.0 >Reporter: lixiaobao >Assignee: lixiaobao >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: HBASE-21692.patch > > > h2. 36. On the number of column families > Currently, flushing and compactions are done on a per Region basis so if one > column family is carrying the bulk of the data bringing on flushes. > from Version 1.2 ,flushing and compactions are done on a per Column Family > basis. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-21500) Jetty aliases parameter need to change as per jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21500: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Jetty aliases parameter need to change as per jetty 9.x version > --- > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0, 2.0.1, 2.1.1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-21500) Jetty aliases parameter need to change as per jetty 9.x version
[ https://issues.apache.org/jira/browse/HBASE-21500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797443#comment-17797443 ] Duo Zhang commented on HBASE-21500: --- [~nihaljain.cs] Mind putting up a PR for this change? Thanks. > Jetty aliases parameter need to change as per jetty 9.x version > --- > > Key: HBASE-21500 > URL: https://issues.apache.org/jira/browse/HBASE-21500 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.1.0, 2.0.0, 2.0.1, 2.1.1 >Reporter: Bhupendra Kumar Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-1 > > Attachments: HBASE-21500.master.001.patch > > > Noticed that Jetty aliases parameter in HttpServer.java > "*org.mortbay.jetty.servlet.Default.aliases*" is as per old jetty version and > need to change as per jetty 9.x new version after the HBASE-12894 > Refer > https://github.com/apache/hbase/blob/405bf5e6383a09f435baadbac6c389e9f6c43ac6/hbase-http/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L647 > It should be *"org.eclipse.jetty.servlet.Default.aliases"* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-20756) reference guide examples still contain references to EOM versions
[
https://issues.apache.org/jira/browse/HBASE-20756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20756:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> reference guide examples still contain references to EOM versions
> -
>
> Key: HBASE-20756
> URL: https://issues.apache.org/jira/browse/HBASE-20756
> Project: HBase
> Issue Type: Bug
> Components: community, documentation
>Affects Versions: 3.0.0-alpha-1
>Reporter: Sean Busbey
>Assignee: Kevin Su
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-20756.v0.patch, HBASE-20756.v1.patch,
> HBASE-20756.v2.patch, image-2019-03-05-23-07-57-228.png
>
>
> the reference guide still has examples that refer to EOM versions. e.g. this
> shell output that has 0.98 in it:
> {code}
> $ echo "describe 'test1'" | ./hbase shell -n
> Version 0.98.3-hadoop2, rd5e65a9144e315bb0a964e7730871af32f5018d5, Sat May 31
> 19:56:09 PDT 2014
> describe 'test1'
> DESCRIPTION ENABLED
> 'test1', {NAME => 'cf', DATA_BLOCK_ENCODING => 'NON true
> E', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
> VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIO
> NS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
> 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false'
> , BLOCKCACHE => 'true'}
> 1 row(s) in 3.2410 seconds
> {code}
> these should be redone with a current release. Ideally a version in the minor
> release line the docs are for, but even just updating to the stable pointer
> would be a big improvement.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20712) HBase eclipse formatter should not format the ASF license header
[
https://issues.apache.org/jira/browse/HBASE-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20712:
--
Fix Version/s: (was: 3.0.0-beta-1)
Assignee: (was: Nihal Jain)
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
Since we use spotless to format the code and also add license headers, I do not
think this is still a problem now.
Resolve.
Feel free to reopen if you think this is still a problem worth fixing.
Thanks [~nihaljain.cs].
> HBase eclipse formatter should not format the ASF license header
>
>
> Key: HBASE-20712
> URL: https://issues.apache.org/jira/browse/HBASE-20712
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Nihal Jain
>Priority: Minor
> Attachments: HBASE-20712.master.001.patch
>
>
> Whenever we add a new class along with the ASF license header, we cannot
> press {{ctrl + A}} to format the whole file as it also formats the header
> text.
> IMO we should disable formatting of headers in our code formatter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20693: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-20693.master.001.patch, rest_home_after_patch.png, > thrift_home_after_patch.png, thrift_log_level_after_patch.png, > thrift_log_level_before_patch.png > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' from them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-20528) Revise collections copying from iteration to built-in function
[ https://issues.apache.org/jira/browse/HBASE-20528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-20528: -- Fix Version/s: 2.6.0 2.4.18 2.5.8 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to all active branches. The code for StochasticLoadBalancer is changed so I do not apply the changes for that file. Thanks [~jackyhobingo] for contributing! > Revise collections copying from iteration to built-in function > -- > > Key: HBASE-20528 > URL: https://issues.apache.org/jira/browse/HBASE-20528 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-1 >Reporter: Hua-Yi Ho >Assignee: Hua-Yi Ho >Priority: Minor > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8 > > Attachments: HBASE-20528.master.001.patch > > > Some collection codes in file > StochasticLoadBalancer.java, AbstractHBaseTool.java, HFileInputFormat.java, > Result.java, and WalPlayer.java, using iterations to copy whole data in > collections. The iterations can just replace by just Colletions.addAll and > Arrays.copyOf. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-20693) Refactor rest and thrift jsp's and extract header and footer
[ https://issues.apache.org/jira/browse/HBASE-20693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797440#comment-17797440 ] Duo Zhang commented on HBASE-20693: --- Any updates here [~nihaljain.cs]? Mind putting up a PR, as per [~ndimiduk] said? Thanks. > Refactor rest and thrift jsp's and extract header and footer > > > Key: HBASE-20693 > URL: https://issues.apache.org/jira/browse/HBASE-20693 > Project: HBase > Issue Type: Improvement >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0-beta-1 > > Attachments: HBASE-20693.master.001.patch, rest_home_after_patch.png, > thrift_home_after_patch.png, thrift_log_level_after_patch.png, > thrift_log_level_before_patch.png > > > Log Level page design was changed to include header and footers in > HBASE-20577. Since, thrift and rest do not have header and footer jsp's, the > log level page will be as it were before HBASE-20577 i.e without the > navigation bar. This JIRA will refactor rest and thrift and extract > 'header.jsp' and 'footer.jsp' from them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-17285) Misconfiguration of JVM GC options in HADOOP_CLIENT_OPTS may break `bin/hbase`
[
https://issues.apache.org/jira/browse/HBASE-17285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-17285:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Misconfiguration of JVM GC options in HADOOP_CLIENT_OPTS may break `bin/hbase`
> --
>
> Key: HBASE-17285
> URL: https://issues.apache.org/jira/browse/HBASE-17285
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-17285.001.patch
>
>
> Had the great fun of digging through this one. Had a user reporting that
> hiveserver2 was no longer finding HBase jars on the classpath. This is
> supposed to happen via {{hbase mapredcp}}.
> It turned out that they had configured hive-env.sh to set
> {{HADOOP_CLIENT_OPTS="-XX:+PrintGCDetails"}} (among other things), which
> creates a big multi-line string instead of just a directory. Because of poor
> quoting in {{bin/hbase}}, this gives you a wonderfully intuitive error:
> {noformat}
> Error: Could not find or load main class Heap
> {noformat}
> That {{Heap}} is actually from the JVM GC details that it was told to print.
> While I don't expect this to be a common problem people run into, it's one
> that we can address with better quoting. e.g.
> {noformat}
> + exec
> /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/bin/java
> -Dproc_mapredcp '-XX:OnOutOfMemoryError=kill -9 %p' -XX:+UseConcMarkSweepGC
> -Dhbase.log.dir=/usr/local/lib/hbase//logs -Dhbase.log.file=hbase.log
> -Dhbase.home.dir=/usr/local/lib/hbase/ -Dhbase.id.str=
> -Dhbase.root.logger=INFO,console
> '-Djava.library.path='\''/usr/local/lib/hadoop//lib/native' Heap PSYoungGen
> total 76800K, used 7942K '[0x0007f550,' 0x0007faa8,
> '0x0008)' eden space 66048K, 12% used
> '[0x0007f550,0x0007f5cc19c0,0x0007f958)' from space
> 10752K, 0% used '[0x0007fa00,0x0007fa00,0x0007faa8)'
> to space 10752K, 0% used
> '[0x0007f958,0x0007f958,0x0007fa00)' ParOldGen total
> 174592K, used 0K '[0x0007e000,' 0x0007eaa8,
> '0x0007f550)' object space 174592K, 0% used
> '[0x0007e000,0x0007e000,0x0007eaa8)' PSPermGen total
> 21504K, used 2756K '[0x0007dae0,' 0x0007dc30,
> '0x0007e000)' object space 21504K, 12% used
> '[0x0007dae0,0x0007db0b11b8,0x0007dc30)'\'''
> -Dhbase.security.logger=INFO,NullAppender
> org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-27921: Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively [hbase]
Apache9 commented on PR #5274: URL: https://github.com/apache/hbase/pull/5274#issuecomment-1858821147 Close in favor of #5568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-27921: Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively [hbase]
Apache9 closed pull request #5274: HBASE-27921: Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively URL: https://github.com/apache/hbase/pull/5274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27921) Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively
[
https://issues.apache.org/jira/browse/HBASE-27921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27921:
--
Fix Version/s: (was: 3.0.0-beta-1)
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Resolve as duplicated by HBASE-28249.
Thanks [~sercan.tekin] for raising the issue and PR.
> Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58
> respectively
>
>
> Key: HBASE-27921
> URL: https://issues.apache.org/jira/browse/HBASE-27921
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4
>Reporter: Sercan Tekin
>Priority: Major
> Labels: jruby, security, shell
>
> Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
> {code:java}
> > find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
> {code}
> *snakeyaml-1.33* has a critical CVE
> [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
> {*}jruby-complete-9.4.2{*}{*}.0{*} uses {*}snakeyaml-engine-2.6{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27198) Clean up error-prone findings
[ https://issues.apache.org/jira/browse/HBASE-27198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27198: -- Fix Version/s: (was: 3.0.0-beta-1) > Clean up error-prone findings > - > > Key: HBASE-27198 > URL: https://issues.apache.org/jira/browse/HBASE-27198 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > > It's time to do another sweep over error-prone findings. Umbrella for > module-by-module work as subtasks. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-21243) Correct java-doc for the method RpcServer.getRemoteAddress()
[
https://issues.apache.org/jira/browse/HBASE-21243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21243:
--
Fix Version/s: 2.6.0
2.4.18
2.5.8
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to all active branches.
Thanks [~nihaljain.cs]!
> Correct java-doc for the method RpcServer.getRemoteAddress()
>
>
> Key: HBASE-21243
> URL: https://issues.apache.org/jira/browse/HBASE-21243
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Trivial
> Labels: beginner, beginners, documentaion
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
> Attachments: HBASE-21243.master.001.patch,
> HBASE-21243.master.002.patch
>
>
> Correct the java-doc for the method {{RpcServer.getRemoteAddress()}}.
> Currently it look like as below:
> {code:java}
> /**
>* @return Address of remote client if a request is ongoing, else null
>*/
> public static Optional getRemoteAddress() {
> return getCurrentCall().map(RpcCall::getRemoteAddress);
> }
> {code}
> Contrary to the doc the method will never return null. Rather it may return
> an empty Optional.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28180) Review the usage of RegionStates.getOrCreateServer
[ https://issues.apache.org/jira/browse/HBASE-28180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28180. --- Fix Version/s: 2.6.0 3.0.0-beta-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.6+. Thanks [~Xiaolin Ha] for reviewing! > Review the usage of RegionStates.getOrCreateServer > -- > > Key: HBASE-28180 > URL: https://issues.apache.org/jira/browse/HBASE-28180 > Project: HBase > Issue Type: Bug > Components: amv2, master >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1 > > Attachments: > org.apache.hadoop.hbase.master.TestClusterRestartFailover-output.txt > > > It failed two times in this PR. > https://github.com/apache/hbase/pull/5475 > Filed an issue to track this problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26034: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-beta-2 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-25510) Optimize TableName.valueOf from O(n) to O(1). We can get benefits when the number of tables in the cluster is greater than dozens
[
https://issues.apache.org/jira/browse/HBASE-25510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-25510:
--
Fix Version/s: 3.0.0-beta-2
(was: 2.6.0)
(was: 3.0.0-beta-1)
> Optimize TableName.valueOf from O(n) to O(1). We can get benefits when the
> number of tables in the cluster is greater than dozens
> --
>
> Key: HBASE-25510
> URL: https://issues.apache.org/jira/browse/HBASE-25510
> Project: HBase
> Issue Type: Improvement
> Components: master, Replication
>Affects Versions: 1.2.12, 1.4.13, 2.4.1
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Major
> Fix For: 3.0.0-beta-2
>
> Attachments: TestTableNameJMH.java, optimiz_benchmark,
> optimiz_benchmark_fix, origin_benchmark, stucks-profile-info
>
>
> Now, TableName.valueOf will try to find TableName Object in cache
> linearly(code show as below). So it is too slow when we has thousands of
> tables on cluster.
> {code:java}
> // code placeholder
> for (TableName tn : tableCache) {
> if (Bytes.equals(tn.getQualifier(), qns) && Bytes.equals(tn.getNamespace(),
> bns)) {
> return tn;
> }
> }{code}
> I try to store the object in the hash table, so it can look up more quickly.
> code like this
> {code:java}
> // code placeholder
> TableName oldTable = tableCache.get(nameAsStr);{code}
>
> In our cluster which has tens thousands of tables. (Most of that is KYLIN
> table).
> We found that in the following two cases, the TableName.valueOf method will
> severely restrict our performance.
>
> Common premise: tens of thousands table in cluster
> cause: TableName.valueOf with low performance. (because we need to traverse
> all caches linearly)
>
> Case1. Replication
> premise1: one of table write with high qps, small value, Non-batch request.
> cause too much wal entry
> premise2: deserialize WAL Entry includes calling the TableName.valueOf method.
> Cause: Replicat Stuck. A lot of WAL files pile up.
>
> Case2. Active Master Start up
> NamespaceStateManager init should init all RegionInfo, and regioninfo init
> will call TableName.valueOf. It will cost some time if TableName.valueOf is
> slow.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26034) Add support to take parallel backups
[ https://issues.apache.org/jira/browse/HBASE-26034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797410#comment-17797410 ] Duo Zhang commented on HBASE-26034: --- [~bbeaudreault] Mind taking a look at this backup/restore improvement? Thanks. > Add support to take parallel backups > > > Key: HBASE-26034 > URL: https://issues.apache.org/jira/browse/HBASE-26034 > Project: HBase > Issue Type: Improvement > Components: backuprestore >Affects Versions: 3.0.0-alpha-2 >Reporter: Mallikarjun >Assignee: Mallikarjun >Priority: Major > Fix For: 3.0.0-beta-1 > > Attachments: existing_design.png, proposed_design.png > > > *Existing Design:* > !existing_design.png|width=632,height=1238! > *Proposed Changes:* > *!proposed_design.png|width=637,height=1300!* -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-25510 Optimize TableName.valueOf from O(n) to O(1) [hbase]
Apache9 commented on PR #2885: URL: https://github.com/apache/hbase/pull/2885#issuecomment-1858801496 Any updates here? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-22461) A "NullPointerException" could be thrown; "Result" is nullable
[
https://issues.apache.org/jira/browse/HBASE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22461:
--
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Resolve.
Feel free to reopen if you think this is still a problem which needs to be
fixed.
[~hapihu].
Thanks.
> A "NullPointerException" could be thrown; "Result" is nullable
> --
>
> Key: HBASE-22461
> URL: https://issues.apache.org/jira/browse/HBASE-22461
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.1.4
>Reporter: wuguihu
>Priority: Major
> Attachments: HBASE-22461.patch
>
>
> In hbase-client model the class
> "org.apache.hadoop.hbase.MetaTableAccessor"'s method getRegionInfo(final
> Result r, byte [] qualifier),A "NullPointerException" could be thrown; "r" is
> nullable here.
>
> {code:java}
> @Nullable
> private static RegionInfo getRegionInfo(final Result r, byte [] qualifier) {
> Cell cell = r.getColumnLatestCell(getCatalogFamily(), qualifier);
> if (cell == null) return null;
> return RegionInfo.parseFromOrNull(cell.getValueArray(),
> cell.getValueOffset(), cell.getValueLength());
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-25433) There is no limit on the table name length when creating a table
[ https://issues.apache.org/jira/browse/HBASE-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-25433: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > There is no limit on the table name length when creating a table > > > Key: HBASE-25433 > URL: https://issues.apache.org/jira/browse/HBASE-25433 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.2.3 >Reporter: ZouWeiMing >Priority: Major > Fix For: 3.0.0-beta-2 > > Attachments: attachment_tableNameLengthLimit.html > > > It occurs > Exeception(ERROR：java.util.concurrent.ExecutionExceptionjava.io.IOExpcetionException > in createDir) because of 8000 bytes length limits on HDFS directory name if > the length of HBase table name misses validation when creating. > So,we decide to verify the length of table name when creating a > table.IllegalArgumentExcetion will be thrown when length is larger than 8000 > bytes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-22461) A "NullPointerException" could be thrown; "Result" is nullable
[
https://issues.apache.org/jira/browse/HBASE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-22461:
--
Fix Version/s: (was: 3.0.0-beta-1)
> A "NullPointerException" could be thrown; "Result" is nullable
> --
>
> Key: HBASE-22461
> URL: https://issues.apache.org/jira/browse/HBASE-22461
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.1.4
>Reporter: wuguihu
>Priority: Major
> Attachments: HBASE-22461.patch
>
>
> In hbase-client model the class
> "org.apache.hadoop.hbase.MetaTableAccessor"'s method getRegionInfo(final
> Result r, byte [] qualifier),A "NullPointerException" could be thrown; "r" is
> nullable here.
>
> {code:java}
> @Nullable
> private static RegionInfo getRegionInfo(final Result r, byte [] qualifier) {
> Cell cell = r.getColumnLatestCell(getCatalogFamily(), qualifier);
> if (cell == null) return null;
> return RegionInfo.parseFromOrNull(cell.getValueArray(),
> cell.getValueOffset(), cell.getValueLength());
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21455:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Labels: quota, space
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch, HBASE-21455.master.003.patch,
> HBASE-21455.master.004.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-21455) Update filesystem-space quota fail if there is a space quota for non-existing namespace
[
https://issues.apache.org/jira/browse/HBASE-21455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797408#comment-17797408
]
Duo Zhang commented on HBASE-21455:
---
Any updates here? [~pankaj2461] [~wenbang].
Thanks.
> Update filesystem-space quota fail if there is a space quota for non-existing
> namespace
> ---
>
> Key: HBASE-21455
> URL: https://issues.apache.org/jira/browse/HBASE-21455
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.2
>Reporter: wenbang
>Assignee: wenbang
>Priority: Major
> Labels: quota, space
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-21455.master.001.patch,
> HBASE-21455.master.002.patch, HBASE-21455.master.003.patch,
> HBASE-21455.master.004.patch
>
>
> QuotaObserverChore#fetchAllTablesWithQuotasDefined may fail and throw a
> NamespaceNotFoundException because of namespace does not exist
> {code:java}
> // Collect all of the tables in the namespace
> TableName[] tablesInNS =
> conn.getAdmin().listTableNamesByNamespace(namespace);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-21341) DeadServer shouldn't import unshaded Preconditions
[
https://issues.apache.org/jira/browse/HBASE-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21341:
--
Fix Version/s: (was: 3.0.0-beta-1)
Assignee: (was: Ted Yu)
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
We have introduced the import check enforce rule plugin and forbid the import
for unshaded Preconditions now.
Resolve.
> DeadServer shouldn't import unshaded Preconditions
> --
>
> Key: HBASE-21341
> URL: https://issues.apache.org/jira/browse/HBASE-21341
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
> Attachments: 21341.v1.txt
>
>
> DeadServer currently imports unshaded Preconditions :
> {code}
> import com.google.common.base.Preconditions;
> {code}
> We should import shaded version of Preconditions.
> This is the only place where unshaded class from com.google.common is imported
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-21231) Add documentation for MajorCompactor
[ https://issues.apache.org/jira/browse/HBASE-21231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21231: -- Fix Version/s: 3.0.0-beta-2 (was: 3.0.0-beta-1) > Add documentation for MajorCompactor > > > Key: HBASE-21231 > URL: https://issues.apache.org/jira/browse/HBASE-21231 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-21231.master.001.patch, > HBASE-21231.master.002.patch > > > HBASE-19528 added a new MajorCompactor tool, but it lacks of documentation. > Let's document it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-20821) Re-creating a dropped namespace and contained table inherits previously set space quota settings
[
https://issues.apache.org/jira/browse/HBASE-20821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20821:
--
Fix Version/s: 3.0.0-beta-2
(was: 2.1.0)
(was: 3.0.0-beta-1)
> Re-creating a dropped namespace and contained table inherits previously set
> space quota settings
>
>
> Key: HBASE-20821
> URL: https://issues.apache.org/jira/browse/HBASE-20821
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.1.0
>Reporter: Nihal Jain
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-20821.master.001.patch,
> HBASE-20821.master.002.patch, HBASE-20821.master.003.patch
>
>
> As demonstarted in
> [HBASE-20662.master.002.patch|https://issues.apache.org/jira/secure/attachment/12927187/HBASE-20662.master.002.patch]
> re-creating a dropped namespace and contained table inherits previously set
> space quota settings.
> *Steps:*
> * Create a namespace and a table in it
> * Set space quota on namespace
> * Violate namespace quota
> * Drop table and then namespace
> * Re create same namespace and same table
> * Put data into the table (more than the previosuly set namespace quota
> limit)
> {code:java}
> private void setQuotaAndThenDropNamespace(final String namespace,
> SpaceViolationPolicy policy)
> throws Exception {
> Put put = new Put(Bytes.toBytes("to_reject"));
> put.addColumn(Bytes.toBytes(SpaceQuotaHelperForTests.F1),
> Bytes.toBytes("to"),
> Bytes.toBytes("reject"));
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> // Do puts until we violate space policy
> final TableName tn =
> writeUntilNSSpaceViolationAndVerifyViolation(namespace, policy, put);
> // Now, drop the table
> TEST_UTIL.deleteTable(tn);
> LOG.debug("Successfully deleted table ", tn);
> // Now, drop the namespace
> TEST_UTIL.getAdmin().deleteNamespace(namespace);
> LOG.debug("Successfully deleted the namespace ", namespace);
> // Now re-create the namespace
> createNamespaceIfNotExist(TEST_UTIL.getAdmin(), namespace);
> LOG.debug("Successfully re-created the namespace ", namespace);
> TEST_UTIL.createTable(tn, Bytes.toBytes(SpaceQuotaHelperForTests.F1));
> LOG.debug("Successfully re-created table ", tn);
> // Put some rows now: should not violate as namespace/quota was dropped
> verifyNoViolation(policy, tn, put);
> }
> {code}
> *Expected*: SpaceQuota settings should not exist on the newly re-created
> table and we should be able to put limit less data into the table
> *Actual:* We fail to put data into newly created table as SpaceQuota settings
> (systematically created due to previously added namespace space quota) exist
> on table
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-18822) Create table for peer cluster automatically when creating table in source cluster of using namespace replication.
[ https://issues.apache.org/jira/browse/HBASE-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain reassigned HBASE-18822: -- Assignee: Nihal Jain (was: Zheng Hu) > Create table for peer cluster automatically when creating table in source > cluster of using namespace replication. > - > > Key: HBASE-18822 > URL: https://issues.apache.org/jira/browse/HBASE-18822 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0-alpha-2 >Reporter: Zheng Hu >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-18822.master.001.patch, HBASE-18822.v1.patch, > HBASE-18822.v1.patch > > > In our cluster of using namespace replication, we always forget to create > table in peer cluster, which lead to replication get stuck. > We have implemented the feature in our cluster: create table for peer > cluster automatically when creating table in source cluster of using > namespace replication. > > I'm not sure if someone else needs this feature, so create an issue here for > discussing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-18822) Create table for peer cluster automatically when creating table in source cluster of using namespace replication.
[ https://issues.apache.org/jira/browse/HBASE-18822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797403#comment-17797403 ] Nihal Jain commented on HBASE-18822: I had submitted a working patch for this long back, let me assign this to myself so that I don't forget this jira and raise a PR for the same soon. > Create table for peer cluster automatically when creating table in source > cluster of using namespace replication. > - > > Key: HBASE-18822 > URL: https://issues.apache.org/jira/browse/HBASE-18822 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0-alpha-2 >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0-beta-2 > > Attachments: HBASE-18822.master.001.patch, HBASE-18822.v1.patch, > HBASE-18822.v1.patch > > > In our cluster of using namespace replication, we always forget to create > table in peer cluster, which lead to replication get stuck. > We have implemented the feature in our cluster: create table for peer > cluster automatically when creating table in source cluster of using > namespace replication. > > I'm not sure if someone else needs this feature, so create an issue here for > discussing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-18999) Shell support for multiple columns for DML commands
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797401#comment-17797401
]
Nihal Jain commented on HBASE-18999:
Thank you so much [~zhangduo]. Let me put up a PR by rebasing and testing the
patch soon.
> Shell support for multiple columns for DML commands
>
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0-alpha-1, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-18999) Shell support for multiple columns for DML commands
[
https://issues.apache.org/jira/browse/HBASE-18999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797393#comment-17797393
]
Duo Zhang commented on HBASE-18999:
---
Any updates here [~nihaljain.cs]?
I could help reviewing the patch.
Thanks.
> Shell support for multiple columns for DML commands
>
>
> Key: HBASE-18999
> URL: https://issues.apache.org/jira/browse/HBASE-18999
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.0.0, 3.0.0-alpha-1, 2.0.0
>Reporter: Mike Drob
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-18999.master.001.patch,
> HBASE-18999.master.002.patch
>
>
> A {{Put}} can carry multiple cells, but doing so in the shell is very
> difficult to construct. We should make this easier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-16668) Admin class should have a synchronous Admin#mergeRegions* method
[
https://issues.apache.org/jira/browse/HBASE-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-16668:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Admin class should have a synchronous Admin#mergeRegions* method
> -
>
> Key: HBASE-16668
> URL: https://issues.apache.org/jira/browse/HBASE-16668
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 3.0.0-beta-2
>
> Attachments: 16668.v1.txt
>
>
> In trunk from HBASE-14552, we have deprecated {{void Admin#mergeRegions}} (in
> 1.x this was an asynchronous call) and replaced it with {{Future
> Admin#mergeRegionsAsync}} which is clearly async.
> This leaves us only with the async version.
> We should have an easy way to make {{mergeRegions}} or an equivalant behave
> synchronously.
> For normal java Futures, we could just call the future's {{get()}} method.
> Unforutnately, the future this method returns doesn't follow java Future
> convention and throws Unimplemented operation when a plain {{get()}} is
> called and makes the api harder to use and read. We could make this future
> act more normally, and have the timeout throw an InterruptedException.
> Alternately, we could expose a new method in {{Admin}} that behaves
> synchronously such as {{HBaseAdmin#mergeRegionsSync}}. The caveat here is
> that we shouldn't use the name {{#mergeRegions}} since it exists in 1.x with
> async semantics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20796) STUCK RIT though region successfully assigned (hung RPC)
[
https://issues.apache.org/jira/browse/HBASE-20796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20796:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> STUCK RIT though region successfully assigned (hung RPC)
>
>
> Key: HBASE-20796
> URL: https://issues.apache.org/jira/browse/HBASE-20796
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-beta-2
>
> Attachments: 0001-Test.patch, HBASE-20796.branch-2.0.001.patch
>
>
> This is a good one. We keep logging messages like this:
> {code}
> 2018-06-26 12:32:24,859 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OPENING,
> location=vd0410.X.Y.com,22101,1529611445046,
> table=IntegrationTestBigLinkedList_20180525080406,
> region=e10b35d49528e2453a04c7038e3393d7
> {code}
> ...though the region is successfully assigned.
> Story:
> * Dispatch an assign 2018-06-26 12:31:27,390 INFO
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Dispatch
> pid=370829, ppid=370391, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> AssignProcedure table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046
> * It gets stuck 2018-06-26 12:32:29,860 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OPENING, location=vd0410.X.Y.Z,22101,1529611445046,
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2 (Because the server was killed)
> * We stay STUCK for a while.
> * The Master notices the server as crashed and starts a SCP.
> * SCP kills ongoing assign: 2018-06-26 12:32:54,809 INFO
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure: pid=371105
> found RIT pid=370829, ppid=370391, state=RUNNABLE:REGION_TRANSITION_DISPATCH;
> AssignProcedure table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046
> * The kill brings on a retry ... 2018-06-26 12:32:54,810 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Remote
> call failed pid=370829, ppid=370391,
> state=RUNNABLE:REGION_TRANSITION_DISPATCH; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPENING,
> location=vd0410.X.Y.Z,22101,1529611445046; exception=ServerCrashProcedure
> pid=371105, server=vd0410.X.Y.Z,22101,1529611445046
> * Which eventually succeeds. Successfully deployed to new server
> 2018-06-26 12:32:55,429 INFO
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: Finished pid=370829,
> ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2 in 1mins, 35.379sec
> * But then, it looks like the RPC was ongoing and it broke in following way
> 2018-06-26 12:33:06,378 WARN
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure: Remote
> call failed pid=370829, ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPEN,
> location=vc0614.halxg.cloudera.com,22101,1529611443424; exception=Call to
> vd0410.X.Y.Z/10.10.10.10:22101 failed on local exception:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> syscall:read(..) failed: Connection reset by peer (Notice how state for
> region is OPEN and 'SUCCESS').
> * Then says 2018-06-26 12:33:06,380 INFO
> org.apache.hadoop.hbase.master.assignment.AssignProcedure: Retry=1 of max=10;
> pid=370829, ppid=370391, state=SUCCESS; AssignProcedure
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2; rit=OPEN,
> location=vc0614.X.Y.Z,22101,1529611443424
> * And finally... 2018-06-26 12:34:10,727 WARN
> org.apache.hadoop.hbase.master.assignment.AssignmentManager: STUCK
> Region-In-Transition rit=OFFLINE, location=null,
> table=IntegrationTestBigLinkedList_20180612114844,
> region=f69ccf7d9178ce166b515e0e2ef019d2
> Restart of Master got rid of the STUCK complaints.
> This is interesting because the stuck rpc and the successful reassign are all
> riding on the same pid.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20679) Add the ability to compile JSP dynamically in Jetty
[
https://issues.apache.org/jira/browse/HBASE-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-20679:
--
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Add the ability to compile JSP dynamically in Jetty
> ---
>
> Key: HBASE-20679
> URL: https://issues.apache.org/jira/browse/HBASE-20679
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Allan Yang
>Assignee: Allan Yang
>Priority: Major
> Fix For: 3.0.0-beta-2
>
> Attachments: HBASE-20679.002.patch, HBASE-20679.003.patch,
> HBASE-20679.004.patch, HBASE-20679.005.patch, HBASE-20679.patch
>
>
> As discussed in HBASE-20617, adding the ability to dynamically compile jsp
> enable us to do some hot fix.
> For example, several days ago, in our testing HBase-2.0 cluster,
> procedureWals were corrupted due to some unknown reasons. After restarting
> the cluster, since some procedures(AssignProcedure for example) were
> corrupted and couldn't be replayed. Some regions were stuck in RIT forever.
> We couldn't use HBCK since it haven't support AssignmentV2 yet. As a matter
> of fact, the namespace region was not online, so the master was not inited,
> we even couldn't use shell command like assign/move. But, we wrote a jsp and
> fix this issue easily. The jsp file is like this:
> {code:java}
> <%
> String action = request.getParameter("action");
> HMaster master = (HMaster)getServletContext().getAttribute(HMaster.MASTER);
> List offlineRegionsToAssign = new ArrayList<>();
> List regionRITs =
> master.getAssignmentManager()
> .getRegionStates().getRegionsInTransition();
> for (RegionStates.RegionStateNode regionStateNode : regionRITs) {
> // if regionStateNode don't have a procedure attached, but meta state
> shows
> // this region is in RIT, that means the previous procedure may be
> corrupted
> // we need to create a new assignProcedure to assign them
> if (!regionStateNode.isInTransition()) {
> offlineRegionsToAssign.add(regionStateNode.getRegionInfo());
> out.println("RIT region:" + regionStateNode);
> }
> }
> // Assign offline regions. Uses round-robin.
> if ("fix".equals(action) && offlineRegionsToAssign.size() > 0) {
>
> master.getMasterProcedureExecutor().submitProcedures(master.getAssignmentManager().
> createRoundRobinAssignProcedures(offlineRegionsToAssign));
> } else {
> out.println("use ?action=fix to fix RIT regions");
> }
> %>
> {code}
> Above it is only one example we can do if we have the ability to compile jsp
> dynamically. We think it is very useful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-20563) Use parameterized logging in hbase-common
[ https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797392#comment-17797392 ] Duo Zhang commented on HBASE-20563: --- Any updates here? [~meszibalu] Thanks. > Use parameterized logging in hbase-common > - > > Key: HBASE-20563 > URL: https://issues.apache.org/jira/browse/HBASE-20563 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Fix For: 3.0.0-beta-1 > > Attachments: HBASE-20563.master.001.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
