[jira] [Commented] (HBASE-27185) Rewrite NettyRpcServer to decode rpc request with netty handler
[
https://issues.apache.org/jira/browse/HBASE-27185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572751#comment-17572751
]
Hudson commented on HBASE-27185:
Results for branch branch-2
[build #604 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Rewrite NettyRpcServer to decode rpc request with netty handler
> ---
>
> Key: HBASE-27185
> URL: https://issues.apache.org/jira/browse/HBASE-27185
> Project: HBase
> Issue Type: Improvement
> Components: netty, rpc
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> This is a follow on issue of HBASE-26708.
> The memory leaks happens in the saslReadAndProcess method. It does not follow
> a typical netty style so it is more likely to introduce bugs.
> Since we agree that NettyRpcServer is the future and SimpleRpcServer should
> be deprecated and finally removed, let's first implement dedicated rpc
> request decoder for netty.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27203) Clean up error-prone findings in hbase-client
[
https://issues.apache.org/jira/browse/HBASE-27203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572749#comment-17572749
]
Hudson commented on HBASE-27203:
Results for branch branch-2
[build #604 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Clean up error-prone findings in hbase-client
> -
>
> Key: HBASE-27203
> URL: https://issues.apache.org/jira/browse/HBASE-27203
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27247) TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig
[
https://issues.apache.org/jira/browse/HBASE-27247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572750#comment-17572750
]
Hudson commented on HBASE-27247:
Results for branch branch-2
[build #604 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/604/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of
> ReplicationPeerConfigUtil.parseTableCFsFromConfig
> -
>
> Key: HBASE-27247
> URL: https://issues.apache.org/jira/browse/HBASE-27247
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0-alpha-4
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> HBASE-27203 has modified
> {{ReplicationPeerConfigUtil.parseTableCFsFromConfig}} which causes
> {{TestPerTableCFReplication.testParseTableCFsFromConfig}} broken.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572741#comment-17572741
]
Hudson commented on HBASE-27251:
Results for branch branch-2.4
[build #398 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/398/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/398/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/398/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Assignee: Huaxiang Sun
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
[GitHub] [hbase] Apache-HBase commented on pull request #4665: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
Apache-HBase commented on PR #4665: URL: https://github.com/apache/hbase/pull/4665#issuecomment-1198744875 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 31s | branch-2.4 passed | | +1 :green_heart: | compile | 0m 43s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 4m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 18s | the patch passed | | +1 :green_heart: | compile | 0m 43s | the patch passed | | +1 :green_heart: | javac | 0m 43s | the patch passed | | +1 :green_heart: | shadedjars | 4m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 170m 16s | hbase-server in the patch passed. | | | | 188m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4665 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 751f5daae454 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / aa0fa58d06 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/testReport/ | | Max. process+thread count | 2501 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4665: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
Apache-HBase commented on PR #4665: URL: https://github.com/apache/hbase/pull/4665#issuecomment-1198742454 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 57s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 15s | branch-2.4 passed | | +1 :green_heart: | compile | 0m 33s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 4m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 58s | the patch passed | | +1 :green_heart: | compile | 0m 33s | the patch passed | | +1 :green_heart: | javac | 0m 33s | the patch passed | | +1 :green_heart: | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 165m 36s | hbase-server in the patch passed. | | | | 183m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4665 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d620c9fea379 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / aa0fa58d06 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/testReport/ | | Max. process+thread count | 2539 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #4665: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
apurtell commented on PR #4665: URL: https://github.com/apache/hbase/pull/4665#issuecomment-1198722841 Run spotless:apply before merge or commit, please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-27251:
Assignee: Huaxiang Sun
Status: Patch Available (was: Open)
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Assignee: Huaxiang Sun
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
>
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
>
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
>
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {noformat}
> When I examine the on-disk file system, I see,
> {noformat}
> nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/
> Found 3 items
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.initialized
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.tabledesc
> drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25
> /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682
> nonroot@namenode-0:~$ hdfs dfs
[GitHub] [hbase] Apache-HBase commented on pull request #4665: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
Apache-HBase commented on PR #4665: URL: https://github.com/apache/hbase/pull/4665#issuecomment-1198660231 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | branch-2.4 passed | | +1 :green_heart: | compile | 2m 13s | branch-2.4 passed | | +1 :green_heart: | checkstyle | 0m 32s | branch-2.4 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 20s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 59s | the patch passed | | +1 :green_heart: | compile | 2m 11s | the patch passed | | +1 :green_heart: | javac | 2m 11s | the patch passed | | +1 :green_heart: | checkstyle | 0m 32s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 28s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | -1 :x: | spotless | 0m 38s | patch has 24 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 1m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 30m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4665 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 55f3f36f2385 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / aa0fa58d06 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 64 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4665/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572659#comment-17572659
]
Huaxiang Sun commented on HBASE-27251:
--
Put up an addendum to add a warning log if there are non-region directories
under master store.
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
>
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
>
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
>
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {noformat}
> When I examine the on-disk file system, I see,
> {noformat}
> nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/
> Found 3 items
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.initialized
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.tabledesc
> drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25
> /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682
>
[GitHub] [hbase] huaxiangsun opened a new pull request, #4665: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
huaxiangsun opened a new pull request, #4665: URL: https://github.com/apache/hbase/pull/4665 …File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo' Addendum -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27203) Clean up error-prone findings in hbase-client
[
https://issues.apache.org/jira/browse/HBASE-27203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572624#comment-17572624
]
Hudson commented on HBASE-27203:
Results for branch branch-2.5
[build #174 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Clean up error-prone findings in hbase-client
> -
>
> Key: HBASE-27203
> URL: https://issues.apache.org/jira/browse/HBASE-27203
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27247) TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig
[
https://issues.apache.org/jira/browse/HBASE-27247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572625#comment-17572625
]
Hudson commented on HBASE-27247:
Results for branch branch-2.5
[build #174 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/174/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of
> ReplicationPeerConfigUtil.parseTableCFsFromConfig
> -
>
> Key: HBASE-27247
> URL: https://issues.apache.org/jira/browse/HBASE-27247
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0-alpha-4
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> HBASE-27203 has modified
> {{ReplicationPeerConfigUtil.parseTableCFsFromConfig}} which causes
> {{TestPerTableCFReplication.testParseTableCFsFromConfig}} broken.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27247) TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig
[
https://issues.apache.org/jira/browse/HBASE-27247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572620#comment-17572620
]
Hudson commented on HBASE-27247:
Results for branch master
[build #644 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/644/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/644/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/644/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/644/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of
> ReplicationPeerConfigUtil.parseTableCFsFromConfig
> -
>
> Key: HBASE-27247
> URL: https://issues.apache.org/jira/browse/HBASE-27247
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0-alpha-4
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-4
>
>
> HBASE-27203 has modified
> {{ReplicationPeerConfigUtil.parseTableCFsFromConfig}} which causes
> {{TestPerTableCFReplication.testParseTableCFsFromConfig}} broken.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572609#comment-17572609
]
Andrew Kyle Purtell commented on HBASE-26192:
-
Sure, JMX works too. For us, either way we end up scraping it from a servlet to
get it over to alerting, all good.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572604#comment-17572604
]
Bryan Beaudreault commented on HBASE-26192:
---
Thanks for the update. Not sure if we'll have time soon too, but just wanted to
check.
Just while I have you – if someone on my team did pick it up, would you be ok
with exposing these metrics over JMX instead of a servlet endpoint? Not sure if
that's the way we'd go, it's probably more complicated. But wanted to check if
that would suffice.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572589#comment-17572589
]
Huaxiang Sun commented on HBASE-27251:
--
Sounds good to me, [~apurtell]. Let me put an addendum for logging.
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
>
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
>
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
>
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {noformat}
> When I examine the on-disk file system, I see,
> {noformat}
> nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/
> Found 3 items
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.initialized
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.tabledesc
> drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25
> /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682
> nonroot@namenode-0:~$ hdfs dfs -ls
>
[GitHub] [hbase] huaxiangsun merged pull request #4663: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
huaxiangsun merged PR #4663: URL: https://github.com/apache/hbase/pull/4663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572571#comment-17572571
]
Andrew Kyle Purtell edited comment on HBASE-26192 at 7/28/22 5:21 PM:
--
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I had just used plain servlet API and GSON, but the way I used GSON
could also be said to be too simple. Agreed that Jersey and a model class to
formalize the JSON structure in code is the way to go. I put the PR into draft
status and expect to come back to it someday, but not soon. So let me unassign
this issue for now. If someone else wants to take it in the meantime feel free
to go right ahead.
Subsequently Nick contributed a patch to have HbckChore produce a {{Report}}
class encapsulating its findings so some of the modelling improvement has
already been completed.
was (Author: apurtell):
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I had just used plain servlet API and GSON, but the way I used GSON
could also be said to be too simple. Agreed that Jersey and a model class to
formalize the JSON formatting is the way to go. I put the PR into draft status
and expect to come back to it someday, but not soon. So let me unassign this
issue for now. If someone else wants to take it in the meantime feel free to go
right ahead.
Subsequently Nick contributed a patch to have HbckChore produce a {{Report}}
class encapsulating its findings so some of the modelling improvement has
already been completed.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent
[jira] [Comment Edited] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572571#comment-17572571
]
Andrew Kyle Purtell edited comment on HBASE-26192 at 7/28/22 5:18 PM:
--
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I had just used plain servlet API and GSON, but the way I used GSON
could also be said to be too simple. Agreed that Jersey and a model class to
formalize the JSON formatting is the way to go. I put the PR into draft status
and expect to come back to it someday, but not soon. So let me unassign this
issue for now. If someone else wants to take it in the meantime feel free to go
right ahead.
Subsequently Nick contributed a patch to have HbckChore produce a {{Report}}
class encapsulating its findings so some of the modelling improvement has
already been completed.
was (Author: apurtell):
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I had just used plain servlet API and GSON, but the way I used GSON
could also be said to be too simple. Agreed that Jersey and a model class to
formalize the JSON formatting is the way to go. I put the PR into draft status
and expect to come back to it someday, but not soon. So let me unassign this
issue for now. If someone else wants to take it in the meantime feel free to go
right ahead.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572571#comment-17572571
]
Andrew Kyle Purtell edited comment on HBASE-26192 at 7/28/22 5:17 PM:
--
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I had just used plain servlet API and GSON, but the way I used GSON
could also be said to be too simple. Agreed that Jersey and a model class to
formalize the JSON formatting is the way to go. I put the PR into draft status
and expect to come back to it someday, but not soon. So let me unassign this
issue for now. If someone else wants to take it in the meantime feel free to go
right ahead.
was (Author: apurtell):
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I put the PR into draft status and expect to come back to it someday,
but not soon. So let me unassign this issue for now. If someone else wants to
take it in the meantime feel free to go right ahead.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572571#comment-17572571
]
Andrew Kyle Purtell commented on HBASE-26192:
-
On the PR [~ndimiduk] expressed a preference for implementing the servlet with
Jersey. I put the PR into draft status and expect to come back to it someday,
but not soon. So let me unassign this issue for now. If someone else wants to
take it in the meantime feel free to go right ahead.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell reassigned HBASE-26192:
---
Assignee: (was: Andrew Kyle Purtell)
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572555#comment-17572555
]
Bryan Beaudreault commented on HBASE-26192:
---
It recently came back on my radar because we had a case where overlapping
regions showed up which we didn't realize. This is especially dangerous because
it can result in what appears to be data loss, because clients may read from
the wrong region. We're looking to add monitoring on it, and exposing these
json metrics would probably help systemize.
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572553#comment-17572553
]
Bryan Beaudreault commented on HBASE-26192:
---
Sorry [~apurtell], this fell off my radar. I know you've been very busy, but
thoughts on getting this landed soon?
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572544#comment-17572544
]
Andrew Kyle Purtell commented on HBASE-27251:
-
[~huaxiangsun] Thanks for updating the fixversion of HBASE-26640.
Based on Nick's finding I think ignoring the directories are enough, although
when ignoring them in 2.4 we should print a WARN level log line and reference
this JIRA ID in the warning. Then I think we can handle this nit on the 2.4
side, and advise in release notes for 2.4.14 and 2.5.0 that upgrading to 2.4.14
or later would be a good idea before upgrading to 2.5 to avoid this trouble
during rollback.
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
>
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
>
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
>
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {noformat}
> When I examine the on-disk file system, I see,
> {noformat}
> nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/
>
[GitHub] [hbase] huaxiangsun commented on pull request #4663: HBASE-27251 Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to '…
huaxiangsun commented on PR #4663: URL: https://github.com/apache/hbase/pull/4663#issuecomment-1198344065 > So the isEncodedRegionName will return true when the name starts with '.'? A bit strange, but anyway, let's quick fix the problem on branch-2.4 first. > > +1 Yeah, at master branch, it is fixed. Not in branch-2 though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on pull request #4601: Backport "HBASE-27155 Improvements to low level scanner tracing" to branch-2.5
ndimiduk commented on PR #4601: URL: https://github.com/apache/hbase/pull/4601#issuecomment-1198083702 @apurtell Results from the counter-based patch and `rel/2.4.13` are in the spreadsheet and charted, FYI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-27185) Rewrite NettyRpcServer to decode rpc request with netty handler
[ https://issues.apache.org/jira/browse/HBASE-27185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27185. --- Resolution: Fixed Pushed an addendum to master and branch-2. > Rewrite NettyRpcServer to decode rpc request with netty handler > --- > > Key: HBASE-27185 > URL: https://issues.apache.org/jira/browse/HBASE-27185 > Project: HBase > Issue Type: Improvement > Components: netty, rpc >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > This is a follow on issue of HBASE-26708. > The memory leaks happens in the saslReadAndProcess method. It does not follow > a typical netty style so it is more likely to introduce bugs. > Since we agree that NettyRpcServer is the future and SimpleRpcServer should > be deprecated and finally removed, let's first implement dedicated rpc > request decoder for netty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 commented on a diff in pull request #4647: HBASE-27234 Clean up error-prone warnings in hbase-examples
Apache9 commented on code in PR #4647:
URL: https://github.com/apache/hbase/pull/4647#discussion_r931989397
##
hbase-examples/src/main/java/org/apache/hadoop/hbase/util/ClientUtils.java:
##
@@ -110,4 +109,13 @@ public static String utf8(final byte[] buf) {
}
}
+ /**
+ * Helper to translate a byte buffer to UTF8 strings
+ * @param buf byte buffer
+ * @return UTF8 decoded string value
+ */
+ public static String utf8(final ByteBuffer buf) {
+return buf.toString();
Review Comment:
I do not think this is what we want? We should copy the content a byte array
and then call the above utf8 method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Reopened] (HBASE-27185) Rewrite NettyRpcServer to decode rpc request with netty handler
[ https://issues.apache.org/jira/browse/HBASE-27185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-27185: --- This fails TestShadeSaslAuthenticationProvider on branch-2. Need an addendum. > Rewrite NettyRpcServer to decode rpc request with netty handler > --- > > Key: HBASE-27185 > URL: https://issues.apache.org/jira/browse/HBASE-27185 > Project: HBase > Issue Type: Improvement > Components: netty, rpc >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > This is a follow on issue of HBASE-26708. > The memory leaks happens in the saslReadAndProcess method. It does not follow > a typical netty style so it is more likely to introduce bugs. > Since we agree that NettyRpcServer is the future and SimpleRpcServer should > be deprecated and finally removed, let's first implement dedicated rpc > request decoder for netty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[
https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572331#comment-17572331
]
Nick Dimiduk commented on HBASE-27251:
--
I can delete these two directories and 2.4.13 master comes up.
{noformat}
$ hdfs dfs -rmr /hbase/MasterData/data/master/store/.initialized
$ hdfs dfs -rmr /hbase/MasterData/data/master/store/.tabledesc
{noformat}
> Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo`
> ---
>
> Key: HBASE-27251
> URL: https://issues.apache.org/jira/browse/HBASE-27251
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.5.0
>Reporter: Nick Dimiduk
>Priority: Critical
> Fix For: 2.4.14
>
>
> I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13
> and the master won't start. Stack trace ends in,
> {noformat}
> java.io.FileNotFoundException: File does not exist:
> /hbase/MasterData/data/master/store/.initialized/.regioninfo
>
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
>
>at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
>
>at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089)
>
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762)
>
>
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
>
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572)
>
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
>
>
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> at
> java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
>
>
> at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
>
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
>
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {noformat}
> When I examine the on-disk file system, I see,
> {noformat}
> nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/
> Found 3 items
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.initialized
> drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37
> /hbase/MasterData/data/master/store/.tabledesc
> drwxr-xr-x - nonroot supergroup
[GitHub] [hbase] Apache-HBase commented on pull request #4664: HBASE-27250 MasterRpcService#setRegionStateInMeta does not support re…
Apache-HBase commented on PR #4664: URL: https://github.com/apache/hbase/pull/4664#issuecomment-1197703491 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | master passed | | +1 :green_heart: | compile | 0m 34s | master passed | | +1 :green_heart: | shadedjars | 3m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | the patch passed | | +1 :green_heart: | compile | 0m 35s | the patch passed | | +1 :green_heart: | javac | 0m 35s | the patch passed | | +1 :green_heart: | shadedjars | 4m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 201m 41s | hbase-server in the patch passed. | | | | 219m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4664 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 119fdf34e2ba 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / a3eeab8c56 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/1/testReport/ | | Max. process+thread count | 2661 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
