Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
Apache-HBase commented on PR #5580: URL: https://github.com/apache/hbase/pull/5580#issuecomment-1855326564 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 56s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 5m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 5m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 19s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 219m 23s | hbase-server in the patch passed. | | | | 246m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5580 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6448a636536a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/2/testReport/ | | Max. process+thread count | 4755 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1855291046 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 18s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 17s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 26s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 21s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 224m 50s | hbase-server in the patch passed. | | | | 250m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b431f83dbbd8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/testReport/ | | Max. process+thread count | 4663 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28197 The configs for using meta replica can conflict [hbase]
Apache9 commented on PR #5518: URL: https://github.com/apache/hbase/pull/5518#issuecomment-1855287276 @huaxiangsun WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1855272554 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 37s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 33s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 25s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 216m 9s | hbase-server in the patch passed. | | | | 243m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6d72724f12b6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/testReport/ | | Max. process+thread count | 4498 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
NihalJain commented on code in PR #5580:
URL: https://github.com/apache/hbase/pull/5580#discussion_r1426199381
##
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/tls/X509Util.java:
##
@@ -106,15 +106,13 @@ public final class X509Util {
"hbase.server.netty.tls.supportplaintext";
/**
- * Set the SSL wrapSize for netty. This is only a maximum wrap size. Buffers
smaller
- * than this will not be consolidated, but buffers larger than this will be
split into
- * multiple wrap buffers. The netty default of 16k is not great for hbase
which tends to
- * return larger payloads than that, meaning most responses end up getting
chunked up.
- * This leads to more memory contention in netty's PoolArena.
- * See https://github.com/netty/netty/pull/13551
+ * Set the SSL wrapSize for netty. This is only a maximum wrap size. Buffers
smaller than this
+ * will not be consolidated, but buffers larger than this will be split into
multiple wrap
+ * buffers. The netty default of 16k is not great for hbase which tends to
return larger payloads
+ * than that, meaning most responses end up getting chunked up. This leads
to more memory
+ * contention in netty's PoolArena. See
https://github.com/netty/netty/pull/13551
*/
- public static final String HBASE_SERVER_NETTY_TLS_WRAP_SIZE =
-"hbase.server.netty.tsl.wrapSize";
+ public static final String HBASE_SERVER_NETTY_TLS_WRAP_SIZE =
"hbase.server.netty.tsl.wrapSize";
Review Comment:
Hi @bbeaudreault, i think you meant hbase.server.netty.**tls**.wrapSize
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28197 The configs for using meta replica can conflict [hbase]
sunhelly commented on PR #5518:
URL: https://github.com/apache/hbase/pull/5518#issuecomment-1855105270
> > > Have you check the ref guide? For me, I think setting the replica mode
to something other than NONE indicates that the user wants to use meta replica?
> >
> >
> > Thanks, @Apache9 . The ref guide doesn't bound the relation ship between
the two configs. I think, when we enabling the meta replica, we will set both
the two configs under normal circumstances. But when encountering problems
caused by meta replica, disable it throught the configs is a quick recover
method. The key problem here is that we should disable meta replica by only one
config("hbase.meta.replicas.use") or both the relevant configs. I prefer the
first way, but I'm neutral on both the ways.
>
> Maybe at client side we just do not care about `hbase.meta.replicas.use`?
The `hbase.meta.replicas.use` config is only for server side. So maybe a more
suitable way here is that, at server side, like when initialize a region server
or master, when `hbase.meta.replicas.use` is set to false, we force set the
replica mode to NONE as it is useless to use other replica modes?
Thanks， @Apache9 . I think the config `hbase.meta.replicas.use` appears
before `hbase.locator.meta.replicas.mode`, which is brought by
[HBASE-25126](https://issues.apache.org/jira/browse/HBASE-25126) to introduce
more meta access mode for clients. And at the server side, in the
initialization of regionserver and master, the `hbase.meta.replicas.use`
actively force the setting to false, see line
https://github.com/apache/hbase/blob/29bfc610d0433f720a34bc47aadca1433bbb1882/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L515
and
https://github.com/apache/hbase/blob/29bfc610d0433f720a34bc47aadca1433bbb1882/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L529.
As a result, I think the replica mode should be force set to NONE at both
server side and client side. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
Apache-HBase commented on PR #5580: URL: https://github.com/apache/hbase/pull/5580#issuecomment-1855095256 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 0s | master passed | | +1 :green_heart: | compile | 3m 0s | master passed | | +1 :green_heart: | checkstyle | 0m 47s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 54s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 3m 0s | the patch passed | | +1 :green_heart: | javac | 3m 0s | the patch passed | | +1 :green_heart: | checkstyle | 0m 43s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 53s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 35m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5580 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 2b8265688955 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1855088738 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | master passed | | +1 :green_heart: | compile | 4m 2s | master passed | | +1 :green_heart: | checkstyle | 0m 56s | master passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 3m 33s | the patch passed | | +1 :green_heart: | cc | 3m 33s | the patch passed | | -0 :warning: | javac | 2m 23s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | +1 :green_heart: | checkstyle | 0m 51s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 52s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | hbaseprotoc | 1m 18s | the patch passed | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 44m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool | | uname | Linux 0a57cae98a46 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | Max. process+thread count | 81 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28258) compact_rs not support hostname,port
[
https://issues.apache.org/jira/browse/HBASE-28258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796529#comment-17796529
]
Moran commented on HBASE-28258:
---
Here are several repair solutions：
# Disable in hbase shell, must use
# ServerName adapt to scenarios without startcode, startcode can be set to a
default value of 0，I need to evaluate if it's feasible
# RB adapt to scenarios, startcode with default 0. The evaluation scope is
smaller，more safe
> compact_rs not support hostname,port
>
>
> Key: HBASE-28258
> URL: https://issues.apache.org/jira/browse/HBASE-28258
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.2.7, 2.5.0, 2.4.3, 2.3.6
>Reporter: Moran
>Priority: Minor
>
> hbase:010:0> compact_rs 'hadoop-222-149.hfb.odeon.cn,60020'
> ERROR: begin 28, end 27, length 33
> Should have something to do with ServerName.ValueOf()，It has changed from
> HBASE-24305.
> {code:java}
> public static ServerName valueOf(final String serverName) {
> int firstSep = serverName.indexOf(SERVERNAME_SEPARATOR);
> int lastSep = serverName.lastIndexOf(SERVERNAME_SEPARATOR);
> String hostname = serverName.substring(0, firstSep);
> int port = Integer.parseInt(serverName.substring(firstSep + 1, lastSep));
> long startCode = Long.parseLong(serverName.substring(lastSep + 1));
> return valueOf(hostname, port, startCode);
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
Apache-HBase commented on PR #5580: URL: https://github.com/apache/hbase/pull/5580#issuecomment-1855069679 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 27s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 4m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 4m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 51s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 233m 43s | hbase-server in the patch passed. | | | | 258m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5580 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 08d794cab221 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/testReport/ | | Max. process+thread count | 4511 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
Apache-HBase commented on PR #5580: URL: https://github.com/apache/hbase/pull/5580#issuecomment-1855058975 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 0s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 5m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 42s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 5m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 16s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 220m 27s | hbase-server in the patch passed. | | | | 247m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5580 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cbacef3bd2de 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/testReport/ | | Max. process+thread count | 4722 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1855052992 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 25s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 22s | hbase-client in the patch passed. | | -1 :x: | unit | 227m 10s | hbase-server in the patch failed. | | | | 253m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 25ebf30d3986 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/testReport/ | | Max. process+thread count | 4684 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28258) compact_rs not support hostname,port
[
https://issues.apache.org/jira/browse/HBASE-28258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Moran updated HBASE-28258:
--
Description:
hbase:010:0> compact_rs 'hadoop-222-149.hfb.odeon.cn,60020'
ERROR: begin 28, end 27, length 33
Should have something to do with ServerName.ValueOf()，It has changed from
HBASE-24305.
{code:java}
public static ServerName valueOf(final String serverName) {
int firstSep = serverName.indexOf(SERVERNAME_SEPARATOR);
int lastSep = serverName.lastIndexOf(SERVERNAME_SEPARATOR);
String hostname = serverName.substring(0, firstSep);
int port = Integer.parseInt(serverName.substring(firstSep + 1, lastSep));
long startCode = Long.parseLong(serverName.substring(lastSep + 1));
return valueOf(hostname, port, startCode);
} {code}
was:
hbase:010:0> compact_rs 'hadoop-222-149.hfb.odeon.cn,60020'
ERROR: begin 28, end 27, length 33
Should have something to do with ServerName.ValueOf()，It has changed from
HBASE-24305.
> compact_rs not support hostname,port
>
>
> Key: HBASE-28258
> URL: https://issues.apache.org/jira/browse/HBASE-28258
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.2.7, 2.5.0, 2.4.3, 2.3.6
>Reporter: Moran
>Priority: Minor
>
> hbase:010:0> compact_rs 'hadoop-222-149.hfb.odeon.cn,60020'
> ERROR: begin 28, end 27, length 33
> Should have something to do with ServerName.ValueOf()，It has changed from
> HBASE-24305.
> {code:java}
> public static ServerName valueOf(final String serverName) {
> int firstSep = serverName.indexOf(SERVERNAME_SEPARATOR);
> int lastSep = serverName.lastIndexOf(SERVERNAME_SEPARATOR);
> String hostname = serverName.substring(0, firstSep);
> int port = Integer.parseInt(serverName.substring(firstSep + 1, lastSep));
> long startCode = Long.parseLong(serverName.substring(lastSep + 1));
> return valueOf(hostname, port, startCode);
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28258) compact_rs not support hostname,port
Moran created HBASE-28258: - Summary: compact_rs not support hostname,port Key: HBASE-28258 URL: https://issues.apache.org/jira/browse/HBASE-28258 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.3.6, 2.4.3, 2.5.0, 2.2.7 Reporter: Moran hbase:010:0> compact_rs 'hadoop-222-149.hfb.odeon.cn,60020' ERROR: begin 28, end 27, length 33 Should have something to do with ServerName.ValueOf()，It has changed from HBASE-24305. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1855046902 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 53s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 1m 33s | the patch passed | | +1 :green_heart: | javac | 1m 33s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 34s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 216m 48s | hbase-server in the patch passed. | | | | 244m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 5962d3158145 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/testReport/ | | Max. process+thread count | 4485 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28257) Memstore flushRequest can be blocked by a delayed flush scheduled by PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
[
https://issues.apache.org/jira/browse/HBASE-28257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Manning resolved HBASE-28257.
---
Resolution: Duplicate
> Memstore flushRequest can be blocked by a delayed flush scheduled by
> PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
> -
>
> Key: HBASE-28257
> URL: https://issues.apache.org/jira/browse/HBASE-28257
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 3.0.0
>Reporter: David Manning
>Priority: Minor
>
> *Steps to reproduce:*
> # Make an edit to a region.
> # Wait 1 hour + 10 seconds (default value of
> {{hbase.regionserver.optionalcacheflushinterval}} plus
> {{hbase.regionserver.flush.check.period}}.)
> # Make a very large number of edits to the region (i.e. >= 1GB, pressure the
> memstore.)
> *Expected:*
> Memstore pressure leads to flushes.
> *Result:*
> The PeriodicMemstoreFlusher has scheduled a refresh with a random delay of
> 0-5 minutes (default for
> {{hbase.regionserver.periodicmemstoreflusher.rangeofdelayseconds}}.) Memstore
> pressure flushes are blocked by the scheduled delayed flush. Client receives
> 0-5 minutes of {{RegionTooBusyExceptions}} until the delayed flush executes.
> *Logs:*
> 2023-12-13 06:00:13,573 INFO regionserver.HRegionServer -
> MemstoreFlusherChore requesting flush of because has an old
> edit so flush to free WALs after random delay 166761 ms
> 2023-12-13 06:00:53,219 DEBUG regionserver.HRegion - Flush requested on
>
> 2023-12-13 06:01:47,694 WARN regionserver.HRegion - Region is too busy due to
> exceeding memstore size limit.
> org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G,
> regionName=, server=
> at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
> ...
> repeats
> ...
> 2023-12-13 06:01:52,223 WARN regionserver.HRegion - Region is too busy due to
> exceeding memstore size limit.
> org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G,
> regionName=, server=
> at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
> ...
> 2023-12-13 06:03:00,340 INFO regionserver.HRegion - Flushing 1/1
> column families, dataSize=534.77 MB heapSize=1.00 GB
> 2023-12-13 06:03:09,634 INFO regionserver.HRegion - Finished flush of
> dataSize ~534.77 MB/560744948, heapSize ~1.00 GB/1073816296, currentSize=0
> B/0 for in 9294ms, sequenceid=21310753, compaction requested=false
> Note also this is the same cause as discussed in HBASE-16030 conversation
> https://issues.apache.org/jira/browse/HBASE-16030?focusedCommentId=15340153=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15340153
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28029 Netty SSL throughput improvement [hbase]
Apache-HBase commented on PR #5580: URL: https://github.com/apache/hbase/pull/5580#issuecomment-1854864843 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 54s | master passed | | +1 :green_heart: | compile | 3m 3s | master passed | | +1 :green_heart: | checkstyle | 0m 51s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 43s | the patch passed | | +1 :green_heart: | compile | 2m 59s | the patch passed | | +1 :green_heart: | javac | 2m 59s | the patch passed | | +1 :green_heart: | checkstyle | 0m 51s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 26s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | -1 :x: | spotless | 0m 14s | patch has 35 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 2m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 35m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5580 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 4726000c8a5b 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5580/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
Apache-HBase commented on PR #5579: URL: https://github.com/apache/hbase/pull/5579#issuecomment-1854862102 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 3m 36s | master passed | | +1 :green_heart: | checkstyle | 0m 52s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 5m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | compile | 4m 29s | the patch passed | | +1 :green_heart: | cc | 4m 29s | the patch passed | | -0 :warning: | javac | 3m 5s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | -0 :warning: | checkstyle | 0m 14s | hbase-client: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 0m 33s | hbase-server: The patch generated 3 new + 3 unchanged - 0 fixed = 6 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 53s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | hbaseprotoc | 1m 14s | the patch passed | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | -1 :x: | spotbugs | 0m 49s | hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 46m 7s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-client | | | org.apache.hadoop.hbase.client.TableDescriptorBuilder.ERASURE_CODING_POLICY isn't final but should be At TableDescriptorBuilder.java:be At TableDescriptorBuilder.java:[line 149] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5579 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool | | uname | Linux db84732f6153 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | spotbugs | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/artifact/yetus-general-check/output/new-spotbugs-hbase-client.html | | Max. process+thread count | 81 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5579/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-28257) Memstore flushRequest can be blocked by a delayed flush scheduled by PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
[
https://issues.apache.org/jira/browse/HBASE-28257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Manning reassigned HBASE-28257:
-
Assignee: (was: David Manning)
> Memstore flushRequest can be blocked by a delayed flush scheduled by
> PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
> -
>
> Key: HBASE-28257
> URL: https://issues.apache.org/jira/browse/HBASE-28257
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 3.0.0
>Reporter: David Manning
>Priority: Minor
>
> *Steps to reproduce:*
> # Make an edit to a region.
> # Wait 1 hour + 10 seconds (default value of
> {{hbase.regionserver.optionalcacheflushinterval}} plus
> {{hbase.regionserver.flush.check.period}}.)
> # Make a very large number of edits to the region (i.e. >= 1GB, pressure the
> memstore.)
> *Expected:*
> Memstore pressure leads to flushes.
> *Result:*
> The PeriodicMemstoreFlusher has scheduled a refresh with a random delay of
> 0-5 minutes (default for
> {{hbase.regionserver.periodicmemstoreflusher.rangeofdelayseconds}}.) Memstore
> pressure flushes are blocked by the scheduled delayed flush. Client receives
> 0-5 minutes of {{RegionTooBusyExceptions}} until the delayed flush executes.
> *Logs:*
> 2023-12-13 06:00:13,573 INFO regionserver.HRegionServer -
> MemstoreFlusherChore requesting flush of because has an old
> edit so flush to free WALs after random delay 166761 ms
> 2023-12-13 06:00:53,219 DEBUG regionserver.HRegion - Flush requested on
>
> 2023-12-13 06:01:47,694 WARN regionserver.HRegion - Region is too busy due to
> exceeding memstore size limit.
> org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G,
> regionName=, server=
> at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
> ...
> repeats
> ...
> 2023-12-13 06:01:52,223 WARN regionserver.HRegion - Region is too busy due to
> exceeding memstore size limit.
> org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G,
> regionName=, server=
> at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
> ...
> 2023-12-13 06:03:00,340 INFO regionserver.HRegion - Flushing 1/1
> column families, dataSize=534.77 MB heapSize=1.00 GB
> 2023-12-13 06:03:09,634 INFO regionserver.HRegion - Finished flush of
> dataSize ~534.77 MB/560744948, heapSize ~1.00 GB/1073816296, currentSize=0
> B/0 for in 9294ms, sequenceid=21310753, compaction requested=false
> Note also this is the same cause as discussed in HBASE-16030 conversation
> https://issues.apache.org/jira/browse/HBASE-16030?focusedCommentId=15340153=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15340153
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28257) Memstore flushRequest can be blocked by a delayed flush scheduled by PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
[
https://issues.apache.org/jira/browse/HBASE-28257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Manning updated HBASE-28257:
--
Description:
*Steps to reproduce:*
# Make an edit to a region.
# Wait 1 hour + 10 seconds (default value of
{{hbase.regionserver.optionalcacheflushinterval}} plus
{{hbase.regionserver.flush.check.period}}.)
# Make a very large number of edits to the region (i.e. >= 1GB, pressure the
memstore.)
*Expected:*
Memstore pressure leads to flushes.
*Result:*
The PeriodicMemstoreFlusher has scheduled a refresh with a random delay of 0-5
minutes (default for
{{hbase.regionserver.periodicmemstoreflusher.rangeofdelayseconds}}.) Memstore
pressure flushes are blocked by the scheduled delayed flush. Client receives
0-5 minutes of {{RegionTooBusyExceptions}} until the delayed flush executes.
*Logs:*
2023-12-13 06:00:13,573 INFO regionserver.HRegionServer - MemstoreFlusherChore
requesting flush of because has an old edit so flush to free WALs
after random delay 166761 ms
2023-12-13 06:00:53,219 DEBUG regionserver.HRegion - Flush requested on
2023-12-13 06:01:47,694 WARN regionserver.HRegion - Region is too busy due to
exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException:
Over memstore limit=1.0 G, regionName=, server=
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
...
repeats
...
2023-12-13 06:01:52,223 WARN regionserver.HRegion - Region is too busy due to
exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException:
Over memstore limit=1.0 G, regionName=, server=
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
...
2023-12-13 06:03:00,340 INFO regionserver.HRegion - Flushing 1/1
column families, dataSize=534.77 MB heapSize=1.00 GB
2023-12-13 06:03:09,634 INFO regionserver.HRegion - Finished flush of dataSize
~534.77 MB/560744948, heapSize ~1.00 GB/1073816296, currentSize=0 B/0 for
in 9294ms, sequenceid=21310753, compaction requested=false
Note also this is the same cause as discussed in HBASE-16030 conversation
https://issues.apache.org/jira/browse/HBASE-16030?focusedCommentId=15340153=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15340153
was:
*Steps to reproduce:*
# Make an edit to a region.
# Wait 1 hour (default value of hbase.regionserver.optionalcacheflushinterval.)
# Make a very large number of edits to the region (i.e. >= 1GB, pressure the
memstore.)
*Expected:*
Memstore pressure leads to flushes.
*Result:*
The PeriodicMemstoreFlusher has scheduled a refresh with a random delay of 0-5
minutes (default for
hbase.regionserver.periodicmemstoreflusher.rangeofdelayseconds.) Memstore
pressure flushes are blocked by the scheduled delayed flush. Client receives
0-5 minutes of RegionTooBusyExceptions until the delayed flush executes.
*Logs:*
2023-12-13 06:00:13,573 INFO regionserver.HRegionServer - MemstoreFlusherChore
requesting flush of because has an old edit so flush to free WALs
after random delay 166761 ms
2023-12-13 06:00:53,219 DEBUG regionserver.HRegion - Flush requested on
2023-12-13 06:01:47,694 WARN regionserver.HRegion - Region is too busy due to
exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException:
Over memstore limit=1.0 G, regionName=, server=
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
...
repeats
...
2023-12-13 06:01:52,223 WARN regionserver.HRegion - Region is too busy due to
exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException:
Over memstore limit=1.0 G, regionName=, server=
at org.apache.hadoop.hbase.regionserver.HRegion.checkResources
...
2023-12-13 06:03:00,340 INFO regionserver.HRegion - Flushing 1/1
column families, dataSize=534.77 MB heapSize=1.00 GB
2023-12-13 06:03:09,634 INFO regionserver.HRegion - Finished flush of dataSize
~534.77 MB/560744948, heapSize ~1.00 GB/1073816296, currentSize=0 B/0 for
in 9294ms, sequenceid=21310753, compaction requested=false
Note also this is the same cause as discussed in HBASE-16030 conversation
https://issues.apache.org/jira/browse/HBASE-16030?focusedCommentId=15340153=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15340153
> Memstore flushRequest can be blocked by a delayed flush scheduled by
> PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
> -
>
> Key: HBASE-28257
> URL: https://issues.apache.org/jira/browse/HBASE-28257
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 3.0.0
>Reporter: David Manning
>Assignee: David Manning
>Priority: Minor
>
> *Steps to reproduce:*
> # Make an edit to a region.
> #
[jira] [Created] (HBASE-28257) Memstore flushRequest can be blocked by a delayed flush scheduled by PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes
David Manning created HBASE-28257: - Summary: Memstore flushRequest can be blocked by a delayed flush scheduled by PeriodicMemstoreFlusher, RegionTooBusyExceptions for up to 5 minutes Key: HBASE-28257 URL: https://issues.apache.org/jira/browse/HBASE-28257 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 3.0.0 Reporter: David Manning Assignee: David Manning *Steps to reproduce:* # Make an edit to a region. # Wait 1 hour (default value of hbase.regionserver.optionalcacheflushinterval.) # Make a very large number of edits to the region (i.e. >= 1GB, pressure the memstore.) *Expected:* Memstore pressure leads to flushes. *Result:* The PeriodicMemstoreFlusher has scheduled a refresh with a random delay of 0-5 minutes (default for hbase.regionserver.periodicmemstoreflusher.rangeofdelayseconds.) Memstore pressure flushes are blocked by the scheduled delayed flush. Client receives 0-5 minutes of RegionTooBusyExceptions until the delayed flush executes. *Logs:* 2023-12-13 06:00:13,573 INFO regionserver.HRegionServer - MemstoreFlusherChore requesting flush of because has an old edit so flush to free WALs after random delay 166761 ms 2023-12-13 06:00:53,219 DEBUG regionserver.HRegion - Flush requested on 2023-12-13 06:01:47,694 WARN regionserver.HRegion - Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G, regionName=, server= at org.apache.hadoop.hbase.regionserver.HRegion.checkResources ... repeats ... 2023-12-13 06:01:52,223 WARN regionserver.HRegion - Region is too busy due to exceeding memstore size limit. org.apache.hadoop.hbase.RegionTooBusyException: Over memstore limit=1.0 G, regionName=, server= at org.apache.hadoop.hbase.regionserver.HRegion.checkResources ... 2023-12-13 06:03:00,340 INFO regionserver.HRegion - Flushing 1/1 column families, dataSize=534.77 MB heapSize=1.00 GB 2023-12-13 06:03:09,634 INFO regionserver.HRegion - Finished flush of dataSize ~534.77 MB/560744948, heapSize ~1.00 GB/1073816296, currentSize=0 B/0 for in 9294ms, sequenceid=21310753, compaction requested=false Note also this is the same cause as discussed in HBASE-16030 conversation https://issues.apache.org/jira/browse/HBASE-16030?focusedCommentId=15340153=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-15340153 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-20034) Make periodic flusher delay configurable
[ https://issues.apache.org/jira/browse/HBASE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Manning updated HBASE-20034: -- Resolution: Duplicate Status: Resolved (was: Patch Available) > Make periodic flusher delay configurable > > > Key: HBASE-20034 > URL: https://issues.apache.org/jira/browse/HBASE-20034 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 3.0.0-alpha-1 >Reporter: Vincent Poon >Assignee: Vincent Poon >Priority: Major > Attachments: HBASE-20034.branch-1.patch, HBASE-20034.master.patch > > > PeriodicMemstoreFlusher is currently configured to flush with a random delay > of up to 5 minutes. Make this configurable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28216) HDFS erasure coding support for table data dirs
[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault reassigned HBASE-28216:
-
Assignee: Bryan Beaudreault
> HDFS erasure coding support for table data dirs
> ---
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy {}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28216) HDFS erasure coding support for table data dirs
[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796482#comment-17796482
]
Bryan Beaudreault commented on HBASE-28216:
---
Thanks for the input, that's fair. I've pushed a PR which supports just
table-level EC.
> HDFS erasure coding support for table data dirs
> ---
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Priority: Major
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy {}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28216) HDFS erasure coding support for table data dirs
[
https://issues.apache.org/jira/browse/HBASE-28216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-28216:
--
Labels: patch-available (was: )
> HDFS erasure coding support for table data dirs
> ---
>
> Key: HBASE-28216
> URL: https://issues.apache.org/jira/browse/HBASE-28216
> Project: HBase
> Issue Type: New Feature
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
>
> [Erasure
> coding|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html]
> (EC) is a hadoop-3 feature which can drastically reduce storage
> requirements, at the expense of locality. At my company we have a few hbase
> clusters which are extremely data dense and take mostly write traffic, fewer
> reads (cold data). We'd like to reduce the cost of these clusters, and EC is
> a great way to do that since it can reduce replication related storage costs
> by 50%.
> It's possible to enable EC policies on sub directories of HDFS. One can
> manually set this with {{{}hdfs ec -setPolicy -path
> /hbase/data/default/usertable -policy {}}}. This can work without any
> hbase support.
> One problem with that is a lack of visibility by operators into which tables
> might have EC enabled. I think this is where HBase can help. Here's my
> proposal:
> * Add a new TableDescriptor and ColumnDescriptor field ERASURE_CODING_POLICY
> * In ModifyTableProcedure preflightChecks, if ERASURE_CODING_POLICY is set,
> verify that the requested policy is available and enabled via
> DistributedFileSystem.
> getErasureCodingPolicies().
> * During ModifyTableProcedure, add a new state for
> MODIFY_TABLE_SYNC_ERASURE_CODING_POLICY.
> ** When adding or changing a policy, use DistributedFileSystem.
> setErasureCodingPolicy to sync it for the data and archive dir of that table
> (or column in table)
> ** When removing the property or setting it to empty, use
> DistributedFileSystem.
> unsetErasureCodingPolicy to remove it from the data and archive dir.
> Since this new API is in hadoop-3 only, we'll need to add a reflection
> wrapper class for managing the calls and verifying that the API is available.
> We'll similarly do that API check in preflightChecks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] HBASE-28216 HDFS erasure coding support for table data dirs [hbase]
bbeaudreault opened a new pull request, #5579: URL: https://github.com/apache/hbase/pull/5579 Since we require hadoop-3 for master and branch-3, I can use the EC APIs directly. If we want to backport to branch-2, we'll need to figure out how to do this with reflection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28252) Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase script
[
https://issues.apache.org/jira/browse/HBASE-28252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796458#comment-17796458
]
Hudson commented on HBASE-28252:
Results for branch master
[build #962 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase
> script
> -
>
> Key: HBASE-28252
> URL: https://issues.apache.org/jira/browse/HBASE-28252
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> As noted in HBASE-28247, HBase can run into module permission issues that are
> not handled in the current JDK11 options in the hbase startup script.
> The surefire test config also includes some JDK17 specific options, we should
> also add those as needed.
> We are not yet aware of any additional JVM options required by Java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28199) Phase I: Suspend TRSP and SCP when updating meta
[
https://issues.apache.org/jira/browse/HBASE-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796462#comment-17796462
]
Hudson commented on HBASE-28199:
Results for branch master
[build #962 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Phase I: Suspend TRSP and SCP when updating meta
>
>
> Key: HBASE-28199
> URL: https://issues.apache.org/jira/browse/HBASE-28199
> Project: HBase
> Issue Type: Sub-task
> Components: master, proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-beta-1
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28244) ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
[
https://issues.apache.org/jira/browse/HBASE-28244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796461#comment-17796461
]
Hudson commented on HBASE-28244:
Results for branch master
[build #962 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
> -
>
> Key: HBASE-28244
> URL: https://issues.apache.org/jira/browse/HBASE-28244
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-beta-1
>
>
> In ProcedureTestingUtility.restart, we will reuse the same ProcedureExecutor,
> so when restarting, we need to make sure that all procedures are not executed
> and then clear the scheduler.
> But after HBASE-28199, we may add procedures back to scheduler after a
> CompletableFuture is completed, so even if all the PEWorker are terminated we
> could still add things to scheduler, which may break some tests.
> We need to find a way to deal with this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796459#comment-17796459
]
Hudson commented on HBASE-28247:
Results for branch master
[build #962 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28241) The snapshot operation encountered an NPE and failed.
[
https://issues.apache.org/jira/browse/HBASE-28241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796460#comment-17796460
]
Hudson commented on HBASE-28241:
Results for branch master
[build #962 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/962/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The snapshot operation encountered an NPE and failed.
> -
>
> Key: HBASE-28241
> URL: https://issues.apache.org/jira/browse/HBASE-28241
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Executing a merge region before the snapshot operation is completed will
> result in an NPE error for the snapshot operation and cause it to fail
> Triggering logic
> # Execute the snapshot command.
> # Merge regions before the snapshot is completed.
> # After the merge is completed, the two parent regions will be cleaned up.
> # An NPE will be reported when the snapshot operation is performed on that
> parent region. The log is as follow:
> {code:java}
> 2023-11-20T23:26:07,061 ERROR [PEWorker-13] procedure2.ProcedureExecutor:
> CODE-BUG: Uncaught runtime exception: pid=238720, ppid=238714,
> state=RUNNABLE, hasLock=true; SnapshotRegionProcedure
> c47539f1d6032ba1a037f5279c22baa0
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.master.procedure.SnapshotRegionProcedure.execute(SnapshotRegionProcedure.java:160)
> ~[hbase-server-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.master.procedure.SnapshotRegionProcedure.execute(SnapshotRegionProcedure.java:58)
> ~[hbase-server-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> ~[hbase-common-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28241) The snapshot operation encountered an NPE and failed.
[
https://issues.apache.org/jira/browse/HBASE-28241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796438#comment-17796438
]
Hudson commented on HBASE-28241:
Results for branch branch-3
[build #101 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The snapshot operation encountered an NPE and failed.
> -
>
> Key: HBASE-28241
> URL: https://issues.apache.org/jira/browse/HBASE-28241
> Project: HBase
> Issue Type: Bug
> Components: proc-v2, snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Executing a merge region before the snapshot operation is completed will
> result in an NPE error for the snapshot operation and cause it to fail
> Triggering logic
> # Execute the snapshot command.
> # Merge regions before the snapshot is completed.
> # After the merge is completed, the two parent regions will be cleaned up.
> # An NPE will be reported when the snapshot operation is performed on that
> parent region. The log is as follow:
> {code:java}
> 2023-11-20T23:26:07,061 ERROR [PEWorker-13] procedure2.ProcedureExecutor:
> CODE-BUG: Uncaught runtime exception: pid=238720, ppid=238714,
> state=RUNNABLE, hasLock=true; SnapshotRegionProcedure
> c47539f1d6032ba1a037f5279c22baa0
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.master.procedure.SnapshotRegionProcedure.execute(SnapshotRegionProcedure.java:160)
> ~[hbase-server-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.master.procedure.SnapshotRegionProcedure.execute(SnapshotRegionProcedure.java:58)
> ~[hbase-server-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> ~[hbase-common-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT]
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
> ~[hbase-procedure-3.0.0-beta-1-SNAPSHOT.jar:3.0.0-beta-1-SNAPSHOT] {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28244) ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
[
https://issues.apache.org/jira/browse/HBASE-28244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796439#comment-17796439
]
Hudson commented on HBASE-28244:
Results for branch branch-3
[build #101 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
> -
>
> Key: HBASE-28244
> URL: https://issues.apache.org/jira/browse/HBASE-28244
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-beta-1
>
>
> In ProcedureTestingUtility.restart, we will reuse the same ProcedureExecutor,
> so when restarting, we need to make sure that all procedures are not executed
> and then clear the scheduler.
> But after HBASE-28199, we may add procedures back to scheduler after a
> CompletableFuture is completed, so even if all the PEWorker are terminated we
> could still add things to scheduler, which may break some tests.
> We need to find a way to deal with this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28199) Phase I: Suspend TRSP and SCP when updating meta
[
https://issues.apache.org/jira/browse/HBASE-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796440#comment-17796440
]
Hudson commented on HBASE-28199:
Results for branch branch-3
[build #101 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/101/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Phase I: Suspend TRSP and SCP when updating meta
>
>
> Key: HBASE-28199
> URL: https://issues.apache.org/jira/browse/HBASE-28199
> Project: HBase
> Issue Type: Sub-task
> Components: master, proc-v2, Region Assignment
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-beta-1
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796425#comment-17796425
]
Hudson commented on HBASE-28247:
Results for branch branch-2.5
[build #448 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/448/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/448/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/448/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/448/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/448/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28240 Phase II: Suspend TRSP and SCP when they can not get regi… [hbase]
Apache-HBase commented on PR #5559: URL: https://github.com/apache/hbase/pull/5559#issuecomment-1854538308 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 17s | master passed | | +1 :green_heart: | compile | 0m 37s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 36s | the patch passed | | +1 :green_heart: | javac | 0m 36s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 227m 40s | hbase-server in the patch passed. | | | | 249m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5559 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 737104eb8395 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/testReport/ | | Max. process+thread count | 4530 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28240 Phase II: Suspend TRSP and SCP when they can not get regi… [hbase]
Apache-HBase commented on PR #5559: URL: https://github.com/apache/hbase/pull/5559#issuecomment-1854527568 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 42s | master passed | | +1 :green_heart: | compile | 0m 44s | master passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 43s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed | | +1 :green_heart: | javac | 0m 45s | the patch passed | | +1 :green_heart: | shadedjars | 5m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 218m 57s | hbase-server in the patch passed. | | | | 241m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5559 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2de45269292f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/testReport/ | | Max. process+thread count | 4743 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28187 NPE when flushing a non-existing column family [hbase]
Apache-HBase commented on PR #5553: URL: https://github.com/apache/hbase/pull/5553#issuecomment-1854462441 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 10s | master passed | | +1 :green_heart: | compile | 1m 9s | master passed | | +1 :green_heart: | shadedjars | 6m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 6m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 37s | hbase-client in the patch passed. | | -1 :x: | unit | 262m 37s | hbase-server in the patch failed. | | | | 291m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5553 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0693a000f8b3 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/testReport/ | | Max. process+thread count | 4709 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28187 NPE when flushing a non-existing column family [hbase]
Apache-HBase commented on PR #5553: URL: https://github.com/apache/hbase/pull/5553#issuecomment-1854459005 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 6m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 7s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 5m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 53s | hbase-client in the patch passed. | | -1 :x: | unit | 258m 36s | hbase-server in the patch failed. | | | | 289m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5553 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 96639fe87543 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/testReport/ | | Max. process+thread count | 4716 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28256 Enhance ByteBufferUtils.readVLong to read 8 bytes at a time [hbase]
wchevreuil commented on code in PR #5576:
URL: https://github.com/apache/hbase/pull/5576#discussion_r1425691507
##
hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferUtils.java:
##
@@ -468,38 +468,55 @@ public static void writeVLong(ByteBuffer out, long i) {
}
}
- private interface ByteVisitor {
-byte get();
- }
-
- private static long readVLong(ByteVisitor visitor) {
-byte firstByte = visitor.get();
+ /**
Review Comment:
Nit: could we keep using the Visitor pattern here? It seems both methods
only differ in two lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28240 Phase II: Suspend TRSP and SCP when they can not get regi… [hbase]
Apache-HBase commented on PR #5559: URL: https://github.com/apache/hbase/pull/5559#issuecomment-1854130317 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | master passed | | +1 :green_heart: | compile | 2m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 34s | master passed | | +1 :green_heart: | spotless | 0m 40s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 2m 28s | the patch passed | | +1 :green_heart: | javac | 2m 28s | the patch passed | | +1 :green_heart: | checkstyle | 0m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 1s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | -1 :x: | spotbugs | 1m 35s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 32m 40s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Condition.await() not in loop in org.apache.hadoop.hbase.master.assignment.RegionStateNodeLock$1.await() At RegionStateNodeLock.java:org.apache.hadoop.hbase.master.assignment.RegionStateNodeLock$1.await() At RegionStateNodeLock.java:[line 162] | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5559 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 7f102e51bbea 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | spotbugs | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5559/6/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28243 Bump jackson version to 2.15.2 [hbase]
NihalJain commented on PR #5562: URL: https://github.com/apache/hbase/pull/5562#issuecomment-1854086599 > +1 unit405m 44sroot in the patch passed. > 434m 40s > Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5562/1/testReport/ Also failures should not be related, build 1 had passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28243 Bump jackson version to 2.15.2 [hbase]
NihalJain commented on PR #5562: URL: https://github.com/apache/hbase/pull/5562#issuecomment-1854081066 > Even this should go to all branches having https://issues.apache.org/jira/browse/HBASE-22138 i.e master, branch-3, branch-2, branch-2.6, branch-2.5, branch-2.4, please let me know what do you think? Gentle ping @Apache9, @apurtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28245 Sync internal protobuf version for hbase to be same as hb… [hbase]
NihalJain commented on PR #5564: URL: https://github.com/apache/hbase/pull/5564#issuecomment-1854079926 > In my opinion this should go to all branches having https://issues.apache.org/jira/browse/HBASE-22138 i.e master, branch-3, branch-2, branch-2.6, branch-2.5, branch-2.4, please let me know what do you think? Gentle ping @Apache9, @apurtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28249 Bump jruby to 9.3.13.0 and related joni and jcodings to 2… [hbase]
NihalJain commented on PR #5568: URL: https://github.com/apache/hbase/pull/5568#issuecomment-1854078208 Could you please review the new change @chrajeshbabu. > Given build is success, plan is to put this on all branches having https://issues.apache.org/jira/browse/HBASE-27585 i.e. master, branch-3, branch-2 and branch-2.6. > > Please let me know if any concerns. Also, gentle ping @Apache9, @apurtell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28256 Enhance ByteBufferUtils.readVLong to read 8 bytes at a time [hbase]
jbewing commented on PR #5576: URL: https://github.com/apache/hbase/pull/5576#issuecomment-1854065824 > In general, if we choose to use VLong, it means usually the value will be small, for example, only 1 or 2 bytes, so I wonder whether your JMH covers the most common scenario? Benchmark does cover a variety of vLong sizes. The microbenchmarks above have breakdowns for the following vLongs: 9, 512, 2146483640, and 1700104028981 which can be encoded as 1 byte, 3 byte, 5 byte, and 7 bytes respectively (so a wide range). In practice, ByteBufferUtils.readVLong is only used in the HBase codebase to decode the memstoreTs for the CopyKey, Diff, Fast Diff, Prefix, and RIV1 DBE encodings so performance when decoding a 7 byte vLong (an actual epoch millis timestamp is 6 bytes, but there is a 1 byte overhead for the vLong encoding) is the most important gauge of how this will affect performance. I agree that generally when vLong is used, the caller believes that a fair amount of the data will be smaller than 7 or 8 bytes so the vLong encoding saves space. However, in practice the ByteBufferUtils.readVLong method isn't used in that way. > For random long value which is encoded as VLong, I think read 8 bytes at once will be faster, but what if the values are often only 1 or 2 bytes? From what the benchmarks show, there is absolutely no regression at the 1 byte case as the code is identical up to that point. When a 2 byte vLong is read, the benchmarks show a small performance penalty. It seems like the cost of reading an 8 byte word on a modern machine is about the same cost as reading a single byte word, so performance hit isn't super drastic (at least as benchmarked on my machine). Again, because of how this particular method is used to decode vLongs in the HBase codebase, we should be focused on the performance for the 1700104028981 vLong decoding case as it most closely represents the performance for decoding a timestamp. In any case, if we were to start using vLongs more widely, the performance for the 2 byte case for the optimized method isn't really that much worse than the current behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28256 Enhance ByteBufferUtils.readVLong to read 8 bytes at a time [hbase]
Apache9 commented on PR #5576: URL: https://github.com/apache/hbase/pull/5576#issuecomment-1854034262 In general, if we choose to use VLong, it means usually the value will be small, for example, only 1 or 2 bytes, so I wonder whether your JMH covers the most common scenario? For random long value which is encoded as VLong, I think read 8 bytes at once will be faster, but what if the values are often only 1 or 2 bytes? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28213) Evalueate using hbase-shaded-client-byo-hadoop for Spark connector
[ https://issues.apache.org/jira/browse/HBASE-28213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated HBASE-28213: Summary: Evalueate using hbase-shaded-client-byo-hadoop for Spark connector (was: Evalue using hbase-shaded-client-byo-hadoop for Spark connector) > Evalueate using hbase-shaded-client-byo-hadoop for Spark connector > -- > > Key: HBASE-28213 > URL: https://issues.apache.org/jira/browse/HBASE-28213 > Project: HBase > Issue Type: Improvement > Components: spark >Reporter: Istvan Toth >Priority: Major > > Since 3.2 Spark now uses hadoop-client-api and hadoop-client-runtime. > While we don't actually specify what HBase libraries are needed on the Spark > client side for the connector, at least the Cloudera docs specify the classes > provided by "hbase mapredcp" > which includes the full unshaded Hadoop JAR set. > Investigate whether *hbase-shaded-client-byo-hadoop* and the > *hbase-client-api* and *hbase-client-runtime* is enough for the connector, > and if yes, document how to set the Spark classpath. > Alternatively, if *hbase-shaded-client-byo-hadoop* is not enough, check if > *hbase-shaded-mapreduce* plus the above two shaded Hadoop client JAR provides > everything needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28244) ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
[ https://issues.apache.org/jira/browse/HBASE-28244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28244. --- Fix Version/s: 3.0.0-beta-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-3. Thanks [~GeorryHuang] for reviewing! > ProcedureTestingUtility.restart is broken sometimes after HBASE-28199 > - > > Key: HBASE-28244 > URL: https://issues.apache.org/jira/browse/HBASE-28244 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-beta-1 > > > In ProcedureTestingUtility.restart, we will reuse the same ProcedureExecutor, > so when restarting, we need to make sure that all procedures are not executed > and then clear the scheduler. > But after HBASE-28199, we may add procedures back to scheduler after a > CompletableFuture is completed, so even if all the PEWorker are terminated we > could still add things to scheduler, which may break some tests. > We need to find a way to deal with this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28244) ProcedureTestingUtility.restart is broken sometimes after HBASE-28199
[ https://issues.apache.org/jira/browse/HBASE-28244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28244: -- Component/s: proc-v2 test > ProcedureTestingUtility.restart is broken sometimes after HBASE-28199 > - > > Key: HBASE-28244 > URL: https://issues.apache.org/jira/browse/HBASE-28244 > Project: HBase > Issue Type: Sub-task > Components: proc-v2, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > In ProcedureTestingUtility.restart, we will reuse the same ProcedureExecutor, > so when restarting, we need to make sure that all procedures are not executed > and then clear the scheduler. > But after HBASE-28199, we may add procedures back to scheduler after a > CompletableFuture is completed, so even if all the PEWorker are terminated we > could still add things to scheduler, which may break some tests. > We need to find a way to deal with this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28187 NPE when flushing a non-existing column family [hbase]
Apache-HBase commented on PR #5553: URL: https://github.com/apache/hbase/pull/5553#issuecomment-1853951661 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 51s | master passed | | +1 :green_heart: | compile | 3m 9s | master passed | | +1 :green_heart: | checkstyle | 0m 55s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 42s | the patch passed | | -1 :x: | compile | 2m 20s | hbase-server in the patch failed. | | -0 :warning: | javac | 2m 20s | hbase-server in the patch failed. | | +1 :green_heart: | checkstyle | 0m 52s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 30s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 36m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5553 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 876f63abf57f 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 29bfc610d0 | | Default Java | Eclipse Adoptium-11.0.17+8 | | compile | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | Max. process+thread count | 77 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5553/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28187 NPE when flushing a non-existing column family [hbase]
guluo2016 commented on PR #5553: URL: https://github.com/apache/hbase/pull/5553#issuecomment-1853917563 > For user request from Admin interface or shell, I think we should reject the request and give users a clear error message, this should be done first. > > On the CP part, I'm neutral on whether we should deal with the problem. CP is a low level API only expected to be used by experts, so it is the CP's author's duty to not break HBase by wrong input. Of course, I think it is also OK to make our code more robust if it does not introduce overhead or complicate logic. But as you said, we still need to discuss how to deal with the problem, at least I do not think returning success is the correct way... Thanks for your repla, I update this PR. At client side, i have added check if column family exists and it will return error as early as possible when having non-existing family. And at region side, I alse have added check column family exists and it will return CANNOT_FLUSH for making hbase code more robust. Thanks again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Hbase 28252 2.4 [hbase]
stoty closed pull request #5572: Hbase 28252 2.4 URL: https://github.com/apache/hbase/pull/5572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
