[GitHub] [hbase] Apache-HBase commented on pull request #5007: HBASE-27534 Determine too large requests by response block size rather than just cell size
Apache-HBase commented on PR #5007: URL: https://github.com/apache/hbase/pull/5007#issuecomment-1416929693 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 27s | master passed | | +1 :green_heart: | compile | 4m 8s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 52s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 10s | the patch passed | | +1 :green_heart: | compile | 4m 1s | the patch passed | | +1 :green_heart: | cc | 4m 1s | the patch passed | | -0 :warning: | javac | 2m 22s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 41s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | hbaseprotoc | 1m 41s | the patch passed | | +1 :green_heart: | spotless | 0m 37s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 5m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 54m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5007/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5007 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool | | uname | Linux aefd0faa157a 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7b26b2aefe | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5007/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5007/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27580) Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
[
https://issues.apache.org/jira/browse/HBASE-27580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684236#comment-17684236
]
Hudson commented on HBASE-27580:
Results for branch branch-2
[build #736 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/736/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/736/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/736/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/736/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/736/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
> --
>
> Key: HBASE-27580
> URL: https://issues.apache.org/jira/browse/HBASE-27580
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 3.0.0-alpha-4, 2.4.17, 2.5.4
>
>
> This is easily reproducible, see test below. All you need to do is create a
> table with a DBE, write some puts with setTTL, then do a reverse scan. All 3
> compressing DBE's fail.
> PREFIX throws an exception:
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1355)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1337)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.get(LRUDictionary.java:153)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.access$000(LRUDictionary.java:79)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary.getEntry(LRUDictionary.java:43)
> at
> org.apache.hadoop.hbase.io.TagCompressionContext.uncompressTags(TagCompressionContext.java:152)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.decodeTags(BufferedDataBlockEncoder.java:819)
> at
> org.apache.hadoop.hbase.io.encoding.PrefixKeyDeltaEncoder$SeekerStateBufferedEncodedSeeker.decodeNext(PrefixKeyDeltaEncoder.java:209)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> DIFF throws an exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decode(DiffKeyDeltaEncoder.java:431)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decodeNext(DiffKeyDeltaEncoder.java:502)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> FAST_DIFF throws a similar exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decode(FastDiffDeltaEncoder.java:424)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decodeNext(FastDiffDeltaEncoder.java:490)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
[jira] [Commented] (HBASE-27580) Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
[
https://issues.apache.org/jira/browse/HBASE-27580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684233#comment-17684233
]
Hudson commented on HBASE-27580:
Results for branch branch-2.4
[build #509 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/509/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/509/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/509/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/509/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/509/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
> --
>
> Key: HBASE-27580
> URL: https://issues.apache.org/jira/browse/HBASE-27580
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 3.0.0-alpha-4, 2.4.17, 2.5.4
>
>
> This is easily reproducible, see test below. All you need to do is create a
> table with a DBE, write some puts with setTTL, then do a reverse scan. All 3
> compressing DBE's fail.
> PREFIX throws an exception:
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1355)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1337)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.get(LRUDictionary.java:153)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.access$000(LRUDictionary.java:79)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary.getEntry(LRUDictionary.java:43)
> at
> org.apache.hadoop.hbase.io.TagCompressionContext.uncompressTags(TagCompressionContext.java:152)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.decodeTags(BufferedDataBlockEncoder.java:819)
> at
> org.apache.hadoop.hbase.io.encoding.PrefixKeyDeltaEncoder$SeekerStateBufferedEncodedSeeker.decodeNext(PrefixKeyDeltaEncoder.java:209)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> DIFF throws an exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decode(DiffKeyDeltaEncoder.java:431)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decodeNext(DiffKeyDeltaEncoder.java:502)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> FAST_DIFF throws a similar exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decode(FastDiffDeltaEncoder.java:424)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decodeNext(FastDiffDeltaEncoder.java:490)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
>
[GitHub] [hbase] bbeaudreault opened a new pull request, #5007: HBASE-27534 Determine too large requests by response block size rather than just cell size
bbeaudreault opened a new pull request, #5007: URL: https://github.com/apache/hbase/pull/5007 Mostly all boilerplate, except for the threshold check in RpcServer.java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4966: HBASE-27216 Revisit the ReplicationSyncUp tool
Apache-HBase commented on PR #4966: URL: https://github.com/apache/hbase/pull/4966#issuecomment-1416832500 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 38s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 1m 10s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 40s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 24s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 22s | hbase-replication in the patch passed. | | -1 :x: | unit | 221m 41s | hbase-server in the patch failed. | | | | 245m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4966 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 8bc06da9cf66 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / befa513f6f | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/testReport/ | | Max. process+thread count | 4774 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-replication hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25815) RSGroupBasedLoadBalancer online status never updates after being set to true for the first time
[ https://issues.apache.org/jira/browse/HBASE-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-25815: - Priority: Major (was: Minor) > RSGroupBasedLoadBalancer online status never updates after being set to true > for the first time > --- > > Key: HBASE-25815 > URL: https://issues.apache.org/jira/browse/HBASE-25815 > Project: HBase > Issue Type: Bug >Reporter: Caroline Zhou >Assignee: Caroline Zhou >Priority: Major > > Once the RSGroupBasedLoadBalancer is “online” (it has found the hbase:meta > and hbase:rsgroup tables), it will never update the status again. That means > if hbase:meta or hbase:rsgroup ever go offline, the balancer doesn’t update > its status to “offline,” so some of the code paths will go through the > “online” code path even though the catalog tables aren’t available to be read > from or written to (in particular, anything that calls > RSGroupInfoManagerImpl#flushConfig). > Also, in the RSGroupInfoManagerImpl#flushConfig code path, the call to write > to hbase:rsgroup comes before the update to the rsGroupMap and tableMap which > are stored in memory (see order of [these lines of > code|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java#L664-L670]), > so if hbase:rsgroup goes offline after the RSGroupBasedLoadBalancer is > already marked as “online,” exceptions thrown while trying to write to an > offline hbase:rsgroup table prevent the in-memory rsGroupMap and tableMap > from being updated. In terms of the order just mentioned, in-memory state > should be updated first. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27570) Unify tracking of block IO across all read request types
[ https://issues.apache.org/jira/browse/HBASE-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault updated HBASE-27570: -- Labels: (was: patch-available) > Unify tracking of block IO across all read request types > > > Key: HBASE-27570 > URL: https://issues.apache.org/jira/browse/HBASE-27570 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Various requests in RSRpcServices use > [addSize|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L1303-L1335] > in an attempt to estimate block IO of the request by pulling the capacity of > the underlying cell buffer when that buffer changes. This is just an estimate > and can be inaccurate in certain circumstances, such as when the ordering of > gets in a multiget causes the same buffer to be counted twice. > As of HBASE-27558, we now have a hook in StoreScanner for tracking block IO > via HFileReaderImpl.reportBlockSize method call. We use this for getting > block IO into ScannerContext for tracking limits. We can add a call to > RpcCall.incrementResponseBlockSize here so that we can get an accurate > accounting of block IO for all request types. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27570) Unify tracking of block IO across all read request types
[ https://issues.apache.org/jira/browse/HBASE-27570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault updated HBASE-27570: -- Fix Version/s: 2.6.0 3.0.0-alpha-4 Release Note: Unifies block IO accounting code across all request types. As a result, the behavior of MultiActionResultTooLarge handling may change slightly. In certain circumstances, a multiget might trip the "too large" threshold sooner than it used to in the past. This should largely be transparent to users since the client automatically retries, and the behavior change should only be noticeable for very large multigets that fetch many rows from the same block. Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks for the review [~zhangduo] > Unify tracking of block IO across all read request types > > > Key: HBASE-27570 > URL: https://issues.apache.org/jira/browse/HBASE-27570 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Various requests in RSRpcServices use > [addSize|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L1303-L1335] > in an attempt to estimate block IO of the request by pulling the capacity of > the underlying cell buffer when that buffer changes. This is just an estimate > and can be inaccurate in certain circumstances, such as when the ordering of > gets in a multiget causes the same buffer to be counted twice. > As of HBASE-27558, we now have a hook in StoreScanner for tracking block IO > via HFileReaderImpl.reportBlockSize method call. We use this for getting > block IO into ScannerContext for tracking limits. We can add a call to > RpcCall.incrementResponseBlockSize here so that we can get an accurate > accounting of block IO for all request types. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27580) Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
[
https://issues.apache.org/jira/browse/HBASE-27580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-27580:
--
Fix Version/s: 3.0.0-alpha-4
2.4.17
2.5.4
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to branch-2.4+. Thanks for the review [~zhangduo]
> Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
> --
>
> Key: HBASE-27580
> URL: https://issues.apache.org/jira/browse/HBASE-27580
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 3.0.0-alpha-4, 2.4.17, 2.5.4
>
>
> This is easily reproducible, see test below. All you need to do is create a
> table with a DBE, write some puts with setTTL, then do a reverse scan. All 3
> compressing DBE's fail.
> PREFIX throws an exception:
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1355)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1337)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.get(LRUDictionary.java:153)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary$BidirectionalLRUMap.access$000(LRUDictionary.java:79)
> at
> org.apache.hadoop.hbase.io.util.LRUDictionary.getEntry(LRUDictionary.java:43)
> at
> org.apache.hadoop.hbase.io.TagCompressionContext.uncompressTags(TagCompressionContext.java:152)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.decodeTags(BufferedDataBlockEncoder.java:819)
> at
> org.apache.hadoop.hbase.io.encoding.PrefixKeyDeltaEncoder$SeekerStateBufferedEncodedSeeker.decodeNext(PrefixKeyDeltaEncoder.java:209)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> DIFF throws an exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decode(DiffKeyDeltaEncoder.java:431)
> at
> org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decodeNext(DiffKeyDeltaEncoder.java:502)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> FAST_DIFF throws a similar exception:
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -22 is
> negative
> at java.base/java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1140)
> at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decode(FastDiffDeltaEncoder.java:424)
> at
> org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$FastDiffSeekerStateBufferedEncodedSeeker.decodeNext(FastDiffDeltaEncoder.java:490)
> at
> org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:920)
> {code}
>
> Reproduce with:
> {code:java}
> private static final Logger LOG = LoggerFactory.getLogger(TestTags.class);
> @Test
> public void testReverseScanWithDBE() throws IOException {
> byte[] family = Bytes.toBytes("0");
> Configuration conf = new Configuration(TEST_UTIL.getConfiguration());
> conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
> try (Connection connection = ConnectionFactory.createConnection(conf)) {
> for (DataBlockEncoding encoding : DataBlockEncoding.values()) {
> boolean isCompressing = encoding != DataBlockEncoding.NONE && encoding
> != DataBlockEncoding.ROW_INDEX_V1;
> try {
> testReverseScanWithDBE(connection, encoding, family);
> if (isCompressing) {
> Assert.fail("Expected to throw exception for DBE " + encoding);
> }
> } catch (Exception e) {
> LOG.info("Got exception for DBE {}", encoding, e);
> assertTrue("Only expected compressing encodings
[GitHub] [hbase] bbeaudreault merged pull request #5004: HBASE-27570 Unify tracking of block IO across all read request types
bbeaudreault merged PR #5004: URL: https://github.com/apache/hbase/pull/5004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bbeaudreault merged pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
bbeaudreault merged PR #5006: URL: https://github.com/apache/hbase/pull/5006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
Apache-HBase commented on PR #5006: URL: https://github.com/apache/hbase/pull/5006#issuecomment-1416795324 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 44s | master passed | | +1 :green_heart: | compile | 0m 48s | master passed | | +1 :green_heart: | shadedjars | 4m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 42s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed | | +1 :green_heart: | javac | 0m 48s | the patch passed | | +1 :green_heart: | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 40s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 201m 47s | hbase-server in the patch passed. | | | | 226m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5006 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0ab3743a7db1 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e45c59c71b | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/testReport/ | | Max. process+thread count | 2660 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
Apache-HBase commented on PR #5006: URL: https://github.com/apache/hbase/pull/5006#issuecomment-1416794492 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 22s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 19s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 24s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 195m 36s | hbase-server in the patch passed. | | | | 221m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5006 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2188d2ad64e4 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e45c59c71b | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/testReport/ | | Max. process+thread count | 2484 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4966: HBASE-27216 Revisit the ReplicationSyncUp tool
Apache-HBase commented on PR #4966: URL: https://github.com/apache/hbase/pull/4966#issuecomment-1416791904 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 1s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 47s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 3m 22s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | checkstyle | 0m 46s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | spotless | 0m 46s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 3s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 3m 19s | the patch passed | | +1 :green_heart: | cc | 3m 19s | the patch passed | | +1 :green_heart: | javac | 3m 19s | the patch passed | | +1 :green_heart: | checkstyle | 0m 44s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 48s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | hbaseprotoc | 1m 25s | the patch passed | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 50m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4966 | | Optional Tests | dupname asflicense cc hbaseprotoc spotless prototool javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4d34ed989ea4 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / befa513f6f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 80 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-replication hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4966/10/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27216) Revisit the ReplicationSyncUp tool
[ https://issues.apache.org/jira/browse/HBASE-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684170#comment-17684170 ] Duo Zhang commented on HBASE-27216: --- Oh, there is a problem that, in ReplicationSourceManager, when claiming replication queue, we assume that all the wals are under the oldWALs directory. But for replication sync up, SCP may have not been finished yet, so it is possible that the wal files we want to replicate are still under the WALs directory... For me, I prefer we add a new parameter for the claimQueue method in ReplicationSourceManager, when it is called by ReplicationSyncUp, we set it to true, then we will also list WALs directory. Thanks. > Revisit the ReplicationSyncUp tool > -- > > Key: HBASE-27216 > URL: https://issues.apache.org/jira/browse/HBASE-27216 > Project: HBase > Issue Type: Sub-task > Components: Replication, tooling >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27109) Move replication queue storage from zookeeper to a separated HBase table
[
https://issues.apache.org/jira/browse/HBASE-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684168#comment-17684168
]
Hudson commented on HBASE-27109:
Results for branch HBASE-27109/table_based_rqs
[build #49 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/49/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/49/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/49//console].
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/49/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move replication queue storage from zookeeper to a separated HBase table
>
>
> Key: HBASE-27109
> URL: https://issues.apache.org/jira/browse/HBASE-27109
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> This is a more specific issue based on the works which are already done in
> HBASE-15867.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
Apache-HBase commented on PR #5006: URL: https://github.com/apache/hbase/pull/5006#issuecomment-1416751425 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 23s | master passed | | +1 :green_heart: | compile | 2m 58s | master passed | | +1 :green_heart: | checkstyle | 0m 51s | master passed | | +1 :green_heart: | spotless | 0m 40s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 17s | the patch passed | | +1 :green_heart: | compile | 2m 55s | the patch passed | | -0 :warning: | javac | 2m 23s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | +1 :green_heart: | checkstyle | 0m 49s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 30s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 41m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5006 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux f791f7dd87b6 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e45c59c71b | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
Apache-HBase commented on PR #5006: URL: https://github.com/apache/hbase/pull/5006#issuecomment-1416719244 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 54s | master passed | | +1 :green_heart: | compile | 0m 46s | master passed | | +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 0m 50s | the patch passed | | +1 :green_heart: | javac | 0m 50s | the patch passed | | +1 :green_heart: | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 41s | hbase-common in the patch passed. | | -1 :x: | unit | 204m 45s | hbase-server in the patch failed. | | | | 229m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5006 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d4e7416ad7b 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bbc20f3611 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/testReport/ | | Max. process+thread count | 2651 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5006: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
Apache-HBase commented on PR #5006: URL: https://github.com/apache/hbase/pull/5006#issuecomment-1416717998 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 14s | master passed | | +1 :green_heart: | compile | 0m 59s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 12s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 4m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 24s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 195m 58s | hbase-server in the patch passed. | | | | 222m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5006 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 926d3ae56c53 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / bbc20f3611 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/testReport/ | | Max. process+thread count | 2315 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5006/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] thangTang commented on pull request #5003: HBASE-27602 Remove the impact of operating env on testHFileCleaning
thangTang commented on PR #5003: URL: https://github.com/apache/hbase/pull/5003#issuecomment-1416714802 > Could you please explain how this patch could fix the problem described in the jira issue? Sure. You can see the error on my laptop before:`2023-01-31T19:05:14,453 ERROR [hfile_cleaner-dir-scan-pool-0] mob.ManualMobMaintHFileCleaner(87): Failed to determine mob status of 'hdfs://localhost:59285/user/tianhang.tang/test-data/01a0d8a0-6306-7725-b63a-5debcab8cf57/archive/someHFileThatWouldBeAUUID.1675163114186', keeping it just in case. java.lang.IllegalArgumentException: Illegal character <.> at 8. Namespaces may only contain 'alphanumeric characters' from any language and digits: tianhang.tang` That's because the test path is: /user/tianhang.tang/test-data/01a0d8a0-6306-7725-b63a-5debcab8cf57 And the MOCKED path before is just: [archive](https://github.com/apache/hbase/blob/e45c59c71b86163dba85bcf76682f0f2f80f6fc9/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileCleaner.java#L162) The logic in [ManualMobMaintHFileCleaner](https://github.com/apache/hbase/blob/e45c59c71b86163dba85bcf76682f0f2f80f6fc9/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ManualMobMaintHFileCleaner.java#L60) get table/region from the path, so got this error with my root path "user/tianhang.tang". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27600) Make the behavior of hadoop profile consistent
[ https://issues.apache.org/jira/browse/HBASE-27600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27600: -- Component/s: build dependencies hadoop3 pom > Make the behavior of hadoop profile consistent > -- > > Key: HBASE-27600 > URL: https://issues.apache.org/jira/browse/HBASE-27600 > Project: HBase > Issue Type: Bug > Components: build, dependencies, hadoop3, pom >Reporter: tianhang tang >Assignee: tianhang tang >Priority: Major > Fix For: 3.0.0-alpha-4 > > > In HBASE-27599 I found an issue that should cause build failure, but in fact > both the community and my local build jobs succeed, which is not as expected. > The reason is, the activation condition in hbase-shaded-testing-util module > is different with other modules, and it will not activate by default. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27599) Cleanup wrong dependency in dependencyManagement
[ https://issues.apache.org/jira/browse/HBASE-27599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27599. --- Fix Version/s: 3.0.0-alpha-4 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~tangtianhang] for contributing! > Cleanup wrong dependency in dependencyManagement > > > Key: HBASE-27599 > URL: https://issues.apache.org/jira/browse/HBASE-27599 > Project: HBase > Issue Type: Bug >Reporter: tianhang tang >Assignee: tianhang tang >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: screenshot-1.png, screenshot-2.png > > > In HBASE-27340 added a hadoop-3.0 profile in hbase-shaded-testing-util pom, > but missed the version of hadoop-common test jar: > [HBASE-27340|https://github.com/apache/hbase/commit/1b5403cf7d906a22c20bd4f025566a2a5f895d44] > !screenshot-2.png! > This will cause build failure: > !screenshot-1.png! > But I still don't know why this problem does not necessarily arise... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27600) Make the behavior of hadoop profile consistent
[ https://issues.apache.org/jira/browse/HBASE-27600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27600. --- Fix Version/s: 3.0.0-alpha-4 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~tangtianhang] for contributing! > Make the behavior of hadoop profile consistent > -- > > Key: HBASE-27600 > URL: https://issues.apache.org/jira/browse/HBASE-27600 > Project: HBase > Issue Type: Bug >Reporter: tianhang tang >Assignee: tianhang tang >Priority: Major > Fix For: 3.0.0-alpha-4 > > > In HBASE-27599 I found an issue that should cause build failure, but in fact > both the community and my local build jobs succeed, which is not as expected. > The reason is, the activation condition in hbase-shaded-testing-util module > is different with other modules, and it will not activate by default. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27599) Cleanup wrong dependency in dependencyManagement
[ https://issues.apache.org/jira/browse/HBASE-27599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27599: -- Component/s: build dependencies hadoop3 pom > Cleanup wrong dependency in dependencyManagement > > > Key: HBASE-27599 > URL: https://issues.apache.org/jira/browse/HBASE-27599 > Project: HBase > Issue Type: Bug > Components: build, dependencies, hadoop3, pom >Reporter: tianhang tang >Assignee: tianhang tang >Priority: Major > Fix For: 3.0.0-alpha-4 > > Attachments: screenshot-1.png, screenshot-2.png > > > In HBASE-27340 added a hadoop-3.0 profile in hbase-shaded-testing-util pom, > but missed the version of hadoop-common test jar: > [HBASE-27340|https://github.com/apache/hbase/commit/1b5403cf7d906a22c20bd4f025566a2a5f895d44] > !screenshot-2.png! > This will cause build failure: > !screenshot-1.png! > But I still don't know why this problem does not necessarily arise... -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 merged pull request #5000: HBASE-27600 Make the behavior of hadoop profile consistent
Apache9 merged PR #5000: URL: https://github.com/apache/hbase/pull/5000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #4999: HBASE-27599 Cleanup wrong dependency in dependencyManagement
Apache9 merged PR #4999: URL: https://github.com/apache/hbase/pull/4999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
