Re: [PR] HBASE-28420 Update the procedure's field to store for ServerRemoteProcedure [hbase]
Apache-HBase commented on PR #5816: URL: https://github.com/apache/hbase/pull/5816#issuecomment-2119107459 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 1s | master passed | | +1 :green_heart: | compile | 3m 17s | master passed | | +1 :green_heart: | checkstyle | 0m 41s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 23s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 43s | the patch passed | | +1 :green_heart: | compile | 3m 10s | the patch passed | | +1 :green_heart: | cc | 3m 10s | the patch passed | | +1 :green_heart: | javac | 3m 10s | the patch passed | | -0 :warning: | checkstyle | 0m 31s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 5m 6s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | hbaseprotoc | 1m 7s | the patch passed | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 38m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5816/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5816 | | Optional Tests | dupname asflicense cc hbaseprotoc spotless prototool javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a951256f4e14 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6b3f5ae1fc | | Default Java | Eclipse Adoptium-11.0.17+8 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5816/8/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5816/8/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28568) Incremental backup set does not correctly shrink
[
https://issues.apache.org/jira/browse/HBASE-28568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847624#comment-17847624
]
Hudson commented on HBASE-28568:
Results for branch branch-2.6
[build #120 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incremental backup set does not correctly shrink
>
>
> Key: HBASE-28568
> URL: https://issues.apache.org/jira/browse/HBASE-28568
> Project: HBase
> Issue Type: Bug
> Components: backuprestore
>Affects Versions: 2.6.0, 3.0.0
>Reporter: Dieter De Paepe
>Assignee: Dieter De Paepe
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1
>
>
> The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables
> from the incrementalBackupTableSet (= the set of backups to include in every
> incremental backup).
> This can lead to backups failing.
>
> Minimal example to reproduce from source:
> * Add following to `conf/hbase-site.xml` to enable backups:
> {code:java}
>
> hbase.backup.enable
> true
>
>
> hbase.master.logcleaner.plugins
>
> org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
>
>
> hbase.procedure.master.classes
>
> org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>
>
> hbase.procedure.regionserver.classes
>
> org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
>
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.backup.BackupObserver
>
>
> hbase.fs.tmp.dir
> file:/tmp/hbase-tmp
>{code}
> * Start HBase: {{bin/start-hbase.sh}}
> *
> {code:java}
> echo "create 'table1', 'cf'" | bin/hbase shell -n
> echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full
> file:/tmp/hbasebackups -t table1
> bin/hbase backup create full file:/tmp/hbasebackups -t table2
> bin/hbase backup create incremental file:/tmp/hbasebackups
> # Deletes the 2 most recent backups
> bin/hbase backup delete -l $(bin/hbase backup history | head -n1 | tail -n
> -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2 | tail
> -n -1 | grep -o -P "backup_\d+")
> bin/hbase backup create incremental file:/tmp/hbasebackups -t table1
> bin/hbase backup history{code}
> * Output shows the incremental backup still includes table2, this should
> only be table1:
> {code:java}
> {ID=backup_171553763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start
> time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST
> 2024,Progress=100%}
> {ID=backup_171531407,Type=FULL,Tables={table1},State=COMPLETE,Start
> time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST
> 2024,Progress=100%}
> {code}
> PR will follow soon.
> (Edited: my original ticket included a stacktrace of an IllegalStateException
> from a PR for HBASE-28562)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[
https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847623#comment-17847623
]
Hudson commented on HBASE-28595:
Results for branch branch-2.6
[build #120 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Losing exception from scan RPC can lead to partial results
> --
>
> Key: HBASE-28595
> URL: https://issues.apache.org/jira/browse/HBASE-28595
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> This was discovered in Apache Impala using HBase 2.2 based branch hbase
> client and server. It is not clear yet whether other branches are also
> affected.
> The issue happens if the server side of the scan throws an exception and
> closes the scanner, but at the same time, the client gets an rpc connection
> closed error and doesn't process the exception sent by the server. Client
> then thinks it got a network error, which leads to retrying the RPC instead
> of opening a new scanner. But then when the client retry reaches the server,
> the server returns an empty ScanResponse instead of an error, leading to
> closing the scanner on client side without returning any error.
> A few pointers to critical parts:
> region server:
> 1st call throws exception leading to closing (but not deleting) scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539]
> 2nd call (retry of 1st) returns empty results:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403]
> client:
> some exceptions are handled as non-retriable at RPC level and are only
> handled through opening a new scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214]
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367]
> This mechanism in the client only works if it gets the exception from the
> server. If there are connection issues during the RPC then the client won't
> really know the state of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-25972) Dual File Compaction
[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847621#comment-17847621
]
Hudson commented on HBASE-25972:
Results for branch branch-2.6
[build #120 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dual File Compaction
>
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
>Reporter: Kadir Ozdemir
>Assignee: Kadir Ozdemir
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28604) Fix the error message in ReservoirSample's constructor
[
https://issues.apache.org/jira/browse/HBASE-28604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847622#comment-17847622
]
Hudson commented on HBASE-28604:
Results for branch branch-2.6
[build #120 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/120/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the error message in ReservoirSample's constructor
> --
>
> Key: HBASE-28604
> URL: https://issues.apache.org/jira/browse/HBASE-28604
> Project: HBase
> Issue Type: Bug
> Components: util
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[
https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847620#comment-17847620
]
Hudson commented on HBASE-28595:
Results for branch branch-2.4
[build #739 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/739/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/739/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/739/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/739/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/739/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Losing exception from scan RPC can lead to partial results
> --
>
> Key: HBASE-28595
> URL: https://issues.apache.org/jira/browse/HBASE-28595
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> This was discovered in Apache Impala using HBase 2.2 based branch hbase
> client and server. It is not clear yet whether other branches are also
> affected.
> The issue happens if the server side of the scan throws an exception and
> closes the scanner, but at the same time, the client gets an rpc connection
> closed error and doesn't process the exception sent by the server. Client
> then thinks it got a network error, which leads to retrying the RPC instead
> of opening a new scanner. But then when the client retry reaches the server,
> the server returns an empty ScanResponse instead of an error, leading to
> closing the scanner on client side without returning any error.
> A few pointers to critical parts:
> region server:
> 1st call throws exception leading to closing (but not deleting) scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539]
> 2nd call (retry of 1st) returns empty results:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403]
> client:
> some exceptions are handled as non-retriable at RPC level and are only
> handled through opening a new scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214]
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367]
> This mechanism in the client only works if it gets the exception from the
> server. If there are connection issues during the RPC then the client won't
> really know the state of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28599) RowTooBigException is thrown when duplicate increment RPC call is attempted
[ https://issues.apache.org/jira/browse/HBASE-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] youngju kim reassigned HBASE-28599: --- Assignee: youngju kim > RowTooBigException is thrown when duplicate increment RPC call is attempted > --- > > Key: HBASE-28599 > URL: https://issues.apache.org/jira/browse/HBASE-28599 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.5.5, 2.5.6, 2.5.7, 2.5.8 >Reporter: Robin Infant A >Assignee: youngju kim >Priority: Major > Attachments: RowTooBig_trace.txt > > > *Issue:* > `RowTooBigException` is thrown when a duplicate increment RPC call is > attempted. > *Expected Behavior:* > 1. The initial RPC increment call should time out for some reason. > 2. The duplicate RPC call should be converted to a GET request and fetch the > result that I am trying to increment. > 3. The result should contain only the qualifier that I am attempting to > increment. > *Actual Behavior:* > 1. The initial RPC increment call timed out, which is expected. > 2. The duplicate RPC call is converted to a GET request but fails to clone > the qualifier into the GET request. > 3. Hence, the GET request attempts to retrieve all qualifiers for the given > row and columnfamily, resulting in a `RowTooBigException`. > *Steps to Reproduce:* > 1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` > (default = 1073741824) exists. > 2. Nonce property should be enabled `hbase.client.nonces.enabled` which is > actually defaulted to true. > 3. Attempt to increment a qualifier against the same row. > 4. In my case, I am using a postIncrement co-processor which may cause a > delay (longer than the RPC timeout property). > 5. A duplicate increment call should be triggered, which tries to get the > value rather than increment it. > 6. The GET request actually tries to retrieve all the qualifiers for the row, > resulting in a `RowTooBigException`. > *Insights:* > Upon further debugging, I found that qualifiers are not cloned into the GET > instance due to incorrect usage of > [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833] > *Fix Suggestion:* > Removing the `!` operation from `while (!CellScanner.advance)` may resolve > the issue. > Attached Exception Stack Trace for reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28604) Fix the error message in ReservoirSample's constructor
[
https://issues.apache.org/jira/browse/HBASE-28604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847614#comment-17847614
]
Hudson commented on HBASE-28604:
Results for branch branch-2.5
[build #530 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the error message in ReservoirSample's constructor
> --
>
> Key: HBASE-28604
> URL: https://issues.apache.org/jira/browse/HBASE-28604
> Project: HBase
> Issue Type: Bug
> Components: util
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847612#comment-17847612
]
Hudson commented on HBASE-26048:
Results for branch branch-2.5
[build #530 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-25972) Dual File Compaction
[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847613#comment-17847613
]
Hudson commented on HBASE-25972:
Results for branch branch-2.5
[build #530 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dual File Compaction
>
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
>Reporter: Kadir Ozdemir
>Assignee: Kadir Ozdemir
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28598) NPE for writer object access in AsyncFSWAL#closeWriter
[
https://issues.apache.org/jira/browse/HBASE-28598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847611#comment-17847611
]
Hudson commented on HBASE-28598:
Results for branch branch-2.5
[build #530 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/530//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> NPE for writer object access in AsyncFSWAL#closeWriter
> --
>
> Key: HBASE-28598
> URL: https://issues.apache.org/jira/browse/HBASE-28598
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.6.0, 2.4.18, 2.5.9
>Reporter: Vineet Kumar Maheshwari
>Assignee: Vineet Kumar Maheshwari
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 2.6.1, 2.5.9
>
>
> Observed NPE during execution of some of the UT cases.
> Exception is happening in AbstractFSWAL#closeWriter when trying to put null
> writer object in inflightWALClosures map.
> Need to add null check for writer object in AsyncFSWAL#doShutdown function
> before its usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-25972) Dual File Compaction
[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847599#comment-17847599
]
Hudson commented on HBASE-25972:
Results for branch branch-3
[build #207 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dual File Compaction
>
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
>Reporter: Kadir Ozdemir
>Assignee: Kadir Ozdemir
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847598#comment-17847598
]
Hudson commented on HBASE-26048:
Results for branch branch-3
[build #207 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
[
https://issues.apache.org/jira/browse/HBASE-28579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847597#comment-17847597
]
Hudson commented on HBASE-28579:
Results for branch branch-3
[build #207 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Hide HFileScanner related methods in StoreFileReader
>
>
> Key: HBASE-28579
> URL: https://issues.apache.org/jira/browse/HBASE-28579
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, Scanners
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28578) Remove deprecated methods in HFileScanner
[
https://issues.apache.org/jira/browse/HBASE-28578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847596#comment-17847596
]
Hudson commented on HBASE-28578:
Results for branch branch-3
[build #207 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove deprecated methods in HFileScanner
> -
>
> Key: HBASE-28578
> URL: https://issues.apache.org/jira/browse/HBASE-28578
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, Scanners
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28572) Remove deprecated methods in thrift module
[
https://issues.apache.org/jira/browse/HBASE-28572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847595#comment-17847595
]
Hudson commented on HBASE-28572:
Results for branch branch-3
[build #207 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/207//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove deprecated methods in thrift module
> --
>
> Key: HBASE-28572
> URL: https://issues.apache.org/jira/browse/HBASE-28572
> Project: HBase
> Issue Type: Sub-task
> Components: Thrift
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[
https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847582#comment-17847582
]
Hudson commented on HBASE-28595:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Losing exception from scan RPC can lead to partial results
> --
>
> Key: HBASE-28595
> URL: https://issues.apache.org/jira/browse/HBASE-28595
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Critical
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> This was discovered in Apache Impala using HBase 2.2 based branch hbase
> client and server. It is not clear yet whether other branches are also
> affected.
> The issue happens if the server side of the scan throws an exception and
> closes the scanner, but at the same time, the client gets an rpc connection
> closed error and doesn't process the exception sent by the server. Client
> then thinks it got a network error, which leads to retrying the RPC instead
> of opening a new scanner. But then when the client retry reaches the server,
> the server returns an empty ScanResponse instead of an error, leading to
> closing the scanner on client side without returning any error.
> A few pointers to critical parts:
> region server:
> 1st call throws exception leading to closing (but not deleting) scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539]
> 2nd call (retry of 1st) returns empty results:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403]
> client:
> some exceptions are handled as non-retriable at RPC level and are only
> handled through opening a new scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214]
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367]
> This mechanism in the client only works if it gets the exception from the
> server. If there are connection issues during the RPC then the client won't
> really know the state of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847580#comment-17847580
]
Hudson commented on HBASE-26048:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28568) Incremental backup set does not correctly shrink
[
https://issues.apache.org/jira/browse/HBASE-28568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847576#comment-17847576
]
Hudson commented on HBASE-28568:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incremental backup set does not correctly shrink
>
>
> Key: HBASE-28568
> URL: https://issues.apache.org/jira/browse/HBASE-28568
> Project: HBase
> Issue Type: Bug
> Components: backuprestore
>Affects Versions: 2.6.0, 3.0.0
>Reporter: Dieter De Paepe
>Assignee: Dieter De Paepe
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1
>
>
> The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables
> from the incrementalBackupTableSet (= the set of backups to include in every
> incremental backup).
> This can lead to backups failing.
>
> Minimal example to reproduce from source:
> * Add following to `conf/hbase-site.xml` to enable backups:
> {code:java}
>
> hbase.backup.enable
> true
>
>
> hbase.master.logcleaner.plugins
>
> org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
>
>
> hbase.procedure.master.classes
>
> org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>
>
> hbase.procedure.regionserver.classes
>
> org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
>
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.backup.BackupObserver
>
>
> hbase.fs.tmp.dir
> file:/tmp/hbase-tmp
>{code}
> * Start HBase: {{bin/start-hbase.sh}}
> *
> {code:java}
> echo "create 'table1', 'cf'" | bin/hbase shell -n
> echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full
> file:/tmp/hbasebackups -t table1
> bin/hbase backup create full file:/tmp/hbasebackups -t table2
> bin/hbase backup create incremental file:/tmp/hbasebackups
> # Deletes the 2 most recent backups
> bin/hbase backup delete -l $(bin/hbase backup history | head -n1 | tail -n
> -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2 | tail
> -n -1 | grep -o -P "backup_\d+")
> bin/hbase backup create incremental file:/tmp/hbasebackups -t table1
> bin/hbase backup history{code}
> * Output shows the incremental backup still includes table2, this should
> only be table1:
> {code:java}
> {ID=backup_171553763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start
> time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST
> 2024,Progress=100%}
> {ID=backup_171531407,Type=FULL,Tables={table1},State=COMPLETE,Start
> time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST
> 2024,Progress=100%}
> {code}
> PR will follow soon.
> (Edited: my original ticket included a stacktrace of an IllegalStateException
> from a PR for HBASE-28562)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28236) Add 2.6.0 to downloads page
[
https://issues.apache.org/jira/browse/HBASE-28236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847581#comment-17847581
]
Hudson commented on HBASE-28236:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add 2.6.0 to downloads page
> ---
>
> Key: HBASE-28236
> URL: https://issues.apache.org/jira/browse/HBASE-28236
> Project: HBase
> Issue Type: Sub-task
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28572) Remove deprecated methods in thrift module
[
https://issues.apache.org/jira/browse/HBASE-28572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847577#comment-17847577
]
Hudson commented on HBASE-28572:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove deprecated methods in thrift module
> --
>
> Key: HBASE-28572
> URL: https://issues.apache.org/jira/browse/HBASE-28572
> Project: HBase
> Issue Type: Sub-task
> Components: Thrift
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
[
https://issues.apache.org/jira/browse/HBASE-28579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847579#comment-17847579
]
Hudson commented on HBASE-28579:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Hide HFileScanner related methods in StoreFileReader
>
>
> Key: HBASE-28579
> URL: https://issues.apache.org/jira/browse/HBASE-28579
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, Scanners
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28578) Remove deprecated methods in HFileScanner
[
https://issues.apache.org/jira/browse/HBASE-28578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847578#comment-17847578
]
Hudson commented on HBASE-28578:
Results for branch master
[build #1074 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074//console].
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1074/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove deprecated methods in HFileScanner
> -
>
> Key: HBASE-28578
> URL: https://issues.apache.org/jira/browse/HBASE-28578
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, Scanners
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28568) Incremental backup set does not correctly shrink
[
https://issues.apache.org/jira/browse/HBASE-28568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-28568:
-
Fix Version/s: 2.7.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Incremental backup set does not correctly shrink
>
>
> Key: HBASE-28568
> URL: https://issues.apache.org/jira/browse/HBASE-28568
> Project: HBase
> Issue Type: Bug
> Components: backuprestore
>Affects Versions: 2.6.0, 3.0.0
>Reporter: Dieter De Paepe
>Assignee: Dieter De Paepe
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1
>
>
> The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables
> from the incrementalBackupTableSet (= the set of backups to include in every
> incremental backup).
> This can lead to backups failing.
>
> Minimal example to reproduce from source:
> * Add following to `conf/hbase-site.xml` to enable backups:
> {code:java}
>
> hbase.backup.enable
> true
>
>
> hbase.master.logcleaner.plugins
>
> org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
>
>
> hbase.procedure.master.classes
>
> org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>
>
> hbase.procedure.regionserver.classes
>
> org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
>
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.backup.BackupObserver
>
>
> hbase.fs.tmp.dir
> file:/tmp/hbase-tmp
>{code}
> * Start HBase: {{bin/start-hbase.sh}}
> *
> {code:java}
> echo "create 'table1', 'cf'" | bin/hbase shell -n
> echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full
> file:/tmp/hbasebackups -t table1
> bin/hbase backup create full file:/tmp/hbasebackups -t table2
> bin/hbase backup create incremental file:/tmp/hbasebackups
> # Deletes the 2 most recent backups
> bin/hbase backup delete -l $(bin/hbase backup history | head -n1 | tail -n
> -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2 | tail
> -n -1 | grep -o -P "backup_\d+")
> bin/hbase backup create incremental file:/tmp/hbasebackups -t table1
> bin/hbase backup history{code}
> * Output shows the incremental backup still includes table2, this should
> only be table1:
> {code:java}
> {ID=backup_171553763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start
> time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST
> 2024,Progress=100%}
> {ID=backup_171531407,Type=FULL,Tables={table1},State=COMPLETE,Start
> time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST
> 2024,Progress=100%}
> {code}
> PR will follow soon.
> (Edited: my original ticket included a stacktrace of an IllegalStateException
> from a PR for HBASE-28562)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28568) Incremental backup set does not correctly shrink
[
https://issues.apache.org/jira/browse/HBASE-28568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847573#comment-17847573
]
Nick Dimiduk commented on HBASE-28568:
--
Applied to branch-2.6+. Thanks [~dieterdp_ng]!
> Incremental backup set does not correctly shrink
>
>
> Key: HBASE-28568
> URL: https://issues.apache.org/jira/browse/HBASE-28568
> Project: HBase
> Issue Type: Bug
> Components: backuprestore
>Affects Versions: 2.6.0, 3.0.0
>Reporter: Dieter De Paepe
>Assignee: Dieter De Paepe
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 3.0.0-beta-2, 2.6.1
>
>
> The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables
> from the incrementalBackupTableSet (= the set of backups to include in every
> incremental backup).
> This can lead to backups failing.
>
> Minimal example to reproduce from source:
> * Add following to `conf/hbase-site.xml` to enable backups:
> {code:java}
>
> hbase.backup.enable
> true
>
>
> hbase.master.logcleaner.plugins
>
> org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
>
>
> hbase.procedure.master.classes
>
> org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>
>
> hbase.procedure.regionserver.classes
>
> org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
>
>
> hbase.coprocessor.region.classes
> org.apache.hadoop.hbase.backup.BackupObserver
>
>
> hbase.fs.tmp.dir
> file:/tmp/hbase-tmp
>{code}
> * Start HBase: {{bin/start-hbase.sh}}
> *
> {code:java}
> echo "create 'table1', 'cf'" | bin/hbase shell -n
> echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full
> file:/tmp/hbasebackups -t table1
> bin/hbase backup create full file:/tmp/hbasebackups -t table2
> bin/hbase backup create incremental file:/tmp/hbasebackups
> # Deletes the 2 most recent backups
> bin/hbase backup delete -l $(bin/hbase backup history | head -n1 | tail -n
> -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2 | tail
> -n -1 | grep -o -P "backup_\d+")
> bin/hbase backup create incremental file:/tmp/hbasebackups -t table1
> bin/hbase backup history{code}
> * Output shows the incremental backup still includes table2, this should
> only be table1:
> {code:java}
> {ID=backup_171553763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start
> time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST
> 2024,Progress=100%}
> {ID=backup_171531407,Type=FULL,Tables={table1},State=COMPLETE,Start
> time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST
> 2024,Progress=100%}
> {code}
> PR will follow soon.
> (Edited: my original ticket included a stacktrace of an IllegalStateException
> from a PR for HBASE-28562)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] Backport "HBASE-28568 Incremental backup set does not correctly shrink (#5876)" to branch-2.6 [hbase]
ndimiduk merged PR #5916: URL: https://github.com/apache/hbase/pull/5916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Backport "HBASE-28568 Incremental backup set does not correctly shrink (#5876)" to branch-2 [hbase]
ndimiduk merged PR #5915: URL: https://github.com/apache/hbase/pull/5915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Backport "HBASE-28568 Incremental backup set does not correctly shrink (#5876)" to branch-3 [hbase]
ndimiduk merged PR #5914: URL: https://github.com/apache/hbase/pull/5914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-28599) RowTooBigException is thrown when duplicate increment RPC call is attempted
[ https://issues.apache.org/jira/browse/HBASE-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847554#comment-17847554 ] youngju kim edited comment on HBASE-28599 at 5/18/24 3:23 PM: -- Hi,[~robiee17]. This is youngjukim and I'm operating HBase clusters in my company from Korea. Thank you for explaining the detailed problem definition, reproduction process with fix suggestions. I'd like to do HBase contributing. Is it okay if I try to solve this issue? was (Author: JIRAUSER300939): Hi,[~robiee17]. This is youngjukim and I'm operating HBase clusters in my company from Korea. Thank you for explaining the detailed problem definition, reproduction process with some hint. I'd like to do HBase contributing. Is it okay if I try to solve this issue? > RowTooBigException is thrown when duplicate increment RPC call is attempted > --- > > Key: HBASE-28599 > URL: https://issues.apache.org/jira/browse/HBASE-28599 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.5.5, 2.5.6, 2.5.7, 2.5.8 >Reporter: Robin Infant A >Priority: Major > Attachments: RowTooBig_trace.txt > > > *Issue:* > `RowTooBigException` is thrown when a duplicate increment RPC call is > attempted. > *Expected Behavior:* > 1. The initial RPC increment call should time out for some reason. > 2. The duplicate RPC call should be converted to a GET request and fetch the > result that I am trying to increment. > 3. The result should contain only the qualifier that I am attempting to > increment. > *Actual Behavior:* > 1. The initial RPC increment call timed out, which is expected. > 2. The duplicate RPC call is converted to a GET request but fails to clone > the qualifier into the GET request. > 3. Hence, the GET request attempts to retrieve all qualifiers for the given > row and columnfamily, resulting in a `RowTooBigException`. > *Steps to Reproduce:* > 1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` > (default = 1073741824) exists. > 2. Nonce property should be enabled `hbase.client.nonces.enabled` which is > actually defaulted to true. > 3. Attempt to increment a qualifier against the same row. > 4. In my case, I am using a postIncrement co-processor which may cause a > delay (longer than the RPC timeout property). > 5. A duplicate increment call should be triggered, which tries to get the > value rather than increment it. > 6. The GET request actually tries to retrieve all the qualifiers for the row, > resulting in a `RowTooBigException`. > *Insights:* > Upon further debugging, I found that qualifiers are not cloned into the GET > instance due to incorrect usage of > [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833] > *Fix Suggestion:* > Removing the `!` operation from `while (!CellScanner.advance)` may resolve > the issue. > Attached Exception Stack Trace for reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28599) RowTooBigException is thrown when duplicate increment RPC call is attempted
[ https://issues.apache.org/jira/browse/HBASE-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847554#comment-17847554 ] youngju kim edited comment on HBASE-28599 at 5/18/24 3:20 PM: -- Hi,[~robiee17]. This is youngjukim and I'm operating HBase clusters in my company from Korea. Thank you for explaining the detailed problem definition, reproduction process with some hint. I'd like to do HBase contributing. Is it okay if I try to solve this issue? was (Author: JIRAUSER300939): Hi,[~robiee17] . I'm operating HBase clusters in my company from Korea. Thank you for explaining the detailed problem definition, reproduction process with some hint. I'd like to do HBase contributing. Is it okay if I try to solve this issue? > RowTooBigException is thrown when duplicate increment RPC call is attempted > --- > > Key: HBASE-28599 > URL: https://issues.apache.org/jira/browse/HBASE-28599 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.5.5, 2.5.6, 2.5.7, 2.5.8 >Reporter: Robin Infant A >Priority: Major > Attachments: RowTooBig_trace.txt > > > *Issue:* > `RowTooBigException` is thrown when a duplicate increment RPC call is > attempted. > *Expected Behavior:* > 1. The initial RPC increment call should time out for some reason. > 2. The duplicate RPC call should be converted to a GET request and fetch the > result that I am trying to increment. > 3. The result should contain only the qualifier that I am attempting to > increment. > *Actual Behavior:* > 1. The initial RPC increment call timed out, which is expected. > 2. The duplicate RPC call is converted to a GET request but fails to clone > the qualifier into the GET request. > 3. Hence, the GET request attempts to retrieve all qualifiers for the given > row and columnfamily, resulting in a `RowTooBigException`. > *Steps to Reproduce:* > 1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` > (default = 1073741824) exists. > 2. Nonce property should be enabled `hbase.client.nonces.enabled` which is > actually defaulted to true. > 3. Attempt to increment a qualifier against the same row. > 4. In my case, I am using a postIncrement co-processor which may cause a > delay (longer than the RPC timeout property). > 5. A duplicate increment call should be triggered, which tries to get the > value rather than increment it. > 6. The GET request actually tries to retrieve all the qualifiers for the row, > resulting in a `RowTooBigException`. > *Insights:* > Upon further debugging, I found that qualifiers are not cloned into the GET > instance due to incorrect usage of > [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833] > *Fix Suggestion:* > Removing the `!` operation from `while (!CellScanner.advance)` may resolve > the issue. > Attached Exception Stack Trace for reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28599) RowTooBigException is thrown when duplicate increment RPC call is attempted
[ https://issues.apache.org/jira/browse/HBASE-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847554#comment-17847554 ] youngju kim commented on HBASE-28599: - Hi,[~robiee17] . I'm operating HBase clusters in my company from Korea. Thank you for explaining the detailed problem definition, reproduction process with some hint. I'd like to do HBase contributing. Is it okay if I try to solve this issue? > RowTooBigException is thrown when duplicate increment RPC call is attempted > --- > > Key: HBASE-28599 > URL: https://issues.apache.org/jira/browse/HBASE-28599 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.5.5, 2.5.6, 2.5.7, 2.5.8 >Reporter: Robin Infant A >Priority: Major > Attachments: RowTooBig_trace.txt > > > *Issue:* > `RowTooBigException` is thrown when a duplicate increment RPC call is > attempted. > *Expected Behavior:* > 1. The initial RPC increment call should time out for some reason. > 2. The duplicate RPC call should be converted to a GET request and fetch the > result that I am trying to increment. > 3. The result should contain only the qualifier that I am attempting to > increment. > *Actual Behavior:* > 1. The initial RPC increment call timed out, which is expected. > 2. The duplicate RPC call is converted to a GET request but fails to clone > the qualifier into the GET request. > 3. Hence, the GET request attempts to retrieve all qualifiers for the given > row and columnfamily, resulting in a `RowTooBigException`. > *Steps to Reproduce:* > 1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` > (default = 1073741824) exists. > 2. Nonce property should be enabled `hbase.client.nonces.enabled` which is > actually defaulted to true. > 3. Attempt to increment a qualifier against the same row. > 4. In my case, I am using a postIncrement co-processor which may cause a > delay (longer than the RPC timeout property). > 5. A duplicate increment call should be triggered, which tries to get the > value rather than increment it. > 6. The GET request actually tries to retrieve all the qualifiers for the row, > resulting in a `RowTooBigException`. > *Insights:* > Upon further debugging, I found that qualifiers are not cloned into the GET > instance due to incorrect usage of > [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833] > *Fix Suggestion:* > Removing the `!` operation from `while (!CellScanner.advance)` may resolve > the issue. > Attached Exception Stack Trace for reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28174) DELETE endpoint in REST API does not support deleting binary row keys/columns
[
https://issues.apache.org/jira/browse/HBASE-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-28174:
--
Fix Version/s: (was: 4.0.0-alpha-1)
> DELETE endpoint in REST API does not support deleting binary row keys/columns
> -
>
> Key: HBASE-28174
> URL: https://issues.apache.org/jira/browse/HBASE-28174
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1
>Reporter: James Udiljak
>Assignee: James Udiljak
>Priority: Blocker
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.9
>
> Attachments: delete_base64_1.png
>
>
> h2. Notes
> This is the first time I have raised an issue in the ASF Jira. Please let me
> know if there's anything I need to adjust on the issue to fit in with your
> development flow.
> I have marked the priority as "blocker" because this issue blocks me as a
> user of the HBase REST API from deploying an effective solution for our
> setup. Please feel free to change this if the Priority field has another
> meaning to you.
> I have also chosen 2.4.17 as the affected version because this is the version
> I am running, however looking at the source code on GitHub in the default
> branch, I think many other versions would be affected.
> h2. Description of Issue
> The DELETE operation in the [HBase REST
> API|https://hbase.apache.org/1.2/apidocs/org/apache/hadoop/hbase/rest/package-summary.html#operation_delete]
> requires specifying row keys and column families/offsets in the URI (i.e. as
> UTF-8 text). This makes it impossible to specify a delete operation via the
> REST API for a binary row key or column family/offset, as single bytes with a
> decimal value greater than 127 are not valid in UTF-8.
> Percent-encoding these "high" values does not work around the issue, as the
> HBase REST API uses Java's {{URLDecoder.Decode(percentEncodedString,
> "UTF-8")}} function, which replaces any percent-encoded byte in the range
> {{%80}} to {{%FF}} with the [replacement
> character|https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character].
> Even if this were not the case, the row-key is ultimately [converted to a
> byte
> array|https://github.com/apache/hbase/blob/rel/2.4.17/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowSpec.java#L60-L100]
> using UTF-8 encoding, wherein code points >127 are encoded across multiple
> bytes, corrupting the user-supplied row key.
> h2. Proposed Solution
> I do not believe it is possible to allow encoding of arbitrary bytes in the
> URL for the DELETE endpoint without breaking compatibility for any users who
> may have been unknowingly UTF-8 encoding their binary row keys. Even if it
> were possible, the syntax would likely be terse.
> Instead, I propose a new version of the DELETE endpoint that would accept row
> keys and column families/offsets in the request _body_ (using Base64 encoding
> for the JSON and XML formats, and bare binary for protobuf). This new
> endpoint would follow the same conventions as the PUT operations, except that
> cell values would not need to be specified (unless the user is performing a
> check-and-delete operation).
> As an additional benefit, using the request body could potentially allow for
> deleting multiple rows in a single request, which would drastically improve
> the efficiency of my use case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28524) Backport HBASE-28174 to branch-2.4 and branch-2.5
[ https://issues.apache.org/jira/browse/HBASE-28524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28524: -- Fix Version/s: (was: 2.4.18) (was: 2.5.9) > Backport HBASE-28174 to branch-2.4 and branch-2.5 > - > > Key: HBASE-28524 > URL: https://issues.apache.org/jira/browse/HBASE-28524 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.4.17, 2.5.8 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > The changes are backwards compatible and the REST interface is super limited > without them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28417) TestBlockingIPC.testBadPreambleHeader sometimes fails with broken pipe instead of bad auth
[ https://issues.apache.org/jira/browse/HBASE-28417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28417: -- Fix Version/s: (was: 2.4.18) > TestBlockingIPC.testBadPreambleHeader sometimes fails with broken pipe > instead of bad auth > -- > > Key: HBASE-28417 > URL: https://issues.apache.org/jira/browse/HBASE-28417 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 3.0.0-beta-2, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27923) NettyRpcServer may hange if it should skip initial sasl handshake
[
https://issues.apache.org/jira/browse/HBASE-27923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27923:
--
Fix Version/s: 2.5.6
(was: 2.5.9)
> NettyRpcServer may hange if it should skip initial sasl handshake
> -
>
> Key: HBASE-27923
> URL: https://issues.apache.org/jira/browse/HBASE-27923
> Project: HBase
> Issue Type: Bug
> Components: netty, rpc, security
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> {{NettyRpcServer}} may hange if it should skip initial sasl handshake when
> server does not enable security and client enables security, I think this
> problem is caused by two reasons：
> * For Server:
> The type of the response is {{RpcResponse}}, but for
> {{NettyRpcServerPreambleHandler}},when it send {{RpcResponse}}
> ,{{NettyRpcServerResponseEncoder}} does not exist, so {{RpcResponse}}
> messages cannot be sent.
> * For Client
> When {{NettyHBaseSaslRpcClientHandler}} receives
> {{SaslUtil.SWITCH_TO_SIMPLE_AUTH}}, it does not remove
> {{SaslChallengeDecoder}} and {{NettyHBaseSaslRpcClientHandler}}, so the
> latter responses are considered to be incorrect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28255) Correcting spelling errors or annotations with non-standard spelling
[ https://issues.apache.org/jira/browse/HBASE-28255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28255: -- Fix Version/s: 2.4.18 > Correcting spelling errors or annotations with non-standard spelling > > > Key: HBASE-28255 > URL: https://issues.apache.org/jira/browse/HBASE-28255 > Project: HBase > Issue Type: Improvement >Reporter: mazhengxuan >Priority: Minor > Labels: documentation > Fix For: 2.6.0, 2.4.18, 2.5.8, 3.0.0-beta-2 > > > Modify some spelling errors or non-standard spelling comments pointed out by > Typo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27923) NettyRpcServer may hange if it should skip initial sasl handshake
[
https://issues.apache.org/jira/browse/HBASE-27923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27923:
--
Fix Version/s: 2.4.18
2.5.9
> NettyRpcServer may hange if it should skip initial sasl handshake
> -
>
> Key: HBASE-27923
> URL: https://issues.apache.org/jira/browse/HBASE-27923
> Project: HBase
> Issue Type: Bug
> Components: netty, rpc, security
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.9
>
>
> {{NettyRpcServer}} may hange if it should skip initial sasl handshake when
> server does not enable security and client enables security, I think this
> problem is caused by two reasons：
> * For Server:
> The type of the response is {{RpcResponse}}, but for
> {{NettyRpcServerPreambleHandler}},when it send {{RpcResponse}}
> ,{{NettyRpcServerResponseEncoder}} does not exist, so {{RpcResponse}}
> messages cannot be sent.
> * For Client
> When {{NettyHBaseSaslRpcClientHandler}} receives
> {{SaslUtil.SWITCH_TO_SIMPLE_AUTH}}, it does not remove
> {{SaslChallengeDecoder}} and {{NettyHBaseSaslRpcClientHandler}}, so the
> latter responses are considered to be incorrect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27758) Inconsistent synchronization in MetricsUserSourceImpl
[ https://issues.apache.org/jira/browse/HBASE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27758: -- Fix Version/s: 2.4.18 (was: 2.4.17) > Inconsistent synchronization in MetricsUserSourceImpl > - > > Key: HBASE-27758 > URL: https://issues.apache.org/jira/browse/HBASE-27758 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18 > > > Picked up by spotbugs: > > ||Code||Warning|| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 253] > Synchronized access at MetricsUserSourceImpl.java:[line 146]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto; locked > 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 241] > Synchronized access at MetricsUserSourceImpl.java:[line 141]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 247] > Synchronized access at MetricsUserSourceImpl.java:[line 145]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 264] > Synchronized access at MetricsUserSourceImpl.java:[line 142]| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27758) Inconsistent synchronization in MetricsUserSourceImpl
[ https://issues.apache.org/jira/browse/HBASE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27758: -- Component/s: metrics > Inconsistent synchronization in MetricsUserSourceImpl > - > > Key: HBASE-27758 > URL: https://issues.apache.org/jira/browse/HBASE-27758 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18 > > > Picked up by spotbugs: > > ||Code||Warning|| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 253] > Synchronized access at MetricsUserSourceImpl.java:[line 146]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto; locked > 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 241] > Synchronized access at MetricsUserSourceImpl.java:[line 141]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 247] > Synchronized access at MetricsUserSourceImpl.java:[line 145]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 264] > Synchronized access at MetricsUserSourceImpl.java:[line 142]| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27704:
--
Component/s: Quotas
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18
>
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27704:
--
Fix Version/s: 2.4.18
(was: 2.4.17)
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18
>
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28595: client side fix for partial results when exception from scan RPC is lost [hbase]
Apache9 closed pull request #5904: HBASE-28595: client side fix for partial results when exception from scan RPC is lost URL: https://github.com/apache/hbase/pull/5904 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28595: -- Component/s: (was: Client) > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28595. --- Fix Version/s: 2.4.18 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all active branches. Thanks [~csringhofer] for contributing and all others for helping and reviewing! > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28595: check seq id of scan RPCs for closed scanners (#5910) [hbase]
Apache9 merged PR #5925: URL: https://github.com/apache/hbase/pull/5925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28595: check seq id of scan RPCs for closed scanners (#5910) [hbase]
Apache9 merged PR #5924: URL: https://github.com/apache/hbase/pull/5924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28595: check seq id of scan RPCs for closed scanners (#5910) [hbase]
Apache9 merged PR #5923: URL: https://github.com/apache/hbase/pull/5923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28595: check seq id of scan RPCs for closed scanners (#5910) [hbase]
Apache9 merged PR #5922: URL: https://github.com/apache/hbase/pull/5922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-28294) Support to skip Kerberos authentication for metric endpoints
[ https://issues.apache.org/jira/browse/HBASE-28294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] youngju kim reassigned HBASE-28294: --- Assignee: (was: youngju kim) > Support to skip Kerberos authentication for metric endpoints > > > Key: HBASE-28294 > URL: https://issues.apache.org/jira/browse/HBASE-28294 > Project: HBase > Issue Type: New Feature > Components: metrics, UI >Affects Versions: 2.5.5 >Reporter: YUBI LEE >Priority: Major > > Make HBase support to skip Kerberos authentication for metric endpoints. > (e.g. /jvm, /prometheus, /metrics) > Since HBase uses KerberoAuthenticationHandler.java, whitelist configuration > can be used like > [HADOOP-16527|https://issues.apache.org/jira/browse/HADOOP-16527]. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28592) Backport HBASE-26525 Use unique thread name for group WALs
[ https://issues.apache.org/jira/browse/HBASE-28592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847503#comment-17847503 ] Duo Zhang commented on HBASE-28592: --- [~szucsvillo] If you do not mind, I will close this issue and reopen the parent issue to cherry-pick the commits to other branches, and reset the fix versions of the parent issue. Thanks. > Backport HBASE-26525 Use unique thread name for group WALs > -- > > Key: HBASE-28592 > URL: https://issues.apache.org/jira/browse/HBASE-28592 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28536) Fix `Disable Stripe Compaction` run error in document
[
https://issues.apache.org/jira/browse/HBASE-28536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-28536.
---
Fix Version/s: 4.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
Merged to master.
Thanks [~mrzhao] for contributing!
> Fix `Disable Stripe Compaction` run error in document
> --
>
> Key: HBASE-28536
> URL: https://issues.apache.org/jira/browse/HBASE-28536
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.5.6
>Reporter: Moran
>Assignee: Moran
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1
>
>
> *Disable Stripe Compaction* in document is
> {code:java}
> alter 'orders_table', CONFIGURATION =>
> {'hbase.hstore.engine.class' =>
> 'rg.apache.hadoop.hbase.regionserver.DefaultStoreEngine'}{code}
> This should be 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine'
> This will cause all regions to be in the openning state.Finally, I went
> through the disable table and corrected it before enable it.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28536 Fix `Disable Stripe Compaction` run error in document [hbase]
Apache9 merged PR #5836: URL: https://github.com/apache/hbase/pull/5836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28604) Fix the error message in ReservoirSample's constructor
[ https://issues.apache.org/jira/browse/HBASE-28604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28604. --- Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks [~ndimiduk] for reviewing! > Fix the error message in ReservoirSample's constructor > -- > > Key: HBASE-28604 > URL: https://issues.apache.org/jira/browse/HBASE-28604 > Project: HBase > Issue Type: Bug > Components: util >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28604 Fix the error message in ReservoirSample's constructor [hbase]
Apache9 merged PR #5920: URL: https://github.com/apache/hbase/pull/5920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-28294) Support to skip Kerberos authentication for metric endpoints
[ https://issues.apache.org/jira/browse/HBASE-28294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] youngju kim reassigned HBASE-28294: --- Assignee: youngju kim > Support to skip Kerberos authentication for metric endpoints > > > Key: HBASE-28294 > URL: https://issues.apache.org/jira/browse/HBASE-28294 > Project: HBase > Issue Type: New Feature > Components: metrics, UI >Affects Versions: 2.5.5 >Reporter: YUBI LEE >Assignee: youngju kim >Priority: Major > > Make HBase support to skip Kerberos authentication for metric endpoints. > (e.g. /jvm, /prometheus, /metrics) > Since HBase uses KerberoAuthenticationHandler.java, whitelist configuration > can be used like > [HADOOP-16527|https://issues.apache.org/jira/browse/HADOOP-16527]. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28606 Support for build on mac M1 [hbase-connectors]
Apache-HBase commented on PR #129: URL: https://github.com/apache/hbase-connectors/pull/129#issuecomment-2118671349 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 43s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 7s | master passed | | +1 :green_heart: | compile | 0m 52s | master passed | | +1 :green_heart: | spotless | 0m 14s | branch has no errors when running spotless:check. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 49s | the patch passed | | +1 :green_heart: | compile | 0m 52s | the patch passed | | +1 :green_heart: | javac | 0m 52s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | spotless | 0m 12s | patch has no errors when running spotless:check. | | +1 :green_heart: | javadoc | 0m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 40s | root in the patch passed. | | | | 18m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-129/1/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-connectors/pull/129 | | Optional Tests | dupname javac javadoc unit spotless xml compile | | uname | Linux e70c12538c7f 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 GNU/Linux | | Build tool | hb_maven | | Personality | dev-support/jenkins/hbase-personality.sh | | git revision | master / 307607c | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-129/1/testReport/ | | Max. process+thread count | 944 (vs. ulimit of 12500) | | modules | C: . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-129/1/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25972) Dual File Compaction
[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847492#comment-17847492
]
Hudson commented on HBASE-25972:
Results for branch branch-2
[build #1057 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Dual File Compaction
>
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
>Reporter: Kadir Ozdemir
>Assignee: Kadir Ozdemir
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847491#comment-17847491
]
Hudson commented on HBASE-26048:
Results for branch branch-2
[build #1057 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28598) NPE for writer object access in AsyncFSWAL#closeWriter
[
https://issues.apache.org/jira/browse/HBASE-28598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847490#comment-17847490
]
Hudson commented on HBASE-28598:
Results for branch branch-2
[build #1057 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1057/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> NPE for writer object access in AsyncFSWAL#closeWriter
> --
>
> Key: HBASE-28598
> URL: https://issues.apache.org/jira/browse/HBASE-28598
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 2.6.0, 2.4.18, 2.5.9
>Reporter: Vineet Kumar Maheshwari
>Assignee: Vineet Kumar Maheshwari
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 2.6.1, 2.5.9
>
>
> Observed NPE during execution of some of the UT cases.
> Exception is happening in AbstractFSWAL#closeWriter when trying to put null
> writer object in inflightWALClosures map.
> Need to add null check for writer object in AsyncFSWAL#doShutdown function
> before its usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] HBASE-28606 Support for build on mac M1 [hbase-connectors]
nikita15p opened a new pull request, #129: URL: https://github.com/apache/hbase-connectors/pull/129 This change will allow hbase spark connector to compile on mac M1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28606) [hbase-connectors] Support for build on mac M1
Nikita Pande created HBASE-28606: Summary: [hbase-connectors] Support for build on mac M1 Key: HBASE-28606 URL: https://issues.apache.org/jira/browse/HBASE-28606 Project: HBase Issue Type: Improvement Reporter: Nikita Pande [INFO] --- protobuf:0.6.1:compile (compile-protoc) @ hbase-spark-protocol --- [ERROR] PROTOC FAILED: -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28606) [hbase-connectors] Support for build on mac M1
[ https://issues.apache.org/jira/browse/HBASE-28606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita Pande reassigned HBASE-28606: Assignee: Nikita Pande > [hbase-connectors] Support for build on mac M1 > -- > > Key: HBASE-28606 > URL: https://issues.apache.org/jira/browse/HBASE-28606 > Project: HBase > Issue Type: Improvement >Reporter: Nikita Pande >Assignee: Nikita Pande >Priority: Major > > [INFO] --- protobuf:0.6.1:compile (compile-protoc) @ hbase-spark-protocol --- > [ERROR] PROTOC FAILED: -- This message was sent by Atlassian Jira (v8.20.10#820010)
