[jira] [Updated] (HBASE-28616) Remove/Deprecated the rs.* related configuration in TableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-28616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28616: -- Status: Patch Available (was: Open) > Remove/Deprecated the rs.* related configuration in TableOutputFormat > - > > Key: HBASE-28616 > URL: https://issues.apache.org/jira/browse/HBASE-28616 > Project: HBase > Issue Type: Task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28474. --- Resolution: Fixed Done. > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28471) Release 2.4.18
[ https://issues.apache.org/jira/browse/HBASE-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28471. --- Resolution: Fixed Done. > Release 2.4.18 > -- > > Key: HBASE-28471 > URL: https://issues.apache.org/jira/browse/HBASE-28471 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > The 2.6.0 release vote is ongoing, let's release 2.4.18 and mark 2.4.x as EOL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849451#comment-17849451 ] Duo Zhang edited comment on HBASE-28474 at 5/25/24 2:54 PM: # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) (/) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email (/) was (Author: apache9): # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) (/) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28617) Add trademark statement in footer on our website
Duo Zhang created HBASE-28617: - Summary: Add trademark statement in footer on our website Key: HBASE-28617 URL: https://issues.apache.org/jira/browse/HBASE-28617 Project: HBase Issue Type: Task Components: website Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28616) Remove/Deprecated the rs.* related configuration in TableOutputFormat
Duo Zhang created HBASE-28616: - Summary: Remove/Deprecated the rs.* related configuration in TableOutputFormat Key: HBASE-28616 URL: https://issues.apache.org/jira/browse/HBASE-28616 Project: HBase Issue Type: Task Components: mapreduce Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28473) Add 2.4.18 to download page
[ https://issues.apache.org/jira/browse/HBASE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28473. --- Fix Version/s: 4.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~sunxin] for reviewing! > Add 2.4.18 to download page > --- > > Key: HBASE-28473 > URL: https://issues.apache.org/jira/browse/HBASE-28473 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849451#comment-17849451 ] Duo Zhang edited comment on HBASE-28474 at 5/25/24 12:55 PM: - # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) (/) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email was (Author: apache9): # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28591) Backport HBASE-26123 Restore fields dropped by HBASE-25986 to public interfaces
[ https://issues.apache.org/jira/browse/HBASE-28591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849454#comment-17849454 ] Duo Zhang commented on HBASE-28591: --- Why the parent issue was only committed to branch-2.4? And for deprecation, we should mention in which version we plan to remove it. Thanks. > Backport HBASE-26123 Restore fields dropped by HBASE-25986 to public > interfaces > --- > > Key: HBASE-28591 > URL: https://issues.apache.org/jira/browse/HBASE-28591 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849451#comment-17849451 ] Duo Zhang edited comment on HBASE-28474 at 5/25/24 10:39 AM: - # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email was (Author: apache9): # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28474: - Assignee: Duo Zhang > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28474: -- Description: # Release the artifacts on repository.apache.org # Move the binaries from dist-dev to dist-release # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 (/) # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 # Add release data on https://reporter.apache.org/addrelease.html?hbase # Send announcement email was: # Release the artifacts on repository.apache.org # Move the binaries from dist-dev to dist-release # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 # Add release data on https://reporter.apache.org/addrelease.html?hbase # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 (/) > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849451#comment-17849451 ] Duo Zhang commented on HBASE-28474: --- # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28474) Finish 2.4.18 release
[ https://issues.apache.org/jira/browse/HBASE-28474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849451#comment-17849451 ] Duo Zhang edited comment on HBASE-28474 at 5/25/24 10:29 AM: - # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 (/) # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email was (Author: apache9): # Release the artifacts on repository.apache.org (/) # Move the binaries from dist-dev to dist-release (/) # Add xml to download page(via HBASE-28473) # Push tag 2.4.18RCx as tag rel/2.4.18 # Release 2.4.18 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12353080 # Add release data on https://reporter.apache.org/addrelease.html?hbase (/) # Send announcement email > Finish 2.4.18 release > - > > Key: HBASE-28474 > URL: https://issues.apache.org/jira/browse/HBASE-28474 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Priority: Major > > # Release the artifacts on repository.apache.org > # Move the binaries from dist-dev to dist-release > # Add xml to download page(via HBASE-28473) > # Push tag 2.4.18RCx as tag rel/2.4.18 > # Release 2.4.18 on JIRA > https://issues.apache.org/jira/projects/HBASE/versions/12353080 > # Add release data on https://reporter.apache.org/addrelease.html?hbase > # Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28473) Add 2.4.18 to download page
[ https://issues.apache.org/jira/browse/HBASE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28473: - Assignee: Duo Zhang > Add 2.4.18 to download page > --- > > Key: HBASE-28473 > URL: https://issues.apache.org/jira/browse/HBASE-28473 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28473) Add 2.4.18 to download page
[ https://issues.apache.org/jira/browse/HBASE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28473 started by Duo Zhang. - > Add 2.4.18 to download page > --- > > Key: HBASE-28473 > URL: https://issues.apache.org/jira/browse/HBASE-28473 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28472) Put up 2.4.18RC0
[ https://issues.apache.org/jira/browse/HBASE-28472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28472. --- Resolution: Fixed Done. > Put up 2.4.18RC0 > > > Key: HBASE-28472 > URL: https://issues.apache.org/jira/browse/HBASE-28472 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28473) Add 2.4.18 to download page
[ https://issues.apache.org/jira/browse/HBASE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28473: -- Component/s: website > Add 2.4.18 to download page > --- > > Key: HBASE-28473 > URL: https://issues.apache.org/jira/browse/HBASE-28473 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28425) Allow specify cluster key without zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-28425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28425. --- Fix Version/s: 3.0.0-beta-2 Hadoop Flags: Reviewed Release Note: Now you can use connection uri to fill the replication peer's cluster key field. The old cluster key format is still supported but not recommended. Resolution: Fixed > Allow specify cluster key without zookeeper in replication > -- > > Key: HBASE-28425 > URL: https://issues.apache.org/jira/browse/HBASE-28425 > Project: HBase > Issue Type: Improvement > Components: Replication, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > > When reviewing the usage of zookeeper in HBase, I found out that, we still > rely on zookeeper when specifying the cluster key when setting up > replication. If we want to completely hide zookeeper from outside a cluster, > we should also remove this cluster key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28577) Remove deprecated methods in KeyValue
[ https://issues.apache.org/jira/browse/HBASE-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28577: -- Release Note: These classes, fields and methods are removed from KeyValue KVComparator MetaComparator COMPARATOR META_COMPARATOR oswrite(KeyValue, OutputStream, boolean) > Remove deprecated methods in KeyValue > - > > Key: HBASE-28577 > URL: https://issues.apache.org/jira/browse/HBASE-28577 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: lixiaobao >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28615) Bump requests from 2.31.0 to 2.32.2 in /dev-support/git-jira-release-audit
[ https://issues.apache.org/jira/browse/HBASE-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28615. --- Fix Version/s: 4.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > Bump requests from 2.31.0 to 2.32.2 in /dev-support/git-jira-release-audit > -- > > Key: HBASE-28615 > URL: https://issues.apache.org/jira/browse/HBASE-28615 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28615) Bump requests from 2.31.0 to 2.32.2 in /dev-support/git-jira-release-audit
Duo Zhang created HBASE-28615: - Summary: Bump requests from 2.31.0 to 2.32.2 in /dev-support/git-jira-release-audit Key: HBASE-28615 URL: https://issues.apache.org/jira/browse/HBASE-28615 Project: HBase Issue Type: Task Components: dependabot, scripts, security Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28577) Remove deprecated methods in KeyValue
[ https://issues.apache.org/jira/browse/HBASE-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28577: -- Description: (was: Removed these classses, fields and methods from KeyValue KeyValue.oswrite(KeyValue, OutputStream, boolean) KeyValue.KVComparator KeyValue.MetaComparator KeyValue.COMPARATOR KeyValue.META_COMPARATOR) > Remove deprecated methods in KeyValue > - > > Key: HBASE-28577 > URL: https://issues.apache.org/jira/browse/HBASE-28577 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: lixiaobao >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28577) Remove deprecated methods in KeyValue
[ https://issues.apache.org/jira/browse/HBASE-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28577: -- Description: Removed these classses, fields and methods from KeyValue KeyValue.oswrite(KeyValue, OutputStream, boolean) KeyValue.KVComparator KeyValue.MetaComparator KeyValue.COMPARATOR KeyValue.META_COMPARATOR > Remove deprecated methods in KeyValue > - > > Key: HBASE-28577 > URL: https://issues.apache.org/jira/browse/HBASE-28577 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: lixiaobao >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > > Removed these classses, fields and methods from KeyValue > KeyValue.oswrite(KeyValue, OutputStream, boolean) > KeyValue.KVComparator > KeyValue.MetaComparator > KeyValue.COMPARATOR > KeyValue.META_COMPARATOR -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28577) Remove deprecated methods in KeyValue
[ https://issues.apache.org/jira/browse/HBASE-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28577. --- Fix Version/s: 3.0.0-beta-2 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-3. Thanks [~q977734161] for contributing! > Remove deprecated methods in KeyValue > - > > Key: HBASE-28577 > URL: https://issues.apache.org/jira/browse/HBASE-28577 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: lixiaobao >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26525) Use unique thread name for group WALs
[
https://issues.apache.org/jira/browse/HBASE-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848096#comment-17848096
]
Duo Zhang commented on HBASE-26525:
---
We missed the commit on branch-2 in the past so we need to reset the fix
versions.
Thanks [~szucsvillo] for helping!
> Use unique thread name for group WALs
> -
>
> Key: HBASE-26525
> URL: https://issues.apache.org/jira/browse/HBASE-26525
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0-alpha-2, 2.4.9, 2.7.0, 2.6.1, 2.5.9
>
> Attachments: image-2021-12-01-16-20-18-912.png,
> image-2021-12-01-16-21-18-032.png, image-2021-12-02-17-38-21-959.png
>
>
> The consumer threads for each WAL group has the same name, since they only
> use the WAL root dir in the thread name.
> {code:java}
> new ThreadFactoryBuilder().setNameFormat("AsyncFSWAL-%d-" +
> rootDir.toString()).
> setDaemon(true).build()); {code}
> For example, for BoundedGroupingStrategy, the consumer threads names are as
> follows,
> !image-2021-12-01-16-20-18-912.png|width=1199,height=130!
> We can use the log prefix instead, the consumer threads names will be changed
> to
> !image-2021-12-02-17-38-21-959.png|width=1102,height=197!
> So we can clearly see what happens from the log and the jstack info if
> something wrong with the WAL.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-26525) Use unique thread name for group WALs
[
https://issues.apache.org/jira/browse/HBASE-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-26525.
---
Fix Version/s: 2.7.0
2.6.1
2.5.9
(was: 2.5.0)
Hadoop Flags: Reviewed
Resolution: Fixed
> Use unique thread name for group WALs
> -
>
> Key: HBASE-26525
> URL: https://issues.apache.org/jira/browse/HBASE-26525
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 2.6.1, 2.5.9, 2.4.9, 3.0.0-alpha-2
>
> Attachments: image-2021-12-01-16-20-18-912.png,
> image-2021-12-01-16-21-18-032.png, image-2021-12-02-17-38-21-959.png
>
>
> The consumer threads for each WAL group has the same name, since they only
> use the WAL root dir in the thread name.
> {code:java}
> new ThreadFactoryBuilder().setNameFormat("AsyncFSWAL-%d-" +
> rootDir.toString()).
> setDaemon(true).build()); {code}
> For example, for BoundedGroupingStrategy, the consumer threads names are as
> follows,
> !image-2021-12-01-16-20-18-912.png|width=1199,height=130!
> We can use the log prefix instead, the consumer threads names will be changed
> to
> !image-2021-12-02-17-38-21-959.png|width=1102,height=197!
> So we can clearly see what happens from the log and the jstack info if
> something wrong with the WAL.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-26525) Use unique thread name for group WALs
[
https://issues.apache.org/jira/browse/HBASE-26525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reopened HBASE-26525:
---
> Use unique thread name for group WALs
> -
>
> Key: HBASE-26525
> URL: https://issues.apache.org/jira/browse/HBASE-26525
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.9
>
> Attachments: image-2021-12-01-16-20-18-912.png,
> image-2021-12-01-16-21-18-032.png, image-2021-12-02-17-38-21-959.png
>
>
> The consumer threads for each WAL group has the same name, since they only
> use the WAL root dir in the thread name.
> {code:java}
> new ThreadFactoryBuilder().setNameFormat("AsyncFSWAL-%d-" +
> rootDir.toString()).
> setDaemon(true).build()); {code}
> For example, for BoundedGroupingStrategy, the consumer threads names are as
> follows,
> !image-2021-12-01-16-20-18-912.png|width=1199,height=130!
> We can use the log prefix instead, the consumer threads names will be changed
> to
> !image-2021-12-02-17-38-21-959.png|width=1102,height=197!
> So we can clearly see what happens from the log and the jstack info if
> something wrong with the WAL.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28592) Backport HBASE-26525 Use unique thread name for group WALs
[ https://issues.apache.org/jira/browse/HBASE-28592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28592. --- Resolution: Fixed Just reuse the parent issue to land the PRs so do not set any fix versions here. Thanks [~szucsvillo] for catching this! > Backport HBASE-26525 Use unique thread name for group WALs > -- > > Key: HBASE-28592 > URL: https://issues.apache.org/jira/browse/HBASE-28592 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28607) Bump requests from 2.31.0 to 2.32.0 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28607. --- Fix Version/s: 2.4.18 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all active branches. > Bump requests from 2.31.0 to 2.32.0 in /dev-support/flaky-tests > --- > > Key: HBASE-28607 > URL: https://issues.apache.org/jira/browse/HBASE-28607 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28607) Bump requests from 2.31.0 to 2.32.0 in /dev-support/flaky-tests
Duo Zhang created HBASE-28607: - Summary: Bump requests from 2.31.0 to 2.32.0 in /dev-support/flaky-tests Key: HBASE-28607 URL: https://issues.apache.org/jira/browse/HBASE-28607 Project: HBase Issue Type: Task Components: dependabot, scripts, security Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28547) Support specifying connection configuration through queries of the connection uri
[ https://issues.apache.org/jira/browse/HBASE-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28547. --- Fix Version/s: 2.7.0 3.0.0-beta-2 Resolution: Fixed Pushed to master, branch-3 and branch-2. Thanks [~ndimiduk] for reviewing and all others for helping! > Support specifying connection configuration through queries of the connection > uri > - > > Key: HBASE-28547 > URL: https://issues.apache.org/jira/browse/HBASE-28547 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28547) Support specifying connection configuration through queries of the connection uri
[ https://issues.apache.org/jira/browse/HBASE-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28547: -- Hadoop Flags: Reviewed Release Note: Now you can specify connection configurations through the connection URI queries. For example hbase+zk://zkserver:2181/hbase?zookeeper.session.timeout=15000 By introducing the zookeeper.session.timeout=15000 query in the URI, you can override the existing configuration value for zookeeper.session.timeout in the Configuration instance you used when calling ConnectionFactory.createConnection or createAsyncConnection. > Support specifying connection configuration through queries of the connection > uri > - > > Key: HBASE-28547 > URL: https://issues.apache.org/jira/browse/HBASE-28547 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28599) RowTooBigException is thrown when duplicate increment RPC call is attempted
[ https://issues.apache.org/jira/browse/HBASE-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28599. --- Fix Version/s: 2.4.18 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all active branches. Thanks [~fjvbn2003] for contributing and [~robiee17] for the great analyzing! > RowTooBigException is thrown when duplicate increment RPC call is attempted > --- > > Key: HBASE-28599 > URL: https://issues.apache.org/jira/browse/HBASE-28599 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.5.5, 2.5.6, 2.5.7, 2.5.8 >Reporter: Robin Infant A >Assignee: youngju kim >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > Attachments: RowTooBig_trace.txt > > > *Issue:* > `RowTooBigException` is thrown when a duplicate increment RPC call is > attempted. > *Expected Behavior:* > 1. The initial RPC increment call should time out for some reason. > 2. The duplicate RPC call should be converted to a GET request and fetch the > result that I am trying to increment. > 3. The result should contain only the qualifier that I am attempting to > increment. > *Actual Behavior:* > 1. The initial RPC increment call timed out, which is expected. > 2. The duplicate RPC call is converted to a GET request but fails to clone > the qualifier into the GET request. > 3. Hence, the GET request attempts to retrieve all qualifiers for the given > row and columnfamily, resulting in a `RowTooBigException`. > *Steps to Reproduce:* > 1. Ensure a row with a total value size exceeding `hbase.table.max.rowsize` > (default = 1073741824) exists. > 2. Nonce property should be enabled `hbase.client.nonces.enabled` which is > actually defaulted to true. > 3. Attempt to increment a qualifier against the same row. > 4. In my case, I am using a postIncrement co-processor which may cause a > delay (longer than the RPC timeout property). > 5. A duplicate increment call should be triggered, which tries to get the > value rather than increment it. > 6. The GET request actually tries to retrieve all the qualifiers for the row, > resulting in a `RowTooBigException`. > *Insights:* > Upon further debugging, I found that qualifiers are not cloned into the GET > instance due to incorrect usage of > [CellScanner.advance|https://github.com/apache/hbase/blob/7ebd4381261fefd78fc2acf258a95184f4147cee/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3833] > *Fix Suggestion:* > Removing the `!` operation from `while (!CellScanner.advance)` may resolve > the issue. > Attached Exception Stack Trace for reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (HBASE-28501) Support non-SPNEGO authentication methods and implement session handling in REST java client library
[ https://issues.apache.org/jira/browse/HBASE-28501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-28501: --- When putting up 2.4.18RC0, I found that the commit for this issue on branch-2.4 removed a constructor from IA.Public class Client.java in the rest module. https://dist.apache.org/repos/dist/dev/hbase/2.4.18RC0/api_compare_2.4.17_to_2.4.18RC0.html#Binary_Removed Client.Client ( Cluster cluster, Configuration conf, String trustStorePath, Optional trustStorePassword, Optional trustStoreType ) We should follow the deprecated cycle when we want to remove a method from a IA.Public class, unelss there are some special reasons. > Support non-SPNEGO authentication methods and implement session handling in > REST java client library > > > Key: HBASE-28501 > URL: https://issues.apache.org/jira/browse/HBASE-28501 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > The current java client only supports the SPENGO authentication method. > This does not support the case when an application proxy like Apache Knox > performs AAA conversion from BASIC/DIGEST to kerberos authentication. > Add support for BASIC username/password auth the client. > Generally, the authentication code in the client looks quite backwards, it > seems that most of the kerberos / auth cookie code duplicates HttpClient > functionality. AFAICT setting HttpClient up (or letting user set it up) , and > letting it handle authentication by itself would be a better and more generic > solution. > -Also add support for specifying a prefix for the URL path.- -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28174) DELETE endpoint in REST API does not support deleting binary row keys/columns
[
https://issues.apache.org/jira/browse/HBASE-28174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-28174:
--
Fix Version/s: (was: 4.0.0-alpha-1)
> DELETE endpoint in REST API does not support deleting binary row keys/columns
> -
>
> Key: HBASE-28174
> URL: https://issues.apache.org/jira/browse/HBASE-28174
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1
>Reporter: James Udiljak
>Assignee: James Udiljak
>Priority: Blocker
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.9
>
> Attachments: delete_base64_1.png
>
>
> h2. Notes
> This is the first time I have raised an issue in the ASF Jira. Please let me
> know if there's anything I need to adjust on the issue to fit in with your
> development flow.
> I have marked the priority as "blocker" because this issue blocks me as a
> user of the HBase REST API from deploying an effective solution for our
> setup. Please feel free to change this if the Priority field has another
> meaning to you.
> I have also chosen 2.4.17 as the affected version because this is the version
> I am running, however looking at the source code on GitHub in the default
> branch, I think many other versions would be affected.
> h2. Description of Issue
> The DELETE operation in the [HBase REST
> API|https://hbase.apache.org/1.2/apidocs/org/apache/hadoop/hbase/rest/package-summary.html#operation_delete]
> requires specifying row keys and column families/offsets in the URI (i.e. as
> UTF-8 text). This makes it impossible to specify a delete operation via the
> REST API for a binary row key or column family/offset, as single bytes with a
> decimal value greater than 127 are not valid in UTF-8.
> Percent-encoding these "high" values does not work around the issue, as the
> HBase REST API uses Java's {{URLDecoder.Decode(percentEncodedString,
> "UTF-8")}} function, which replaces any percent-encoded byte in the range
> {{%80}} to {{%FF}} with the [replacement
> character|https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character].
> Even if this were not the case, the row-key is ultimately [converted to a
> byte
> array|https://github.com/apache/hbase/blob/rel/2.4.17/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RowSpec.java#L60-L100]
> using UTF-8 encoding, wherein code points >127 are encoded across multiple
> bytes, corrupting the user-supplied row key.
> h2. Proposed Solution
> I do not believe it is possible to allow encoding of arbitrary bytes in the
> URL for the DELETE endpoint without breaking compatibility for any users who
> may have been unknowingly UTF-8 encoding their binary row keys. Even if it
> were possible, the syntax would likely be terse.
> Instead, I propose a new version of the DELETE endpoint that would accept row
> keys and column families/offsets in the request _body_ (using Base64 encoding
> for the JSON and XML formats, and bare binary for protobuf). This new
> endpoint would follow the same conventions as the PUT operations, except that
> cell values would not need to be specified (unless the user is performing a
> check-and-delete operation).
> As an additional benefit, using the request body could potentially allow for
> deleting multiple rows in a single request, which would drastically improve
> the efficiency of my use case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28524) Backport HBASE-28174 to branch-2.4 and branch-2.5
[ https://issues.apache.org/jira/browse/HBASE-28524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28524: -- Fix Version/s: (was: 2.4.18) (was: 2.5.9) > Backport HBASE-28174 to branch-2.4 and branch-2.5 > - > > Key: HBASE-28524 > URL: https://issues.apache.org/jira/browse/HBASE-28524 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.4.17, 2.5.8 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > The changes are backwards compatible and the REST interface is super limited > without them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28417) TestBlockingIPC.testBadPreambleHeader sometimes fails with broken pipe instead of bad auth
[ https://issues.apache.org/jira/browse/HBASE-28417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28417: -- Fix Version/s: (was: 2.4.18) > TestBlockingIPC.testBadPreambleHeader sometimes fails with broken pipe > instead of bad auth > -- > > Key: HBASE-28417 > URL: https://issues.apache.org/jira/browse/HBASE-28417 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 3.0.0-beta-2, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27923) NettyRpcServer may hange if it should skip initial sasl handshake
[
https://issues.apache.org/jira/browse/HBASE-27923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27923:
--
Fix Version/s: 2.5.6
(was: 2.5.9)
> NettyRpcServer may hange if it should skip initial sasl handshake
> -
>
> Key: HBASE-27923
> URL: https://issues.apache.org/jira/browse/HBASE-27923
> Project: HBase
> Issue Type: Bug
> Components: netty, rpc, security
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> {{NettyRpcServer}} may hange if it should skip initial sasl handshake when
> server does not enable security and client enables security, I think this
> problem is caused by two reasons：
> * For Server:
> The type of the response is {{RpcResponse}}, but for
> {{NettyRpcServerPreambleHandler}},when it send {{RpcResponse}}
> ,{{NettyRpcServerResponseEncoder}} does not exist, so {{RpcResponse}}
> messages cannot be sent.
> * For Client
> When {{NettyHBaseSaslRpcClientHandler}} receives
> {{SaslUtil.SWITCH_TO_SIMPLE_AUTH}}, it does not remove
> {{SaslChallengeDecoder}} and {{NettyHBaseSaslRpcClientHandler}}, so the
> latter responses are considered to be incorrect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28255) Correcting spelling errors or annotations with non-standard spelling
[ https://issues.apache.org/jira/browse/HBASE-28255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28255: -- Fix Version/s: 2.4.18 > Correcting spelling errors or annotations with non-standard spelling > > > Key: HBASE-28255 > URL: https://issues.apache.org/jira/browse/HBASE-28255 > Project: HBase > Issue Type: Improvement >Reporter: mazhengxuan >Priority: Minor > Labels: documentation > Fix For: 2.6.0, 2.4.18, 2.5.8, 3.0.0-beta-2 > > > Modify some spelling errors or non-standard spelling comments pointed out by > Typo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27923) NettyRpcServer may hange if it should skip initial sasl handshake
[
https://issues.apache.org/jira/browse/HBASE-27923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27923:
--
Fix Version/s: 2.4.18
2.5.9
> NettyRpcServer may hange if it should skip initial sasl handshake
> -
>
> Key: HBASE-27923
> URL: https://issues.apache.org/jira/browse/HBASE-27923
> Project: HBase
> Issue Type: Bug
> Components: netty, rpc, security
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 4.0.0-alpha-1
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.9
>
>
> {{NettyRpcServer}} may hange if it should skip initial sasl handshake when
> server does not enable security and client enables security, I think this
> problem is caused by two reasons：
> * For Server:
> The type of the response is {{RpcResponse}}, but for
> {{NettyRpcServerPreambleHandler}},when it send {{RpcResponse}}
> ,{{NettyRpcServerResponseEncoder}} does not exist, so {{RpcResponse}}
> messages cannot be sent.
> * For Client
> When {{NettyHBaseSaslRpcClientHandler}} receives
> {{SaslUtil.SWITCH_TO_SIMPLE_AUTH}}, it does not remove
> {{SaslChallengeDecoder}} and {{NettyHBaseSaslRpcClientHandler}}, so the
> latter responses are considered to be incorrect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27758) Inconsistent synchronization in MetricsUserSourceImpl
[ https://issues.apache.org/jira/browse/HBASE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27758: -- Fix Version/s: 2.4.18 (was: 2.4.17) > Inconsistent synchronization in MetricsUserSourceImpl > - > > Key: HBASE-27758 > URL: https://issues.apache.org/jira/browse/HBASE-27758 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18 > > > Picked up by spotbugs: > > ||Code||Warning|| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 253] > Synchronized access at MetricsUserSourceImpl.java:[line 146]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto; locked > 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 241] > Synchronized access at MetricsUserSourceImpl.java:[line 141]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 247] > Synchronized access at MetricsUserSourceImpl.java:[line 145]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 264] > Synchronized access at MetricsUserSourceImpl.java:[line 142]| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27758) Inconsistent synchronization in MetricsUserSourceImpl
[ https://issues.apache.org/jira/browse/HBASE-27758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27758: -- Component/s: metrics > Inconsistent synchronization in MetricsUserSourceImpl > - > > Key: HBASE-27758 > URL: https://issues.apache.org/jira/browse/HBASE-27758 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Major > Labels: patch-available > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18 > > > Picked up by spotbugs: > > ||Code||Warning|| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.appendHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 253] > Synchronized access at MetricsUserSourceImpl.java:[line 146]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto; locked > 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.getHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 241] > Synchronized access at MetricsUserSourceImpl.java:[line 141]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.incrementHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 247] > Synchronized access at MetricsUserSourceImpl.java:[line 145]| > |IS|Inconsistent synchronization of > org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto; > locked 50% of time| > | |[Bug type IS2_INCONSISTENT_SYNC (click for > details)|https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5067/7/artifact/yetus-general-check/output/new-spotbugs-hbase-hadoop-compat.html#IS2_INCONSISTENT_SYNC] > In class org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl > Field org.apache.hadoop.hbase.regionserver.MetricsUserSourceImpl.scanTimeHisto > Synchronized 50% of the time > Unsynchronized access at MetricsUserSourceImpl.java:[line 264] > Synchronized access at MetricsUserSourceImpl.java:[line 142]| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27704:
--
Component/s: Quotas
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
> Components: Quotas
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18
>
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27704) Quotas can drastically overflow configured limit
[
https://issues.apache.org/jira/browse/HBASE-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27704:
--
Fix Version/s: 2.4.18
(was: 2.4.17)
> Quotas can drastically overflow configured limit
>
>
> Key: HBASE-27704
> URL: https://issues.apache.org/jira/browse/HBASE-27704
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.4, 2.4.18
>
> Attachments: Screenshot 2023-03-10 at 5.17.51 PM.png
>
>
> The original implementation did not allow exceeding quota. For example, you
> specify a limit of 10 resource/sec and consume 20 resources, it takes 1.1
> seconds to be able submit another request. This was covered by the
> [testOverconsumption in
> TestRateLimiter|https://github.com/apache/hbase/blame/587b0b4f20bdc0415b6541023e611b69c87dba15/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L97].
> As an incidental part of HBASE-13686, that logic was changed. There is no
> mention of the reasoning behind the change in the issue comments or review
> board, I think it was missed. The goal of that issue was to add different
> refill strategies, but it also modified the over consumption. The
> testOverconsumption was [split out for both refill
> strategies|https://github.com/apache/hbase/blame/master/hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRateLimiter.java#L104-L159],
> but the core reasoning was lost. The comment says:
> {code:java}
> // 10 resources are available, but we need to consume 20 resources109
> // Verify that we have to wait at least 1.1sec to have 1 resource available
> {code}
> But the actual test was updated to only require a new resource after 100ms.
> This is incorrect.
> The problem is, when consuming if you go negative it sets to 0
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L187-L191].
> Additionally, when refilling the new logic does a Math.max(0, available +
> refillAmount):
> [here|https://github.com/apache/hbase/blame/master/hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RateLimiter.java#L159-L163].
> So it's really impossible to get below 0, which is impractical for a rate
> limiter.
> With this setup it's very easy to drastically overconsume the rate limiter.
> See attached screenshot, which shows two humps. The first one has the current
> logic, the second hump has my fix which removes both of those problems. The
> rate limit was set to 500mb/s, but I was easily able to go over 700 mb/s
> without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28595: -- Component/s: (was: Client) > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28595. --- Fix Version/s: 2.4.18 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all active branches. Thanks [~csringhofer] for contributing and all others for helping and reviewing! > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28592) Backport HBASE-26525 Use unique thread name for group WALs
[ https://issues.apache.org/jira/browse/HBASE-28592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847503#comment-17847503 ] Duo Zhang commented on HBASE-28592: --- [~szucsvillo] If you do not mind, I will close this issue and reopen the parent issue to cherry-pick the commits to other branches, and reset the fix versions of the parent issue. Thanks. > Backport HBASE-26525 Use unique thread name for group WALs > -- > > Key: HBASE-28592 > URL: https://issues.apache.org/jira/browse/HBASE-28592 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28536) Fix `Disable Stripe Compaction` run error in document
[
https://issues.apache.org/jira/browse/HBASE-28536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-28536.
---
Fix Version/s: 4.0.0-alpha-1
Hadoop Flags: Reviewed
Resolution: Fixed
Merged to master.
Thanks [~mrzhao] for contributing!
> Fix `Disable Stripe Compaction` run error in document
> --
>
> Key: HBASE-28536
> URL: https://issues.apache.org/jira/browse/HBASE-28536
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.5.6
>Reporter: Moran
>Assignee: Moran
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1
>
>
> *Disable Stripe Compaction* in document is
> {code:java}
> alter 'orders_table', CONFIGURATION =>
> {'hbase.hstore.engine.class' =>
> 'rg.apache.hadoop.hbase.regionserver.DefaultStoreEngine'}{code}
> This should be 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine'
> This will cause all regions to be in the openning state.Finally, I went
> through the disable table and corrected it before enable it.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28604) Fix the error message in ReservoirSample's constructor
[ https://issues.apache.org/jira/browse/HBASE-28604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28604. --- Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks [~ndimiduk] for reviewing! > Fix the error message in ReservoirSample's constructor > -- > > Key: HBASE-28604 > URL: https://issues.apache.org/jira/browse/HBASE-28604 > Project: HBase > Issue Type: Bug > Components: util >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-26048.
---
Fix Version/s: 2.4.18
2.7.0
3.0.0-beta-2
2.6.1
2.5.9
Hadoop Flags: Reviewed
Resolution: Fixed
Pushed to all active branches.
Thanks [~sunxin] for reviewing!
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9
>
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28598) NPE for writer object access in AsyncFSWAL#closeWriter
[ https://issues.apache.org/jira/browse/HBASE-28598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28598. --- Fix Version/s: 2.4.18 2.7.0 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all branch-2.x. Thanks [~vineet.4008]! > NPE for writer object access in AsyncFSWAL#closeWriter > -- > > Key: HBASE-28598 > URL: https://issues.apache.org/jira/browse/HBASE-28598 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.6.0, 2.4.18, 2.5.9 >Reporter: Vineet Kumar Maheshwari >Assignee: Vineet Kumar Maheshwari >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 2.6.1, 2.5.9 > > > Observed NPE during execution of some of the UT cases. > Exception is happening in AbstractFSWAL#closeWriter when trying to put null > writer object in inflightWALClosures map. > Need to add null check for writer object in AsyncFSWAL#doShutdown function > before its usage. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28604) Fix the error message in ReservoirSample's constructor
Duo Zhang created HBASE-28604: - Summary: Fix the error message in ReservoirSample's constructor Key: HBASE-28604 URL: https://issues.apache.org/jira/browse/HBASE-28604 Project: HBase Issue Type: Bug Components: util Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28472) Put up 2.4.18RC0
[ https://issues.apache.org/jira/browse/HBASE-28472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28472 started by Duo Zhang. - > Put up 2.4.18RC0 > > > Key: HBASE-28472 > URL: https://issues.apache.org/jira/browse/HBASE-28472 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28472) Put up 2.4.18RC0
[ https://issues.apache.org/jira/browse/HBASE-28472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28472: - Assignee: Duo Zhang > Put up 2.4.18RC0 > > > Key: HBASE-28472 > URL: https://issues.apache.org/jira/browse/HBASE-28472 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28472) Put up 2.4.18RC0
[ https://issues.apache.org/jira/browse/HBASE-28472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847339#comment-17847339 ] Duo Zhang commented on HBASE-28472: --- 2.6.0 is out, let's start the process for the last 2.4.x release. > Put up 2.4.18RC0 > > > Key: HBASE-28472 > URL: https://issues.apache.org/jira/browse/HBASE-28472 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
[ https://issues.apache.org/jira/browse/HBASE-28579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28579. --- Fix Version/s: 3.0.0-beta-2 Hadoop Flags: Reviewed Release Note: Change the below method in StoreFileReader from public to protected HFileScanner getScanner(boolean, boolean) And change to use StoreFileReader instead of HFileScanner in test code as much as possible. Resolution: Fixed > Hide HFileScanner related methods in StoreFileReader > > > Key: HBASE-28579 > URL: https://issues.apache.org/jira/browse/HBASE-28579 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28578) Remove deprecated methods in HFileScanner
[ https://issues.apache.org/jira/browse/HBASE-28578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28578. --- Fix Version/s: 3.0.0-beta-2 Hadoop Flags: Reviewed Release Note: Remove getKeyString and getValueString methods in HFileScanner. Resolution: Fixed > Remove deprecated methods in HFileScanner > - > > Key: HBASE-28578 > URL: https://issues.apache.org/jira/browse/HBASE-28578 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28572) Remove deprecated methods in thrift module
[ https://issues.apache.org/jira/browse/HBASE-28572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28572. --- Hadoop Flags: Incompatible change,Reviewed Release Note: Remove isTableAvailableWithSplit in thrift API. Regenerate all the thrift java/python code in hbase-thrift and hbase-examples modules. Resolution: Fixed > Remove deprecated methods in thrift module > -- > > Key: HBASE-28572 > URL: https://issues.apache.org/jira/browse/HBASE-28572 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28383) [JDK17] Update hbase-env.sh with alternates to JVM flags which are no longer supported
[
https://issues.apache.org/jira/browse/HBASE-28383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847219#comment-17847219
]
Duo Zhang commented on HBASE-28383:
---
Any updates here?
[~nihaljain.cs]
Thanks.
> [JDK17] Update hbase-env.sh with alternates to JVM flags which are no longer
> supported
> --
>
> Key: HBASE-28383
> URL: https://issues.apache.org/jira/browse/HBASE-28383
> Project: HBase
> Issue Type: Sub-task
> Components: java, scripts
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
>
> Some JVM flags like {{{}-XX:+PrintGCDetails{}}}, {{-XX:+PrintGCDateStamps}}
> etc. are no longer supported with JDK 17 and hbase would fail to start if
> these are passed.
> We should do an audit and update
> [https://github.com/apache/hbase/blob/master/conf/hbase-env.sh] to capture
> alternate/fix.
> Will refer following for a fix/repalacement:
> *
> [https://stackoverflow.com/questions/54144713/is-there-a-replacement-for-the-garbage-collection-jvm-args-in-java-11]
> *
> [https://docs.oracle.com/javase/9/tools/java.htm#GUID-BE93ABDC-999C-4CB5-A88B-1994AAAC74D5__CONVERTGCLOGGINGFLAGSTOXLOG-A5046BD1]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-26048:
-
Assignee: Duo Zhang
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-26048:
--
Component/s: proc-v2
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work started] (HBASE-26048) [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
[
https://issues.apache.org/jira/browse/HBASE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-26048 started by Duo Zhang.
-
> [JDK17] Replace the usage of deprecated API ThreadGroup.destroy()
> -
>
> Key: HBASE-26048
> URL: https://issues.apache.org/jira/browse/HBASE-26048
> Project: HBase
> Issue Type: Sub-task
> Components: proc-v2
>Reporter: Wei-Chiu Chuang
>Assignee: Duo Zhang
>Priority: Major
>
> According to the JDK17 doc, ThreadGroup.destroy() is deprecated because
> {quote}Deprecated, for removal: This API element is subject to removal in a
> future version.
> {quote}
> The API and mechanism for destroying a ThreadGroup is inherently flawed. The
> ability to explicitly or automatically destroy a thread group will be removed
> in a future release.
> [https://download.java.net/java/early_access/jdk17/docs/api/java.base/java/lang/ThreadGroup.html#destroy(])
> We don't necessarily need to remove this usage now, but the warning sounds
> bad enough.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-26047) [JDK17] Track JDK17 unit test failures
[
https://issues.apache.org/jira/browse/HBASE-26047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-26047.
---
Resolution: Fixed
Now we have JDK17 tests in pre commit for master branch, so I think we can
resolve this issue now.
Feel free to reopen if you think we need to keep this open.
> [JDK17] Track JDK17 unit test failures
> --
>
> Key: HBASE-26047
> URL: https://issues.apache.org/jira/browse/HBASE-26047
> Project: HBase
> Issue Type: Sub-task
>Reporter: Wei-Chiu Chuang
>Assignee: Yutong Xiao
>Priority: Major
>
> As of now, there are still two failed unit tests after exporting JDK internal
> modules and the modifier access hack.
> {noformat}
> [ERROR] Tests run: 7, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.217
> s <<< FAILURE! - in org.apache.hadoop.hbase.io.TestHeapSize
> [ERROR] org.apache.hadoop.hbase.io.TestHeapSize.testSizes Time elapsed:
> 0.041 s <<< FAILURE!
> java.lang.AssertionError: expected:<160> but was:<152>
> at
> org.apache.hadoop.hbase.io.TestHeapSize.testSizes(TestHeapSize.java:335)
> [ERROR] org.apache.hadoop.hbase.io.TestHeapSize.testNativeSizes Time
> elapsed: 0.01 s <<< FAILURE!
> java.lang.AssertionError: expected:<72> but was:<64>
> at
> org.apache.hadoop.hbase.io.TestHeapSize.testNativeSizes(TestHeapSize.java:134)
> [INFO] Running org.apache.hadoop.hbase.io.Tes
> [ERROR] Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.697
> s <<< FAILURE! - in org.apache.hadoop.hbase.ipc.TestBufferChain
> [ERROR] org.apache.hadoop.hbase.ipc.TestBufferChain.testWithSpy Time
> elapsed: 0.537 s <<< ERROR!
> java.lang.NullPointerException: Cannot enter synchronized block because
> "this.closeLock" is null
> at
> org.apache.hadoop.hbase.ipc.TestBufferChain.testWithSpy(TestBufferChain.java:119)
> {noformat}
> It appears that JDK17 makes the heap size estimate different than before. Not
> sure why.
> TestBufferChain.testWithSpy failure might be because of yet another
> unexported module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847004#comment-17847004
]
Duo Zhang commented on HBASE-28522:
---
The PR for HBASE-28582 is ready, I also created a test for it to show that it
will wait for RIT to finish before continue. PTAL.
In general, we can use the same solution for DisableTableProcedure, but we need
to change holdLock from true to false, which makes me a bit uncomfortable.
I will try to see if we do something like draining before starting to schedule
TRSP in DisableTableProcedure, so we can still keep holdLock to true in later
processing to simplify the logic. If not, let's change to use the same solution
in HBASE-28582, i.e, introduce a special CloseTableRegionsProcedure, to close
all regions for a table.
Thanks.
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2, Region Assignment
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Critical
> Attachments: timeline.jpg
>
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[
https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846860#comment-17846860
]
Duo Zhang commented on HBASE-28595:
---
The server side logic is for keep compatibility with some old clients, it is
really a pain if new client needs for keeping compatiblity with this piece of
code...
It does not make sense.
{quote}
1. last scan rpc arrives to server, rows are returned,
more_results_in_region=false
2. the response is lost due to network issues and the scan rpc is retried
3. as the scanner was closed on the server side in 1., an empty result is
returned
4. the client skips the rows returned in 1. and returns without an error
{quote}
We could also record the last callSeq in the closed scanners, so if the callSeq
repeats, we should throw a UnknownScannerException or
OutOfOrderScannerException.
If we still can not cover all the cases at server side, I would argue that we
just revert the code in HBASE-18042, as it is just for keep compatibility with
the thirdparty client in OpenTSDB. If we can not even make our official client
correct, we do not need to consider thirdparty client then...
Thanks.
> Losing exception from scan RPC can lead to partial results
> --
>
> Key: HBASE-28595
> URL: https://issues.apache.org/jira/browse/HBASE-28595
> Project: HBase
> Issue Type: Bug
> Components: Client, regionserver, Scanners
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Critical
> Labels: pull-request-available
>
> This was discovered in Apache Impala using HBase 2.2 based branch hbase
> client and server. It is not clear yet whether other branches are also
> affected.
> The issue happens if the server side of the scan throws an exception and
> closes the scanner, but at the same time, the client gets an rpc connection
> closed error and doesn't process the exception sent by the server. Client
> then thinks it got a network error, which leads to retrying the RPC instead
> of opening a new scanner. But then when the client retry reaches the server,
> the server returns an empty ScanResponse instead of an error, leading to
> closing the scanner on client side without returning any error.
> A few pointers to critical parts:
> region server:
> 1st call throws exception leading to closing (but not deleting) scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539]
> 2nd call (retry of 1st) returns empty results:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403]
> client:
> some exceptions are handled as non-retriable at RPC level and are only
> handled through opening a new scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214]
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367]
> This mechanism in the client only works if it gets the exception from the
> server. If there are connection issues during the RPC then the client won't
> really know the state of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846774#comment-17846774 ] Duo Zhang commented on HBASE-28595: --- [~MikaelSmith] Connection can be broken in any reason, it is not always reliable. The problem here is that, when server hits an exception and closes the scanner, it fails to return the exception to client because of a network issue. The client just receives a connection error(connection closed or connection reset, whatever), so it generates a retry RPC. But at server side, because of the compatiblity code we added, we will consider the request is coming from an old client where we will still issue a next or close when the scanner is exhausted, so we just return an empty result and cause unexpected behavior at client side. I think this should be fixed at server side, a simple fix would be that, only record a closed scanner when it is closed because of exhausted, if it is closed because of an exception, we should not record it so we will generate the correct UnknownScannerException to later client requests. Thanks. > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > Labels: pull-request-available > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28592) Backport HBASE-26525 Use unique thread name for group WALs
[ https://issues.apache.org/jira/browse/HBASE-28592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846690#comment-17846690 ] Duo Zhang commented on HBASE-28592: --- I think we should just reopen the parent issue and cherry-pick the commit to other branches, and change the fix versions... > Backport HBASE-26525 Use unique thread name for group WALs > -- > > Key: HBASE-28592 > URL: https://issues.apache.org/jira/browse/HBASE-28592 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28592) Backport HBASE-26525 Use unique thread name for group WALs
[ https://issues.apache.org/jira/browse/HBASE-28592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846688#comment-17846688 ] Duo Zhang commented on HBASE-28592: --- So we missed the parent commit for branch-2? That's bad. > Backport HBASE-26525 Use unique thread name for group WALs > -- > > Key: HBASE-28592 > URL: https://issues.apache.org/jira/browse/HBASE-28592 > Project: HBase > Issue Type: Sub-task >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[
https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846657#comment-17846657
]
Duo Zhang commented on HBASE-28595:
---
{quote}
Also, it seems we keep closed scanners for the time defined by
hbase.client.scanner.timeout.period, which is the actual scanner lease
expiration timeout. Maybe it should be a good idea to have a specific property
for cleaning the closed scanners collection.
{quote}
This is just for keep compatibility with older clients so I do not think we
need to introduce new configs.
> Losing exception from scan RPC can lead to partial results
> --
>
> Key: HBASE-28595
> URL: https://issues.apache.org/jira/browse/HBASE-28595
> Project: HBase
> Issue Type: Bug
> Components: Client, regionserver, Scanners
>Reporter: Csaba Ringhofer
>Assignee: Csaba Ringhofer
>Priority: Critical
>
> This was discovered in Apache Impala using HBase 2.2 based branch hbase
> client and server. It is not clear yet whether other branches are also
> affected.
> The issue happens if the server side of the scan throws an exception and
> closes the scanner, but at the same time, the client gets an rpc connection
> closed error and doesn't process the exception sent by the server. Client
> then thinks it got a network error, which leads to retrying the RPC instead
> of opening a new scanner. But then when the client retry reaches the server,
> the server returns an empty ScanResponse instead of an error, leading to
> closing the scanner on client side without returning any error.
> A few pointers to critical parts:
> region server:
> 1st call throws exception leading to closing (but not deleting) scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539]
> 2nd call (retry of 1st) returns empty results:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403]
> client:
> some exceptions are handled as non-retriable at RPC level and are only
> handled through opening a new scanner:
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214]
> [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367]
> This mechanism in the client only works if it gets the exception from the
> server. If there are connection issues during the RPC then the client won't
> really know the state of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846656#comment-17846656 ] Duo Zhang commented on HBASE-28595: --- In HBASE-17489, for keeping compatiblity with 1.x client, we introduced the logic but we will make sure it is a close scanner request. In HBASE-18042, for maintaining compatibility with the async hbase client used by OpenTSDB, we removed the check for request. I think this is root cause. Not sure if it is still a problem for OpenTSDB now, if not a problem we can just change the code back. Otherwise I think we need to check whether the scanner close is triggerred by exhausted, if not we should not return empty to client, we should just throw UnknownScannerException. > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but at the same time, the client gets an rpc connection > closed error and doesn't process the exception sent by the server. Client > then thinks it got a network error, which leads to retrying the RPC instead > of opening a new scanner. But then when the client retry reaches the server, > the server returns an empty ScanResponse instead of an error, leading to > closing the scanner on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539] > 2nd call (retry of 1st) returns empty results: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403] > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214] > [https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367] > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28595) Losing exception from scan RPC can lead to partial results
[ https://issues.apache.org/jira/browse/HBASE-28595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846649#comment-17846649 ] Duo Zhang commented on HBASE-28595: --- By design, if the server side has already closed the scanner, the retry RPC should receive a UnknownScannerException. Let me check the code. Thanks for providing this. > Losing exception from scan RPC can lead to partial results > -- > > Key: HBASE-28595 > URL: https://issues.apache.org/jira/browse/HBASE-28595 > Project: HBase > Issue Type: Bug > Components: Client, regionserver, Scanners >Reporter: Csaba Ringhofer >Assignee: Csaba Ringhofer >Priority: Critical > > This was discovered in Apache Impala using HBase 2.2 based branch hbase > client and server. It is not clear yet whether other branches are also > affected. > The issue happens if the server side of the scan throws an exception and > closes the scanner, but the client doesn't get the exact exception and it > treats it as network error, which leads to retrying the RPC instead of > opening a new scanner. In this case the server returns an empty ScanResponse > instead of an error when the RPC is retried, leading to closing the scanner > on client side without returning any error. > A few pointers to critical parts: > region server: > 1st call throws exception leading to closing (but not deleting) scanner: > https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3539 > 2nd call (retry of 1st) returns empty results: > https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L3403 > client: > some exceptions are handled as non-retriable at RPC level and are only > handled through opening a new scanner: > https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java#L214 > https://github.com/apache/hbase/blob/0c8607a35008b7dca15e9daaec41ec362d159d67/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java#L367 > This mechanism in the client only works if it gets the exception from the > server. If there are connection issues during the RPC then the client won't > really know the state of the server. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28471) Release 2.4.18
[ https://issues.apache.org/jira/browse/HBASE-28471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28471: - Assignee: Duo Zhang > Release 2.4.18 > -- > > Key: HBASE-28471 > URL: https://issues.apache.org/jira/browse/HBASE-28471 > Project: HBase > Issue Type: Umbrella > Components: community >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > The 2.6.0 release vote is ongoing, let's release 2.4.18 and mark 2.4.x as EOL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28586) Backport HBASE-24791 Improve HFileOutputFormat2 to avoid always call getTableRelativePath method
[ https://issues.apache.org/jira/browse/HBASE-28586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28586. --- Fix Version/s: 2.4.18 2.7.0 2.6.1 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Pushed to all branch-2.x. Thanks [~szucsvillo]! > Backport HBASE-24791 Improve HFileOutputFormat2 to avoid always call > getTableRelativePath method > > > Key: HBASE-28586 > URL: https://issues.apache.org/jira/browse/HBASE-28586 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0 >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 2.6.1, 2.5.9 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28586) Backport HBASE-24791 Improve HFileOutputFormat2 to avoid always call getTableRelativePath method
[ https://issues.apache.org/jira/browse/HBASE-28586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28586: - Assignee: Szucs Villo > Backport HBASE-24791 Improve HFileOutputFormat2 to avoid always call > getTableRelativePath method > > > Key: HBASE-28586 > URL: https://issues.apache.org/jira/browse/HBASE-28586 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0 >Reporter: Szucs Villo >Assignee: Szucs Villo >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28581) Remove deprecated methods in TableDescriotorBuilder
[ https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28581: -- Fix Version/s: 3.0.0-beta-2 > Remove deprecated methods in TableDescriotorBuilder > --- > > Key: HBASE-28581 > URL: https://issues.apache.org/jira/browse/HBASE-28581 > Project: HBase > Issue Type: Sub-task > Components: API, Client >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28547) Support specifying connection configuration through queries of the connection uri
[ https://issues.apache.org/jira/browse/HBASE-28547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28547: -- Summary: Support specifying connection configuration through queries of the connection uri (was: Support specifying connection registry configuration through queries of the connection uri) > Support specifying connection configuration through queries of the connection > uri > - > > Key: HBASE-28547 > URL: https://issues.apache.org/jira/browse/HBASE-28547 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28581) Remove deprecated methods in TableDescriotorBuilder
[ https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28581. --- Resolution: Fixed Pushed to master and branch-3. Thanks [~heliangjun] for contributing! > Remove deprecated methods in TableDescriotorBuilder > --- > > Key: HBASE-28581 > URL: https://issues.apache.org/jira/browse/HBASE-28581 > Project: HBase > Issue Type: Sub-task > Components: API, Client >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28581) Remove deprecated methods in TableDescriotorBuilder
[ https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28581: -- Hadoop Flags: Incompatible change,Reviewed Release Note: Remove TableDescriptorBuilder.setCoprocessorWithSpec method. > Remove deprecated methods in TableDescriotorBuilder > --- > > Key: HBASE-28581 > URL: https://issues.apache.org/jira/browse/HBASE-28581 > Project: HBase > Issue Type: Sub-task > Components: API, Client >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28576) Remove FirstKeyValueMatchingQualifiersFilter
[ https://issues.apache.org/jira/browse/HBASE-28576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28576: -- Hadoop Flags: Incompatible change,Reviewed Release Note: Remove FirstKeyValueMatchingQualifiersFilter class. > Remove FirstKeyValueMatchingQualifiersFilter > > > Key: HBASE-28576 > URL: https://issues.apache.org/jira/browse/HBASE-28576 > Project: HBase > Issue Type: Sub-task > Components: Filters >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28576) Remove FirstKeyValueMatchingQualifiersFilter
[ https://issues.apache.org/jira/browse/HBASE-28576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28576. --- Fix Version/s: 3.0.0-beta-2 Resolution: Fixed Pushed to master and branch-3. Thanks [~heliangjun] for contributing! > Remove FirstKeyValueMatchingQualifiersFilter > > > Key: HBASE-28576 > URL: https://issues.apache.org/jira/browse/HBASE-28576 > Project: HBase > Issue Type: Sub-task > Components: Filters >Reporter: Duo Zhang >Assignee: Liangjun He >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28586) Backport HBASE-24791 to branch-2.6
[ https://issues.apache.org/jira/browse/HBASE-28586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845232#comment-17845232 ] Duo Zhang commented on HBASE-28586: --- Yes, we should backport to branch-2 too if we want this in branch-2.6. Branch-2.5/2.4 could be considered too but not a must. Thanks. > Backport HBASE-24791 to branch-2.6 > -- > > Key: HBASE-28586 > URL: https://issues.apache.org/jira/browse/HBASE-28586 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.6.0 >Reporter: Szucs Villo >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28588) Remove deprecated methods in WAL
Duo Zhang created HBASE-28588: - Summary: Remove deprecated methods in WAL Key: HBASE-28588 URL: https://issues.apache.org/jira/browse/HBASE-28588 Project: HBase Issue Type: Sub-task Components: wal Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28587) Remove deprecated methods in Cell
Duo Zhang created HBASE-28587: - Summary: Remove deprecated methods in Cell Key: HBASE-28587 URL: https://issues.apache.org/jira/browse/HBASE-28587 Project: HBase Issue Type: Sub-task Components: API, Client Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
[ https://issues.apache.org/jira/browse/HBASE-28579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28579: - Assignee: Duo Zhang > Hide HFileScanner related methods in StoreFileReader > > > Key: HBASE-28579 > URL: https://issues.apache.org/jira/browse/HBASE-28579 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28578) Remove deprecated methods in HFileScanner
[ https://issues.apache.org/jira/browse/HBASE-28578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28578 started by Duo Zhang. - > Remove deprecated methods in HFileScanner > - > > Key: HBASE-28578 > URL: https://issues.apache.org/jira/browse/HBASE-28578 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
[ https://issues.apache.org/jira/browse/HBASE-28579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28579 started by Duo Zhang. - > Hide HFileScanner related methods in StoreFileReader > > > Key: HBASE-28579 > URL: https://issues.apache.org/jira/browse/HBASE-28579 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28578) Remove deprecated methods in HFileScanner
[ https://issues.apache.org/jira/browse/HBASE-28578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28578: - Assignee: Duo Zhang > Remove deprecated methods in HFileScanner > - > > Key: HBASE-28578 > URL: https://issues.apache.org/jira/browse/HBASE-28578 > Project: HBase > Issue Type: Sub-task > Components: HFile, Scanners >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28582) ModifyTableProcedure should not reset TRSP on region node when closing unused region replicas
Duo Zhang created HBASE-28582: - Summary: ModifyTableProcedure should not reset TRSP on region node when closing unused region replicas Key: HBASE-28582 URL: https://issues.apache.org/jira/browse/HBASE-28582 Project: HBase Issue Type: Bug Components: proc-v2 Reporter: Duo Zhang Assignee: Duo Zhang Found this when digging HBASE-28522. First, this is not safe as MTP does not like DTP where we hold the exclusive lock all the time. Second, even if we hold the exclusive lock all the time, as showed in HBASE-28522, we may still hang there forever because SCP will not interrupt the TRSP. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845000#comment-17845000
]
Duo Zhang commented on HBASE-28522:
---
ModifyTableProcedure may have the same problem when reducing region replica
count, and the implementation is totally wrong as ModifyTableProcedure does not
hold exclusive lock all the time...
Let file an issue to fix it too.
The fix is DTP is more difficult, as for DTP, we want it to hold the exclusive
lock all the time so it will not skew with other table procedures...
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2, Region Assignment
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Critical
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28572) Remove deprecated methods in thrift module
[ https://issues.apache.org/jira/browse/HBASE-28572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28572: - Assignee: Duo Zhang > Remove deprecated methods in thrift module > -- > > Key: HBASE-28572 > URL: https://issues.apache.org/jira/browse/HBASE-28572 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28572) Remove deprecated methods in thrift module
[ https://issues.apache.org/jira/browse/HBASE-28572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28572: -- Fix Version/s: 3.0.0-beta-2 > Remove deprecated methods in thrift module > -- > > Key: HBASE-28572 > URL: https://issues.apache.org/jira/browse/HBASE-28572 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-28522:
--
Component/s: Region Assignment
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2, Region Assignment
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Critical
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-28522:
--
Priority: Critical (was: Minor)
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Critical
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844884#comment-17844884
]
Duo Zhang commented on HBASE-28522:
---
Oh, reuse seems impossible, as the old TRSP can not be executed since
DisableTableProcedure will hold the xlock of this table all the time...
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Minor
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844875#comment-17844875
]
Duo Zhang commented on HBASE-28522:
---
After checking the code, I think a possible race is that, in SCP, we have
schedule a TRSP to assign the region, but at the same time,
DisableTableProcedure has unset the procedure and scheduled a new one, but
actually, the new TRSP can not be executed because the target server is already
dead, so the unassign will fail and wait for SCP to interrupte it, but at the
same time, SCP is waiting the old TRSP(scheduled by SCP) to finish, so it has
no change to interrupte the new TRSP(and even if it can execute, in the current
logic it will just finish itself without interrupting any other TRSPs).
I think the root cause is here, in forceCreateUnssignProcedure
{code}
regionNode.lock();
try {
if (regionNode.isInState(State.OFFLINE, State.CLOSED, State.SPLIT)) {
return null;
}
// in general, a split parent should be in CLOSED or SPLIT state, but
anyway, let's check it
// here for safety
if (regionNode.getRegionInfo().isSplit()) {
LOG.warn("{} is a split parent but not in CLOSED or SPLIT state",
regionNode);
return null;
}
// As in DisableTableProcedure or ModifyTableProcedure, we will hold the
xlock for table, so
// we can make sure that this procedure has not been executed yet, as
TRSP will hold the
// shared lock for table all the time. So here we will unset it and when
it is actually
// executed, it will find that the attach procedure is not itself and
quit immediately.
if (regionNode.getProcedure() != null) {
regionNode.unsetProcedure(regionNode.getProcedure());
}
return
regionNode.setProcedure(TransitRegionStateProcedure.unassign(getProcedureEnvironment(),
regionNode.getRegionInfo()));
} finally {
regionNode.unlock();
}
{code}
We should reuse the same TRSP, instead of unsetting it and scheduling a new one.
Let me think if this is possible.
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Minor
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28522) UNASSIGN proc indefinitely stuck on dead rs
[
https://issues.apache.org/jira/browse/HBASE-28522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844874#comment-17844874
]
Duo Zhang commented on HBASE-28522:
---
Because we can not make sure it is closed normally...
The region server may crash before finishing unassign the region right?
> UNASSIGN proc indefinitely stuck on dead rs
> ---
>
> Key: HBASE-28522
> URL: https://issues.apache.org/jira/browse/HBASE-28522
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
>Reporter: Prathyusha
>Assignee: Prathyusha
>Priority: Minor
>
> One scenario we noticed in production -
> we had DisableTableProc and SCP almost triggered at similar time
> 2024-03-16 17:59:23,014 INFO [PEWorker-11] procedure.DisableTableProcedure -
> Set to state=DISABLING
> 2024-03-16 17:59:15,243 INFO [PEWorker-26] procedure.ServerCrashProcedure -
> Start pid=21592440, state=RUNNABLE:SERVER_CRASH_START, locked=true;
> ServerCrashProcedure
> , splitWal=true, meta=false
> DisabeTableProc creates unassign procs, and at this time ASSIGNs of SCP is
> not completed
> {{2024-03-16 17:59:23,003 DEBUG [PEWorker-40] procedure2.ProcedureExecutor -
> LOCK_EVENT_WAIT pid=21594220, ppid=21592440,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE;
> TransitRegionStateProcedure table=, region=, ASSIGN}}
> UNASSIGN created by DisableTableProc is stuck on the dead regionserver and we
> had to manually bypass unassign of DisableTableProc and then do ASSIGN.
> If we can break the loop for UNASSIGN procedure to not retry if there is scp
> for that server, we do not need manual intervention?, at least the
> DisableTableProc can go to a rollback state?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28581) Remove deprecated methods in TableDescriptor
Duo Zhang created HBASE-28581: - Summary: Remove deprecated methods in TableDescriptor Key: HBASE-28581 URL: https://issues.apache.org/jira/browse/HBASE-28581 Project: HBase Issue Type: Sub-task Components: API, Client Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28580) Remove deprecated methods in WALObserver
Duo Zhang created HBASE-28580: - Summary: Remove deprecated methods in WALObserver Key: HBASE-28580 URL: https://issues.apache.org/jira/browse/HBASE-28580 Project: HBase Issue Type: Sub-task Components: Coprocessors, wal Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28579) Hide HFileScanner related methods in StoreFileReader
Duo Zhang created HBASE-28579: - Summary: Hide HFileScanner related methods in StoreFileReader Key: HBASE-28579 URL: https://issues.apache.org/jira/browse/HBASE-28579 Project: HBase Issue Type: Sub-task Components: HFile, Scanners Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
