[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[
https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791010#comment-17791010
]
Zhuoyue Huang commented on HBASE-25549:
---
{quote}I wonder if https://issues.apache.org/jira/browse/HBASE-28215 is better
for handling RIT storms, since the hmaster procedure framework already has
error handling and state recovery built in, just not throttling (yet).
{quote}
I agree that HBASE-28215 is another good way to handle RIT storms.
In our scenario, we once faced a critical bug that occurred when we added a
feature's configuration to a table, causing regions can never leave the
'opening' state. In this case, all regions are 'dead' finally. So even if
HMaster could throttle the speed of this progress, we still need a way to
carefully alter a table. e.g, altering a table but not reopening regions, so
that we can check if our altering is safe enough to go forward. Otherwise, we
can roll back and affect only one region.
> Provide a switch that allows avoiding reopening all regions when modifying a
> table to prevent RIT storms.
> -
>
> Key: HBASE-25549
> URL: https://issues.apache.org/jira/browse/HBASE-25549
> Project: HBase
> Issue Type: Improvement
> Components: master, shell
>Affects Versions: 3.0.0-alpha-1
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7
>
>
> Under normal circumstances, modifying a table will cause all regions
> belonging to the table to enter RIT. Imagine the following two scenarios：
> # Someone entered the wrong configuration （e.g. negative
> 'hbase.busy.wait.multiplier.max' value） when altering the table, causing
> thousands of online regions to fail to open, leading to online accidents.
> # Modify the configuration of a table, but this modification is not urgent,
> the regions are not expected to enter RIT immediately.
> -'alter_lazy' is a new command to modify a table without reopening any online
> regions except those regions were assigned by other threads or split etc.-
>
> Provide an optional lazy_mode for the alter command to modify the
> TableDescriptor without the region entering the RIT. The modification will
> take effect when the region is reopened.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[
https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790990#comment-17790990
]
Zhuoyue Huang commented on HBASE-25549:
---
{quote}I'm curious how you plan to reopen regions with this feature. I don't
think we have a user API for reopening regions. You can unassign and then
assign, but that leaves you at risk of handling errors and recovery in your
automation.
{quote}
I have implemented an API(including hbase-shell command) that can reopen a
specific region on the HBase version I am using. This API allows the region to
be reopened in place without changing the location of the region(to prevent the
region from being assigned to a new RS and affecting its data locality).
I will submit this feature later.
> Provide a switch that allows avoiding reopening all regions when modifying a
> table to prevent RIT storms.
> -
>
> Key: HBASE-25549
> URL: https://issues.apache.org/jira/browse/HBASE-25549
> Project: HBase
> Issue Type: Improvement
> Components: master, shell
>Affects Versions: 3.0.0-alpha-1
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7
>
>
> Under normal circumstances, modifying a table will cause all regions
> belonging to the table to enter RIT. Imagine the following two scenarios：
> # Someone entered the wrong configuration （e.g. negative
> 'hbase.busy.wait.multiplier.max' value） when altering the table, causing
> thousands of online regions to fail to open, leading to online accidents.
> # Modify the configuration of a table, but this modification is not urgent,
> the regions are not expected to enter RIT immediately.
> -'alter_lazy' is a new command to modify a table without reopening any online
> regions except those regions were assigned by other threads or split etc.-
>
> Provide an optional lazy_mode for the alter command to modify the
> TableDescriptor without the region entering the RIT. The modification will
> take effect when the region is reopened.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788673#comment-17788673 ] Zhuoyue Huang commented on HBASE-28120: --- [~gourab.taparia] Are you still working on it? As [~bbeaudreault] said, maybe you can do a back port for branch-2 in the jira of HBASE-25549. > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784264#comment-17784264 ] Zhuoyue Huang commented on HBASE-28120: --- The commit has been merged into the master branch. You can continue your work to backport to branch-2 when you have time.[~gourab.taparia] > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784263#comment-17784263 ] Zhuoyue Huang commented on HBASE-25549: --- Thanks [~bbeaudreault] ,[~wchevreuil] , [~vjasani] ,[~bharathv] , [~esteban] , [~busbey] ,[~gourab.taparia] for reviewing and advising. > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784262#comment-17784262 ] Zhuoyue Huang commented on HBASE-25549: --- The current commit [GitHub Pull Request #2924|https://github.com/apache/hbase/pull/2924] has been merged into the master branch. > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781580#comment-17781580 ] Zhuoyue Huang commented on HBASE-28120: --- Thank you for your work. Could we possibly wait until HBASE-25549 is first merged into the master, and then you can proceed with the follow-up work by cherry-pick it into HBase2? Currently, there are some new changes in HBASE-25549 waiting for review. > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772946#comment-17772946 ] Zhuoyue Huang commented on HBASE-25549: --- Hi everyone, I've completed the patch and all the tests have passed successfully. My commit is ready for review. I would appreciate it if you could take a look and provide your feedback. > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-25549: -- Status: Patch Available (was: In Progress) > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28121) Port the switch to avoid reopening regions in the alter async in HBase 2
[ https://issues.apache.org/jira/browse/HBASE-28121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-28121: - Assignee: Gourab Taparia > Port the switch to avoid reopening regions in the alter async in HBase 2 > > > Key: HBASE-28121 > URL: https://issues.apache.org/jira/browse/HBASE-28121 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to port the feature added in HBase 3 alter(async default) layer to > HBase 2's async side. > There is a separate sub-task for adding it to HBase 2's sync side. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772728#comment-17772728 ] Zhuoyue Huang commented on HBASE-28120: --- [~gourab.taparia] Done! > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-28120: - Assignee: Gourab Taparia > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771697#comment-17771697 ] Zhuoyue Huang commented on HBASE-25549: --- I agree with [~gourab.taparia] 's point of view, I will adjust the code later. I also agree with [~bbeaudreault] regarding not reopening regions by default for specific configurations. I hope this can be addressed in future issues.;) > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769104#comment-17769104 ] Zhuoyue Huang commented on HBASE-25549: --- Hi [~esteban] , [~bharathv] , [~vjasani] ,[~gourab.taparia] . I've commited the newly PR and would greatly appreciate your review and guidance when you have the time. Thank you in advance! > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769100#comment-17769100 ] Zhuoyue Huang commented on HBASE-25549: --- To avoid potential issues, the current version does not allow modifications without reopening regions for the following: 1. Changing region replicas is not allowed. 2. Modifying coprocessors is not allowed. 3. Adding or removing Column Families (CF) is not allowed. 4. Modifying CF properties is not allowed. > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769099#comment-17769099 ] Zhuoyue Huang commented on HBASE-25549: --- I've made some adjustments for this issue: 1. Remove the concept of "lazy mode," as I believe it could confuse developers. So, I've replaced all instances of the boolean variable "lazy_mode" in my code with "reopenRegions," which is more straightforward. 2. By default, any table modification operation will set "reopenRegions" to true. 3. If you modify a table using a shell command like "alter 't1', METHOD=>'no_reopen_regions'," "reopenRegions" will be set to false, and the table modification operation will not reopen any regions. 4. I've changed the title of this issue from "Provide lazy mode when modifying table to avoid RIT storm" to "Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms" > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-25549) Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms.
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-25549: -- Summary: Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms. (was: Provide lazy mode when modifying table to avoid RIT storm) > Provide a switch that allows avoiding reopening all regions when modifying a > table to prevent RIT storms. > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide lazy mode when modifying table to avoid RIT storm
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769094#comment-17769094 ] Zhuoyue Huang commented on HBASE-25549: --- [~gourab.taparia] Yes, I've already started. I should have checked my email earlier and seen your reply. I'll submit the PR for conflict resolution later today. > Provide lazy mode when modifying table to avoid RIT storm > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-25549) Provide lazy mode when modifying table to avoid RIT storm
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-25549: - Assignee: Zhuoyue Huang > Provide lazy mode when modifying table to avoid RIT storm > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-25549) Provide lazy mode when modifying table to avoid RIT storm
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25549 started by Zhuoyue Huang. - > Provide lazy mode when modifying table to avoid RIT storm > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25549) Provide lazy mode when modifying table to avoid RIT storm
[ https://issues.apache.org/jira/browse/HBASE-25549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766619#comment-17766619 ] Zhuoyue Huang commented on HBASE-25549: --- [~vjasani] Sorry for the delay in this work. I'm now resuming the task, and I'll have the patch ready soon. > Provide lazy mode when modifying table to avoid RIT storm > - > > Key: HBASE-25549 > URL: https://issues.apache.org/jira/browse/HBASE-25549 > Project: HBase > Issue Type: Improvement > Components: master, shell >Affects Versions: 3.0.0-alpha-1 >Reporter: Zhuoyue Huang >Priority: Major > Fix For: 2.6.0, 2.5.6, 3.0.0-beta-1 > > > Under normal circumstances, modifying a table will cause all regions > belonging to the table to enter RIT. Imagine the following two scenarios： > # Someone entered the wrong configuration （e.g. negative > 'hbase.busy.wait.multiplier.max' value） when altering the table, causing > thousands of online regions to fail to open, leading to online accidents. > # Modify the configuration of a table, but this modification is not urgent, > the regions are not expected to enter RIT immediately. > -'alter_lazy' is a new command to modify a table without reopening any online > regions except those regions were assigned by other threads or split etc.- > > Provide an optional lazy_mode for the alter command to modify the > TableDescriptor without the region entering the RIT. The modification will > take effect when the region is reopened. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27974) CompactionServer cause the loss of HFile references in snapshot
[
https://issues.apache.org/jira/browse/HBASE-27974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-27974:
--
Description:
We encountered an issue with the loss of HFile references in the snapshot after
enabling the CompactionServer feature in our cluster.
The relevant log fragment is approximately as follows:
{code:java}
File does not exist:
/.../.tmp/data/default//b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../data/default/.../b72cab4efb074defb1bd9acd9087891f {code}
>From the displayed HDFS logs, we observed that this HFile
>'b72cab4efb074defb1bd9acd9087891f' was renamed by the CompactionServer and
>RegionServer, and eventually deleted by the HMaster.
{code:java}
2022-07-13,00:50:01,727 INFO FSNamesystem.audit:
cmd=rename operator=CompactionServer
src=/./data/default/./b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
2022-07-13,00:51:23,802 INFO FSNamesystem.audit:
cmd=rename
operator=RegionServer
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
2022-07-13,01:51:57,823 INFO FSNamesystem.audit:
cmd=delete
operator=HMaster
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
{code}
Based on HBASE-26722 and HBASE-22163, we understand that if a region A on RS1
is not closed, and another RS2 (in this case, CompactionServer) opens the same
region A, it may trigger an "archived" state. Consequently, when RS1 closes
this region, it will be archived again, resulting in the deletion of the HFile
from the archived directory. As a result, the snapshot will lose its reference
to the HFile.
was:
We encountered an issue with the loss of HFile references in the snapshot after
enabling the CompactionServer feature in our cluster.
The relevant log fragment is approximately as follows:
{code:java}
File does not exist:
/.../.tmp/data/default//b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../data/default/.../b72cab4efb074defb1bd9acd9087891f {code}
>From the displayed HDFS logs, we observed that this HFile
>'b72cab4efb074defb1bd9acd9087891f' was renamed by the CompactionServer and
>RegionServer, and eventually deleted by the HMaster.
{code:java}
2022-07-13,00:50:01,727 INFO FSNamesystem.audit:
cmd=rename operator=CompactionServer
src=/./data/default/./b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
2022-07-13,00:51:23,802 INFO FSNamesystem.audit:
cmd=rename
operator=RegionServer
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
2022-07-13,01:51:57,823 INFO FSNamesystem.audit:
cmd=delete
operator=HMaster
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
{code}
> CompactionServer cause the loss of HFile references in snapshot
> ---
>
> Key: HBASE-27974
> URL: https://issues.apache.org/jira/browse/HBASE-27974
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
>
> We encountered an issue with the loss of HFile references in the snapshot
> after enabling the CompactionServer feature in our cluster.
> The relevant log fragment is approximately as follows:
> {code:java}
> File does not exist:
> /.../.tmp/data/default//b72cab4efb074defb1bd9acd9087891f
> File does not exist:
> /../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> File does not exist:
> /../data/default/.../b72cab4efb074defb1bd9acd9087891f {code}
> From the displayed HDFS logs, we observed that this HFile
> 'b72cab4efb074defb1bd9acd9087891f' was renamed by the CompactionServer and
> RegionServer, and eventually deleted by the HMaster.
> {code:java}
> 2022-07-13,00:50:01,727 INFO FSNamesystem.audit:
> cmd=rename operator=CompactionServer
> src=/./data/default/./b72cab4efb074defb1bd9acd9087891f
> dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> 2022-07-13,00:51:23,802 INFO FSNamesystem.audit:
> cmd=rename
> operator=RegionServer
> src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
>
> 2022-07-13,01:51:57,823 INFO FSNamesystem.audit:
>
[jira] [Updated] (HBASE-27974) CompactionServer cause the loss of HFile references in snapshot
[
https://issues.apache.org/jira/browse/HBASE-27974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-27974:
--
Description:
We encountered an issue with the loss of HFile references in the snapshot after
enabling the CompactionServer feature in our cluster.
The relevant log fragment is approximately as follows:
{code:java}
File does not exist:
/.../.tmp/data/default//b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
File does not exist:
/../data/default/.../b72cab4efb074defb1bd9acd9087891f {code}
>From the displayed HDFS logs, we observed that this HFile
>'b72cab4efb074defb1bd9acd9087891f' was renamed by the CompactionServer and
>RegionServer, and eventually deleted by the HMaster.
{code:java}
2022-07-13,00:50:01,727 INFO FSNamesystem.audit:
cmd=rename operator=CompactionServer
src=/./data/default/./b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
2022-07-13,00:51:23,802 INFO FSNamesystem.audit:
cmd=rename
operator=RegionServer
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
2022-07-13,01:51:57,823 INFO FSNamesystem.audit:
cmd=delete
operator=HMaster
src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
{code}
> CompactionServer cause the loss of HFile references in snapshot
> ---
>
> Key: HBASE-27974
> URL: https://issues.apache.org/jira/browse/HBASE-27974
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
>
> We encountered an issue with the loss of HFile references in the snapshot
> after enabling the CompactionServer feature in our cluster.
> The relevant log fragment is approximately as follows:
> {code:java}
> File does not exist:
> /.../.tmp/data/default//b72cab4efb074defb1bd9acd9087891f
> File does not exist:
> /../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> File does not exist:
> /../data/default/.../b72cab4efb074defb1bd9acd9087891f {code}
> From the displayed HDFS logs, we observed that this HFile
> 'b72cab4efb074defb1bd9acd9087891f' was renamed by the CompactionServer and
> RegionServer, and eventually deleted by the HMaster.
> {code:java}
> 2022-07-13,00:50:01,727 INFO FSNamesystem.audit:
> cmd=rename operator=CompactionServer
> src=/./data/default/./b72cab4efb074defb1bd9acd9087891f
> dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> 2022-07-13,00:51:23,802 INFO FSNamesystem.audit:
> cmd=rename
> operator=RegionServer
> src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f
> dst=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
>
> 2022-07-13,01:51:57,823 INFO FSNamesystem.audit:
> cmd=delete
> operator=HMaster
> src=/../archive/data/default/../b72cab4efb074defb1bd9acd9087891f.1657644683801
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-27974) CompactionServer cause the loss of HFile references in snapshot
Zhuoyue Huang created HBASE-27974: - Summary: CompactionServer cause the loss of HFile references in snapshot Key: HBASE-27974 URL: https://issues.apache.org/jira/browse/HBASE-27974 Project: HBase Issue Type: Sub-task Reporter: Zhuoyue Huang Assignee: Zhuoyue Huang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27184) Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2
[ https://issues.apache.org/jira/browse/HBASE-27184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564180#comment-17564180 ] Zhuoyue Huang commented on HBASE-27184: --- [~yangzhichao] You should submit this issue again in English so that developers from all over the world can read your description. You also need to add more information such as hbase version you are using and the steps to reproduce the problem. (您应该用英文再次提交此问题，以便来自世界各地的开发人员可以阅读您的描述。 您还需要添加更多信息，例如 hbase 版本和重现问题的步骤。) > Hbase使用get查询的时候，部分rowkey查不到数据，ERROR: java.io.TOException: > java.lang.NoclassDefFoundError: org/apache/hadoop/hdfs/DFSInputStreamS2 > - > > Key: HBASE-27184 > URL: https://issues.apache.org/jira/browse/HBASE-27184 > Project: HBase > Issue Type: Bug >Reporter: yangzhichao >Priority: Major > Attachments: image-2022-07-08-16-59-52-808.png > > > !image-2022-07-08-16-59-52-808.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-26877) TestRegisterPeerWorkerWhenRestarting fails with IllegalStateException
[
https://issues.apache.org/jira/browse/HBASE-26877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511551#comment-17511551
]
Zhuoyue Huang commented on HBASE-26877:
---
I tried to run this UT many times on my local machine, can't reproduce it.
> TestRegisterPeerWorkerWhenRestarting fails with IllegalStateException
> -
>
> Key: HBASE-26877
> URL: https://issues.apache.org/jira/browse/HBASE-26877
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Duo Zhang
>Priority: Major
> Attachments:
> org.apache.hadoop.hbase.master.replication.TestRegisterPeerWorkerWhenRestarting-output.txt
>
>
> {noformat}
> 2022-03-21T12:25:09,067 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.ReplicationSource(434): Unexpected exception in
> RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:553)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:834) ~[?:?]
> 2022-03-21T12:25:09,068 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.HRegionServer(2284): * ABORTING region server
> b0aef29cdb13,33413,1647865503466: Unexpected exception in
> RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1 *
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:553)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:834) ~[?:?]
> 2022-03-21T12:25:09,069 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.HRegionServer(2291): RegionServer abort: loaded coprocessors
> are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2022-03-21T12:25:09,072 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.ReplicationSource(434): Unexpected exception in
> RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:553)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:834) ~[?:?]
> 2022-03-21T12:25:09,072 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.HRegionServer(2284): * ABORTING region server
> b0aef29cdb13,41089,1647865503422: Unexpected exception in
> RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1 *
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:553)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:834) ~[?:?]
> 2022-03-21T12:25:09,072 ERROR
> [RS_REFRESH_PEER-regionserver/b0aef29cdb13:0-0.replicationSource,1]
> regionserver.HRegionServer(2291): RegionServer abort: loaded coprocessors
> are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26742) Comparator of NOT_EQUAL NULL is invalid for checkAndMutate
[
https://issues.apache.org/jira/browse/HBASE-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489354#comment-17489354
]
Zhuoyue Huang commented on HBASE-26742:
---
{quote}1. result is null, operator is NOT_EQUAL, but matches is true;
{quote}
I don't think you were trying to explain 'result is null' here. 'result'
couldn't be null because we have initialized it in advance.
{code:java}
List result = new ArrayList<>(1);{code}
Did you mean to say 'result is empty'?
[~Xiaolin Ha]
> Comparator of NOT_EQUAL NULL is invalid for checkAndMutate
> --
>
> Key: HBASE-26742
> URL: https://issues.apache.org/jira/browse/HBASE-26742
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> In server side, checkAndMutate ignores CompareOperator for null or empty
> comparator value, but NOT_EQUAL should be treated specially.
> The check logic in HRegion#checkAndMutateInternal is as follows,
> {code:java}
> boolean valueIsNull =
> comparator.getValue() == null || comparator.getValue().length == 0;
> if (result.isEmpty() && valueIsNull) {
> matches = true;
> } else if (result.size() > 0 && result.get(0).getValueLength() == 0 &&
> valueIsNull) {
> matches = true;
> cellTs = result.get(0).getTimestamp();
> } else if (result.size() == 1 && !valueIsNull) {
> Cell kv = result.get(0);
> cellTs = kv.getTimestamp();
> int compareResult = PrivateCellUtil.compareValue(kv, comparator);
> matches = matches(op, compareResult);
> }{code}
> For current logics, here are some counter examples(Comparator value is set
> null),
> # result is null, operator is NOT_EQUAL, but matches is true;
> # result size >0, the value of the first cell is empty, operator is
> NOT_EQUAL, but matches is true;
> # result size is 1, operator is NOT_EQUAL, but matches is false;
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-26692) [HBase-Client] In some cases, no value is obtained using method Result.getValue()
[ https://issues.apache.org/jira/browse/HBASE-26692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26692: - Assignee: wenhao (was: Zhuoyue Huang) > [HBase-Client] In some cases, no value is obtained using method > Result.getValue() > - > > Key: HBASE-26692 > URL: https://issues.apache.org/jira/browse/HBASE-26692 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: wenhao >Assignee: wenhao >Priority: Critical > Attachments: 1.png, 2.png, Test.java > > > Results are not sorted during the query by Result.getValue(). As a result, > existing values cannot be queried. > In normal cases, the results by get or scan are sorted and this problem does > not occur. However, in some cases, such as when the HFile file is read by MR > and the cells obtained are combined as result by rowkey, there is no sorting. > Result.getvalue (byte [] family, byte [] qualifier) is null. > > * The test results are shown in the figures > !1.png!!2.png![^Test.java] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26692) [HBase-Client] In some cases, no value is obtained using method Result.getValue()
[ https://issues.apache.org/jira/browse/HBASE-26692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26692: - Assignee: Zhuoyue Huang (was: wenhao) > [HBase-Client] In some cases, no value is obtained using method > Result.getValue() > - > > Key: HBASE-26692 > URL: https://issues.apache.org/jira/browse/HBASE-26692 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0 >Reporter: wenhao >Assignee: Zhuoyue Huang >Priority: Critical > Attachments: 1.png, 2.png, Test.java > > > Results are not sorted during the query by Result.getValue(). As a result, > existing values cannot be queried. > In normal cases, the results by get or scan are sorted and this problem does > not occur. However, in some cases, such as when the HFile file is read by MR > and the cells obtained are combined as result by rowkey, there is no sorting. > Result.getvalue (byte [] family, byte [] qualifier) is null. > > * The test results are shown in the figures > !1.png!!2.png![^Test.java] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-24749) Direct insert HFiles and Persist in-memory HFile tracking
[
https://issues.apache.org/jira/browse/HBASE-24749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang reassigned HBASE-24749:
-
Assignee: Tak-Lon (Stephen) Wu
> Direct insert HFiles and Persist in-memory HFile tracking
> -
>
> Key: HBASE-24749
> URL: https://issues.apache.org/jira/browse/HBASE-24749
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, HFile
>Affects Versions: 3.0.0-alpha-1
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Tak-Lon (Stephen) Wu
>Priority: Major
> Labels: design, discussion, objectstore, storeFile, storeengine
> Attachments: 1B100m-25m25m-performance.pdf, Apache HBase - Direct
> insert HFiles and Persist in-memory HFile tracking.pdf
>
>
> We propose a new feature (a new store engine) to remove the {{.tmp}}
> directory used in the commit stage for common HFile operations such as flush
> and compaction to improve the write throughput and latency on object stores.
> Specifically for S3 filesystems, this will also mitigate read-after-write
> inconsistencies caused by immediate HFiles validation after moving the
> HFile(s) to data directory.
> Please see attached for this proposal and the initial result captured with
> 25m (25m operations) and 1B (100m operations) YCSB workload A LOAD and RUN,
> and workload C RUN result.
> The goal of this JIRA is to discuss with the community if the proposed
> improvement on the object stores use case makes senses and if we miss
> anything should be included.
> Improvement Highlights
> 1. Lower write latency, especially the p99+
> 2. Higher write throughput on flush and compaction
> 3. Lower MTTR on region (re)open or assignment
> 4. Remove consistent check dependencies (e.g. DynamoDB) supported by file
> system implementation
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-24749) Direct insert HFiles and Persist in-memory HFile tracking
[
https://issues.apache.org/jira/browse/HBASE-24749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang reassigned HBASE-24749:
-
Assignee: (was: Zhuoyue Huang)
> Direct insert HFiles and Persist in-memory HFile tracking
> -
>
> Key: HBASE-24749
> URL: https://issues.apache.org/jira/browse/HBASE-24749
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, HFile
>Affects Versions: 3.0.0-alpha-1
>Reporter: Tak-Lon (Stephen) Wu
>Priority: Major
> Labels: design, discussion, objectstore, storeFile, storeengine
> Attachments: 1B100m-25m25m-performance.pdf, Apache HBase - Direct
> insert HFiles and Persist in-memory HFile tracking.pdf
>
>
> We propose a new feature (a new store engine) to remove the {{.tmp}}
> directory used in the commit stage for common HFile operations such as flush
> and compaction to improve the write throughput and latency on object stores.
> Specifically for S3 filesystems, this will also mitigate read-after-write
> inconsistencies caused by immediate HFiles validation after moving the
> HFile(s) to data directory.
> Please see attached for this proposal and the initial result captured with
> 25m (25m operations) and 1B (100m operations) YCSB workload A LOAD and RUN,
> and workload C RUN result.
> The goal of this JIRA is to discuss with the community if the proposed
> improvement on the object stores use case makes senses and if we miss
> anything should be included.
> Improvement Highlights
> 1. Lower write latency, especially the p99+
> 2. Higher write throughput on flush and compaction
> 3. Lower MTTR on region (re)open or assignment
> 4. Remove consistent check dependencies (e.g. DynamoDB) supported by file
> system implementation
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-24749) Direct insert HFiles and Persist in-memory HFile tracking
[
https://issues.apache.org/jira/browse/HBASE-24749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang reassigned HBASE-24749:
-
Assignee: Zhuoyue Huang (was: Tak-Lon (Stephen) Wu)
> Direct insert HFiles and Persist in-memory HFile tracking
> -
>
> Key: HBASE-24749
> URL: https://issues.apache.org/jira/browse/HBASE-24749
> Project: HBase
> Issue Type: Umbrella
> Components: Compaction, HFile
>Affects Versions: 3.0.0-alpha-1
>Reporter: Tak-Lon (Stephen) Wu
>Assignee: Zhuoyue Huang
>Priority: Major
> Labels: design, discussion, objectstore, storeFile, storeengine
> Attachments: 1B100m-25m25m-performance.pdf, Apache HBase - Direct
> insert HFiles and Persist in-memory HFile tracking.pdf
>
>
> We propose a new feature (a new store engine) to remove the {{.tmp}}
> directory used in the commit stage for common HFile operations such as flush
> and compaction to improve the write throughput and latency on object stores.
> Specifically for S3 filesystems, this will also mitigate read-after-write
> inconsistencies caused by immediate HFiles validation after moving the
> HFile(s) to data directory.
> Please see attached for this proposal and the initial result captured with
> 25m (25m operations) and 1B (100m operations) YCSB workload A LOAD and RUN,
> and workload C RUN result.
> The goal of this JIRA is to discuss with the community if the proposed
> improvement on the object stores use case makes senses and if we miss
> anything should be included.
> Improvement Highlights
> 1. Lower write latency, especially the p99+
> 2. Higher write throughput on flush and compaction
> 3. Lower MTTR on region (re)open or assignment
> 4. Remove consistent check dependencies (e.g. DynamoDB) supported by file
> system implementation
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work started] (HBASE-26505) [hbase-thirdparty] Bump commons-cli version to 1.5.0
[ https://issues.apache.org/jira/browse/HBASE-26505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26505 started by Zhuoyue Huang. - > [hbase-thirdparty] Bump commons-cli version to 1.5.0 > > > Key: HBASE-26505 > URL: https://issues.apache.org/jira/browse/HBASE-26505 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HBASE-26503) [hbase-thirdparty] Bump guava version to 31.0.1-jre
[ https://issues.apache.org/jira/browse/HBASE-26503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26503 started by Zhuoyue Huang. - > [hbase-thirdparty] Bump guava version to 31.0.1-jre > --- > > Key: HBASE-26503 > URL: https://issues.apache.org/jira/browse/HBASE-26503 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26503) [hbase-thirdparty] Bump guava version to 31.0.1-jre
[ https://issues.apache.org/jira/browse/HBASE-26503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26503: - Assignee: Zhuoyue Huang > [hbase-thirdparty] Bump guava version to 31.0.1-jre > --- > > Key: HBASE-26503 > URL: https://issues.apache.org/jira/browse/HBASE-26503 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HBASE-26506) [hbase-thirdparty] Bump jersey version to 2.35
[ https://issues.apache.org/jira/browse/HBASE-26506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26506 started by Zhuoyue Huang. - > [hbase-thirdparty] Bump jersey version to 2.35 > -- > > Key: HBASE-26506 > URL: https://issues.apache.org/jira/browse/HBASE-26506 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26506) [hbase-thirdparty] Bump jersey version to 2.35
[ https://issues.apache.org/jira/browse/HBASE-26506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26506: - Assignee: Zhuoyue Huang > [hbase-thirdparty] Bump jersey version to 2.35 > -- > > Key: HBASE-26506 > URL: https://issues.apache.org/jira/browse/HBASE-26506 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26505) [hbase-thirdparty] Bump commons-cli version to 1.5.0
[ https://issues.apache.org/jira/browse/HBASE-26505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26505: - Assignee: Zhuoyue Huang > [hbase-thirdparty] Bump commons-cli version to 1.5.0 > > > Key: HBASE-26505 > URL: https://issues.apache.org/jira/browse/HBASE-26505 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (HBASE-26450) Server configuration will overwrite HStore configuration after using shell command 'update_config'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang resolved HBASE-26450. --- Resolution: Fixed > Server configuration will overwrite HStore configuration after using shell > command 'update_config' > -- > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26450) Server configuration will overwrite HStore configuration after using shell command 'update_config'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442565#comment-17442565 ] Zhuoyue Huang commented on HBASE-26450: --- Merged to master and branch-2. [~zhangduo] [~zhaobaiqiang] Thanks for reviewing! > Server configuration will overwrite HStore configuration after using shell > command 'update_config' > -- > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26450) Server configuration will overwrite HStore configuration after using shell command 'update_config'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26450: -- Summary: Server configuration will overwrite HStore configuration after using shell command 'update_config' (was: Server configuration will overwrite HStore configuration after using shell command 'updateConfiguration') > Server configuration will overwrite HStore configuration after using shell > command 'update_config' > -- > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26450) Server configuration will overwrite HStore configuration after using shell command 'updateConfiguration'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26450: -- Summary: Server configuration will overwrite HStore configuration after using shell command 'updateConfiguration' (was: Server configuration will overwrite HStore configuration After using shell command 'updateConfiguration') > Server configuration will overwrite HStore configuration after using shell > command 'updateConfiguration' > > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26450) Server configuration will overwrite HStore configuration After using shell command 'updateConfiguration'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26450: - Assignee: Zhuoyue Huang > Server configuration will overwrite HStore configuration After using shell > command 'updateConfiguration' > > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26450) Server configuration will overwrite HStore configuration After using shell command 'updateConfiguration'
[ https://issues.apache.org/jira/browse/HBASE-26450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26450: -- Summary: Server configuration will overwrite HStore configuration After using shell command 'updateConfiguration' (was: Server configuration will overwrite HStore configuration After using shell command'updateConfiguration') > Server configuration will overwrite HStore configuration After using shell > command 'updateConfiguration' > > > Key: HBASE-26450 > URL: https://issues.apache.org/jira/browse/HBASE-26450 > Project: HBase > Issue Type: Bug > Components: Compaction, conf, regionserver >Affects Versions: 2.5.0, 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Priority: Minor > Fix For: 2.5.0, 3.0.0-alpha-2 > > > When we tried to update the configuration of online servers with shell > commands, we found that the configuration of the server would overwrite the > configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26450) Server configuration will overwrite HStore configuration After using shell command'updateConfiguration'
Zhuoyue Huang created HBASE-26450: - Summary: Server configuration will overwrite HStore configuration After using shell command'updateConfiguration' Key: HBASE-26450 URL: https://issues.apache.org/jira/browse/HBASE-26450 Project: HBase Issue Type: Bug Components: Compaction, conf, regionserver Affects Versions: 2.5.0, 3.0.0-alpha-2 Reporter: Zhuoyue Huang Fix For: 2.5.0, 3.0.0-alpha-2 When we tried to update the configuration of online servers with shell commands, we found that the configuration of the server would overwrite the configuration that originally belonged to HStore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26396) Remove duplicate thread creation during migrating rsgroup
[ https://issues.apache.org/jira/browse/HBASE-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435176#comment-17435176 ] Zhuoyue Huang commented on HBASE-26396: --- Thanks for reviewing! [~zhangduo] > Remove duplicate thread creation during migrating rsgroup > - > > Key: HBASE-26396 > URL: https://issues.apache.org/jira/browse/HBASE-26396 > Project: HBase > Issue Type: Bug > Components: master, rsgroup >Affects Versions: 3.0.0-alpha-2 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Minor > Fix For: 3.0.0-alpha-2 > > > There is a thread that migrate the table rs group info from RSGroupInfo into > the table descriptor. > The thread is created when RSGroupManager is initialized and is created again > when RSGroupStartupWorker is started. > I think this is a bug. Since this thread will not exit until all table > rsgroups are migrated, there is no reason to need two threads to run together? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26396) Remove duplicate thread creation during migrating rsgroup
Zhuoyue Huang created HBASE-26396: - Summary: Remove duplicate thread creation during migrating rsgroup Key: HBASE-26396 URL: https://issues.apache.org/jira/browse/HBASE-26396 Project: HBase Issue Type: Bug Components: master, rsgroup Affects Versions: 3.0.0-alpha-2 Reporter: Zhuoyue Huang Assignee: Zhuoyue Huang Fix For: 3.0.0-alpha-2 There is a thread that migrate the table rs group info from RSGroupInfo into the table descriptor. The thread is created when RSGroupManager is initialized and is created again when RSGroupStartupWorker is started. I think this is a bug. Since this thread will not exit until all table rsgroups are migrated, there is no reason to need two threads to run together? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26251) StochasticLoadBalancer metrics should update even if balancer doesn't run
[ https://issues.apache.org/jira/browse/HBASE-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421992#comment-17421992 ] Zhuoyue Huang commented on HBASE-26251: --- [~bbeaudreault] [~zhangduo] Thanks for reviewing! > StochasticLoadBalancer metrics should update even if balancer doesn't run > - > > Key: HBASE-26251 > URL: https://issues.apache.org/jira/browse/HBASE-26251 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Zhuoyue Huang >Priority: Minor > > Currently we only update StochasticLoadBalancerMetrics at the very end of a > balancer run, once a plan has been found. In fact, we update the metrics > based on that plan even if we don't end up executing the plan, which seems > incorrect. > Regardless of whether the balancer decides to run or not, cluster costs are > changing all the time. Since we don't update these metrics any other time, > operators miss out on important information about the balance of their > cluster over time. > I briefly looked into it and it would be relatively trivial to add another > call to updateStochasticCosts at the beginning of the balanceTable method, > before we determine if the cluster is in need of balancing. This would be an > improvement but would still miss cases where the balancer is disabled or > unable to run due to regions in transition, etc. > It would be good if we could make it so updateStochasticCosts get called > periodically regardless of whether the balancer is enabled or can run. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26251) StochasticLoadBalancer metrics should update even if balancer doesn't run
[ https://issues.apache.org/jira/browse/HBASE-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421900#comment-17421900 ] Zhuoyue Huang commented on HBASE-26251: --- No problem! > StochasticLoadBalancer metrics should update even if balancer doesn't run > - > > Key: HBASE-26251 > URL: https://issues.apache.org/jira/browse/HBASE-26251 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Zhuoyue Huang >Priority: Minor > > Currently we only update StochasticLoadBalancerMetrics at the very end of a > balancer run, once a plan has been found. In fact, we update the metrics > based on that plan even if we don't end up executing the plan, which seems > incorrect. > Regardless of whether the balancer decides to run or not, cluster costs are > changing all the time. Since we don't update these metrics any other time, > operators miss out on important information about the balance of their > cluster over time. > I briefly looked into it and it would be relatively trivial to add another > call to updateStochasticCosts at the beginning of the balanceTable method, > before we determine if the cluster is in need of balancing. This would be an > improvement but would still miss cases where the balancer is disabled or > unable to run due to regions in transition, etc. > It would be good if we could make it so updateStochasticCosts get called > periodically regardless of whether the balancer is enabled or can run. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420455#comment-17420455 ] Zhuoyue Huang commented on HBASE-26263: --- Ok, I got it. We can update table's TableDescriptor directly on master side. For the regions on the existing regionserver, since SFT implementation of table has not been changed, there will be no problem. > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419131#comment-17419131 ] Zhuoyue Huang commented on HBASE-26263: --- HBASE-26246 set the SFT configurations to the TableDescriptor to avoid misconfiguration. If I understand correctly, we need a background thread to update those tables that already exist but have not update the TableDescriptor to the latest SFT configuration. If we do not use ModifyTableProcedure , how can we keep the tableDescriptor on the region server side consistent with that on the master? As far as I know, region server side should reopen regions to update its tableDescriptor, which would trigger during ModifyTableProcedure processing if a table already exists. Looking forward to your answer [~zhangduo] > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419132#comment-17419132 ] Zhuoyue Huang commented on HBASE-26263: --- HBASE-26246 set the SFT configurations to the TableDescriptor to avoid misconfiguration. If I understand correctly, we need a background thread to update those tables that already exist but have not update the TableDescriptor to the latest SFT configuration. If we do not use ModifyTableProcedure , how can we keep the tableDescriptor on the region server side consistent with that on the master? As far as I know, region server side should reopen regions to update its tableDescriptor, which would trigger during ModifyTableProcedure processing if a table already exists. Looking forward to your answer [~zhangduo] > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26263: - Assignee: Zhuoyue Huang > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26251) StochasticLoadBalancer metrics should update even if balancer doesn't run
[ https://issues.apache.org/jira/browse/HBASE-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414029#comment-17414029 ] Zhuoyue Huang commented on HBASE-26251: --- HMaster would abort balancer's running if any checking logic was not passed. We should updateStochasticCosts outside the HMaster logic to make sure the updating working regardless of whether the balancer run or not. IMHO, to simplify this improvement, BalancerChore is the best choice to periodically update the cost of StochasticBalanccer. I create a new interface named 'updateBalancerLoadInfo', which StochasticLoadBalancer could implement, and in the future other balancers could update its internal status periodically. > StochasticLoadBalancer metrics should update even if balancer doesn't run > - > > Key: HBASE-26251 > URL: https://issues.apache.org/jira/browse/HBASE-26251 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Zhuoyue Huang >Priority: Minor > > Currently we only update StochasticLoadBalancerMetrics at the very end of a > balancer run, once a plan has been found. In fact, we update the metrics > based on that plan even if we don't end up executing the plan, which seems > incorrect. > Regardless of whether the balancer decides to run or not, cluster costs are > changing all the time. Since we don't update these metrics any other time, > operators miss out on important information about the balance of their > cluster over time. > I briefly looked into it and it would be relatively trivial to add another > call to updateStochasticCosts at the beginning of the balanceTable method, > before we determine if the cluster is in need of balancing. This would be an > improvement but would still miss cases where the balancer is disabled or > unable to run due to regions in transition, etc. > It would be good if we could make it so updateStochasticCosts get called > periodically regardless of whether the balancer is enabled or can run. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26251) StochasticLoadBalancer metrics should update even if balancer doesn't run
[ https://issues.apache.org/jira/browse/HBASE-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409416#comment-17409416 ] Zhuoyue Huang commented on HBASE-26251: --- I'm trying to find a way to handle this issue. Will push code later > StochasticLoadBalancer metrics should update even if balancer doesn't run > - > > Key: HBASE-26251 > URL: https://issues.apache.org/jira/browse/HBASE-26251 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Zhuoyue Huang >Priority: Minor > > Currently we only update StochasticLoadBalancerMetrics at the very end of a > balancer run, once a plan has been found. In fact, we update the metrics > based on that plan even if we don't end up executing the plan, which seems > incorrect. > Regardless of whether the balancer decides to run or not, cluster costs are > changing all the time. Since we don't update these metrics any other time, > operators miss out on important information about the balance of their > cluster over time. > I briefly looked into it and it would be relatively trivial to add another > call to updateStochasticCosts at the beginning of the balanceTable method, > before we determine if the cluster is in need of balancing. This would be an > improvement but would still miss cases where the balancer is disabled or > unable to run due to regions in transition, etc. > It would be good if we could make it so updateStochasticCosts get called > periodically regardless of whether the balancer is enabled or can run. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26251) StochasticLoadBalancer metrics should update even if balancer doesn't run
[ https://issues.apache.org/jira/browse/HBASE-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26251: - Assignee: Zhuoyue Huang > StochasticLoadBalancer metrics should update even if balancer doesn't run > - > > Key: HBASE-26251 > URL: https://issues.apache.org/jira/browse/HBASE-26251 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Zhuoyue Huang >Priority: Minor > > Currently we only update StochasticLoadBalancerMetrics at the very end of a > balancer run, once a plan has been found. In fact, we update the metrics > based on that plan even if we don't end up executing the plan, which seems > incorrect. > Regardless of whether the balancer decides to run or not, cluster costs are > changing all the time. Since we don't update these metrics any other time, > operators miss out on important information about the balance of their > cluster over time. > I briefly looked into it and it would be relatively trivial to add another > call to updateStochasticCosts at the beginning of the balanceTable method, > before we determine if the cluster is in need of balancing. This would be an > improvement but would still miss cases where the balancer is disabled or > unable to run due to regions in transition, etc. > It would be good if we could make it so updateStochasticCosts get called > periodically regardless of whether the balancer is enabled or can run. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26215) The backup master status page should use ActiveMasterManager instead of MasterAddressTracker
[ https://issues.apache.org/jira/browse/HBASE-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405532#comment-17405532 ] Zhuoyue Huang commented on HBASE-26215: --- [~zhangduo] Thanks for reviewing > The backup master status page should use ActiveMasterManager instead of > MasterAddressTracker > > > Key: HBASE-26215 > URL: https://issues.apache.org/jira/browse/HBASE-26215 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: img_68f9b25b-a8c1-46a9-8035-642d4356f80l.png, > img_bf645803-b51d-427b-bb8b-9c9ee8de664l.png > > > On master we already have ActiveMasterManager for tracking active master and > backup masters, so we do not need to use MasterAddressTracker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26215) The backup master status page should use ActiveMasterManager instead of MasterAddressTracker
[ https://issues.apache.org/jira/browse/HBASE-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26215: -- Attachment: img_68f9b25b-a8c1-46a9-8035-642d4356f80l.png > The backup master status page should use ActiveMasterManager instead of > MasterAddressTracker > > > Key: HBASE-26215 > URL: https://issues.apache.org/jira/browse/HBASE-26215 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > Attachments: img_68f9b25b-a8c1-46a9-8035-642d4356f80l.png, > img_bf645803-b51d-427b-bb8b-9c9ee8de664l.png > > > On master we already have ActiveMasterManager for tracking active master and > backup masters, so we do not need to use MasterAddressTracker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26215) The backup master status page should use ActiveMasterManager instead of MasterAddressTracker
[ https://issues.apache.org/jira/browse/HBASE-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26215: -- Attachment: img_bf645803-b51d-427b-bb8b-9c9ee8de664l.png > The backup master status page should use ActiveMasterManager instead of > MasterAddressTracker > > > Key: HBASE-26215 > URL: https://issues.apache.org/jira/browse/HBASE-26215 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > Attachments: img_68f9b25b-a8c1-46a9-8035-642d4356f80l.png, > img_bf645803-b51d-427b-bb8b-9c9ee8de664l.png > > > On master we already have ActiveMasterManager for tracking active master and > backup masters, so we do not need to use MasterAddressTracker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26215) The backup master status page should use ActiveMasterManager instead of MasterAddressTracker
[ https://issues.apache.org/jira/browse/HBASE-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403634#comment-17403634 ] Zhuoyue Huang commented on HBASE-26215: --- Since the backup master status page will remove the MasterAddressTracker, and there is no direct method to obtain ActiveMasterManager (I saw that ActiveMasterManager is a private member of HMaster, and all values in ActiveMasterManager should be returned through HMaster’s public methods). I added some code to make the backup master status page easily retrieve the information it needs. > The backup master status page should use ActiveMasterManager instead of > MasterAddressTracker > > > Key: HBASE-26215 > URL: https://issues.apache.org/jira/browse/HBASE-26215 > Project: HBase > Issue Type: Sub-task > Components: master, UI >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > On master we already have ActiveMasterManager for tracking active master and > backup masters, so we do not need to use MasterAddressTracker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26215) The backup master status page should use ActiveMasterManager instead of MasterAddressTracker
[ https://issues.apache.org/jira/browse/HBASE-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26215: - Assignee: Zhuoyue Huang > The backup master status page should use ActiveMasterManager instead of > MasterAddressTracker > > > Key: HBASE-26215 > URL: https://issues.apache.org/jira/browse/HBASE-26215 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > On master we already have ActiveMasterManager for tracking active master and > backup masters, so we do not need to use MasterAddressTracker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26216) Move HRegionServer.abort(String) to Abortable as a default method
[ https://issues.apache.org/jira/browse/HBASE-26216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26216: - Assignee: Zhuoyue Huang > Move HRegionServer.abort(String) to Abortable as a default method > - > > Key: HBASE-26216 > URL: https://issues.apache.org/jira/browse/HBASE-26216 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > So after decouple HMaster and HRegionServer, we do not need to implement it > again in HMaster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26091) Remove FirstKeyValueMatchingQualifiersFilter
[ https://issues.apache.org/jira/browse/HBASE-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383901#comment-17383901 ] Zhuoyue Huang commented on HBASE-26091: --- Keep parseFrom of FirstKeyValueMatchingQualifiersFilter to guarantee old client could communicate with new server. If users use this filter, an Exception would be throw while the scan start > Remove FirstKeyValueMatchingQualifiersFilter > > > Key: HBASE-26091 > URL: https://issues.apache.org/jira/browse/HBASE-26091 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 3.0.0-alpha-2 > > > see HBASE-13347. FirstKeyValueMatchingQualifiersFilter was deprecated in 2.0 > and should be removed in 3.0. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26091) Remove FirstKeyValueMatchingQualifiersFilter
[
https://issues.apache.org/jira/browse/HBASE-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-26091:
--
Description:
see HBASE-13347. FirstKeyValueMatchingQualifiersFilter was deprecated in 2.0
and should be removed in 3.0.
was:
see HBASE-13347. {{FirstKeyValueMatchingQualifiersFilter was d}}eprecated in
2.0 and should be removed in 3.0.
> Remove FirstKeyValueMatchingQualifiersFilter
>
>
> Key: HBASE-26091
> URL: https://issues.apache.org/jira/browse/HBASE-26091
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-2
>
>
> see HBASE-13347. FirstKeyValueMatchingQualifiersFilter was deprecated in 2.0
> and should be removed in 3.0.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-26091) Remove FirstKeyValueMatchingQualifiersFilter
[
https://issues.apache.org/jira/browse/HBASE-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-26091:
--
Description:
see HBASE-13347. {{FirstKeyValueMatchingQualifiersFilter was d}}eprecated in
2.0 and should be removed in 3.0.
> Remove FirstKeyValueMatchingQualifiersFilter
>
>
> Key: HBASE-26091
> URL: https://issues.apache.org/jira/browse/HBASE-26091
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-2
>
>
> see HBASE-13347. {{FirstKeyValueMatchingQualifiersFilter was d}}eprecated in
> 2.0 and should be removed in 3.0.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-26091) Remove FirstKeyValueMatchingQualifiersFilter
[ https://issues.apache.org/jira/browse/HBASE-26091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-26091: -- Fix Version/s: 3.0.0-alpha-2 > Remove FirstKeyValueMatchingQualifiersFilter > > > Key: HBASE-26091 > URL: https://issues.apache.org/jira/browse/HBASE-26091 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 3.0.0-alpha-2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26091) Remove FirstKeyValueMatchingQualifiersFilter
Zhuoyue Huang created HBASE-26091: - Summary: Remove FirstKeyValueMatchingQualifiersFilter Key: HBASE-26091 URL: https://issues.apache.org/jira/browse/HBASE-26091 Project: HBase Issue Type: Sub-task Components: Client Reporter: Zhuoyue Huang Assignee: Zhuoyue Huang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26015) Should implement getRegionServers(boolean) method in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26015: - Assignee: Zhuoyue Huang > Should implement getRegionServers(boolean) method in AsyncAdmin > --- > > Key: HBASE-26015 > URL: https://issues.apache.org/jira/browse/HBASE-26015 > Project: HBase > Issue Type: Task > Components: Admin, Client >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > We have this method in Admin but not in AsyncAdmin, we should align these two > interfaces. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26005) Update ref guide about the EOL for 2.2.x
[ https://issues.apache.org/jira/browse/HBASE-26005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-26005: - Assignee: Zhuoyue Huang > Update ref guide about the EOL for 2.2.x > > > Key: HBASE-26005 > URL: https://issues.apache.org/jira/browse/HBASE-26005 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > For example, remove the release manager for 2.2.x, and also update the > compatibility matrix with hadoop, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: (was: meta_replica_rit.png)
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: (was: localities.png)
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: (was: base_stats.png)
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: (was: compaction.png)
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360683#comment-17360683
]
Zhuoyue Huang edited comment on HBASE-25980 at 6/12/21, 1:58 PM:
-
I reproduced this bug and saw the same result as the description. Code below
was called in table.js and the rit region make this call throw a
NotAllMetaRegionsOnlineException.
{code:java}
MetaTableLocator.waitMetaRegionLocation(master.getZooKeeper(), j, 1)
{code}
IMHO , as the meta replica is in rit, we should simply skip to show this
region's info in UI
was (Author: georryhuang):
I reproduced this bug and saw the same result as the description. Code below
was called in table.js and the rit region make this call throw a
NotAllMetaRegionsOnlineException.
{code:java}
MetaTableLocator.waitMetaRegionLocation(master.getZooKeeper(), j, 1)
{code}
IMHO , as the meta replica is in rit, we should simply ignore this Exception
and show the default value of meta region replica on UI.
Has tried to fix this, the screenshot in attachments show what it look like
after fixed
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Comment: was deleted
(was: This PR ignores NotAllMetaRegionsOnlineException directly and cannot get
the info of the rit region, but the name of the rit region is still displayed
in the UI without its location information (see base_stats.png). Maybe there
are other implementations that we can hide the rit region on the UI.)
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360743#comment-17360743
]
Zhuoyue Huang commented on HBASE-25980:
---
This bug can also be reproduced on 3.0.0, will make a PR against master branch.
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Fix Version/s: 2.5.0
3.0.0-alpha-1
Affects Version/s: 2.5.0
3.0.0-alpha-1
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360693#comment-17360693
]
Zhuoyue Huang commented on HBASE-25980:
---
I am going to check if master3.0 also has this problem.
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360691#comment-17360691
]
Zhuoyue Huang commented on HBASE-25980:
---
This PR ignores NotAllMetaRegionsOnlineException directly and cannot get the
info of the rit region, but the name of the rit region is still displayed in
the UI without its location information (see base_stats.png). Maybe there are
other implementations that we can hide the rit region on the UI.
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: compaction.png
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360683#comment-17360683
]
Zhuoyue Huang commented on HBASE-25980:
---
I reproduced this bug and saw the same result as the description. Code below
was called in table.js and the rit region make this call throw a
NotAllMetaRegionsOnlineException.
{code:java}
MetaTableLocator.waitMetaRegionLocation(master.getZooKeeper(), j, 1)
{code}
IMHO , as the meta replica is in rit, we should simply ignore this Exception
and show the default value of meta region replica on UI.
Has tried to fix this, the screenshot in attachments show what it look like
after fixed
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: localities.png
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: meta_replica_rit.png
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang updated HBASE-25980:
--
Attachment: base_stats.png
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: base_stats.png, compaction.png, localities.png,
> meta_replica_rit.png
>
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-25980) Master table.jsp pointed at meta throws 500 when no all replicas are online
[
https://issues.apache.org/jira/browse/HBASE-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhuoyue Huang reassigned HBASE-25980:
-
Assignee: Zhuoyue Huang
> Master table.jsp pointed at meta throws 500 when no all replicas are online
> ---
>
> Key: HBASE-25980
> URL: https://issues.apache.org/jira/browse/HBASE-25980
> Project: HBase
> Issue Type: Bug
> Components: master, meta replicas, UI
>Affects Versions: 2.3.5
>Reporter: Nick Dimiduk
>Assignee: Zhuoyue Huang
>Priority: Major
>
> With a replica in a transition state, the UI renders,
> {noformat}
> HTTP ERROR 500
> Problem accessing /table.jsp. Reason:
> Server Error
> Caused by:
> org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out; 1ms
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:190)
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:264)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22708) Remove the deprecated methods in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-22708: -- Release Note: Removed method 'scheduleServerCrashProcedure' of hbck.java in hbase-client module. This method was deprecated since 2.2.1 and declare to be removed in 3.0.0. > Remove the deprecated methods in Hbck interface > --- > > Key: HBASE-22708 > URL: https://issues.apache.org/jira/browse/HBASE-22708 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: 3.0.0-alpha-1 >Reporter: Guanghao Zhang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-22708) Remove the deprecated methods in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-22708: - Assignee: Zhuoyue Huang (was: Guanghao Zhang) > Remove the deprecated methods in Hbck interface > --- > > Key: HBASE-22708 > URL: https://issues.apache.org/jira/browse/HBASE-22708 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1 >Reporter: Guanghao Zhang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25977) Remove 2.2.7 from download page
[ https://issues.apache.org/jira/browse/HBASE-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-25977: - Assignee: Zhuoyue Huang > Remove 2.2.7 from download page > --- > > Key: HBASE-25977 > URL: https://issues.apache.org/jira/browse/HBASE-25977 > Project: HBase > Issue Type: Sub-task > Components: website >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > > I think it is time to do this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25973) Balancer should explain progress in a better way in log
[ https://issues.apache.org/jira/browse/HBASE-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-25973: - Assignee: (was: Zhuoyue Huang) > Balancer should explain progress in a better way in log > --- > > Key: HBASE-25973 > URL: https://issues.apache.org/jira/browse/HBASE-25973 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 3.0.0-alpha-1 >Reporter: Clara Xiong >Priority: Major > > In the log, balancer logs at info level at the beginning of run: > > balancer.StochasticLoadBalancer: start StochasticLoadBalancer.balancer, > initCost=277.3479243125063, functionCost=RegionCountSkewCostFunction : > (500.0, 0.3749771215224234); ServerLocalityCostFunction : (25.0, > 0.5807483226644186); RackLocalityCostFunction : (15.0, 0.0); > TableSkewCostFunction : (1000.0, 0.0019704142954972883); > StoreFileCostFunction : (200.0, 0.3668512059459341); computedMaxSteps: > 42270438200 > the cost is reported without context, it is hard for operator to understand > how unbalanced the cluster is for balancer and how much progress we are > making. > For a large cluster, the calculation can take a long time, we also need to > let operator understand that it will take up to the max time to complete the > calculation. > At the end of computation: > balancer.StochasticLoadBalancer: Finished computing new load balance plan. > Computation took PT40M0.006S to try 1036409 different iterations. Found a > solution that moves 161926 regions; Going from a computed cost of > 118.75715593924485 to a new cost of 1.5509126920967042 > The time to compute the plan is also printed in a format that is not human > readable. we also need to let operator understand that balancer is just > submitting the plan and it be up to execution to complete the move. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25973) Balancer should explain progress in a better way in log
[ https://issues.apache.org/jira/browse/HBASE-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang reassigned HBASE-25973: - Assignee: Zhuoyue Huang > Balancer should explain progress in a better way in log > --- > > Key: HBASE-25973 > URL: https://issues.apache.org/jira/browse/HBASE-25973 > Project: HBase > Issue Type: Bug > Components: Balancer >Affects Versions: 3.0.0-alpha-1 >Reporter: Clara Xiong >Assignee: Zhuoyue Huang >Priority: Major > > In the log, balancer logs at info level at the beginning of run: > > balancer.StochasticLoadBalancer: start StochasticLoadBalancer.balancer, > initCost=277.3479243125063, functionCost=RegionCountSkewCostFunction : > (500.0, 0.3749771215224234); ServerLocalityCostFunction : (25.0, > 0.5807483226644186); RackLocalityCostFunction : (15.0, 0.0); > TableSkewCostFunction : (1000.0, 0.0019704142954972883); > StoreFileCostFunction : (200.0, 0.3668512059459341); computedMaxSteps: > 42270438200 > the cost is reported without context, it is hard for operator to understand > how unbalanced the cluster is for balancer and how much progress we are > making. > For a large cluster, the calculation can take a long time, we also need to > let operator understand that it will take up to the max time to complete the > calculation. > At the end of computation: > balancer.StochasticLoadBalancer: Finished computing new load balance plan. > Computation took PT40M0.006S to try 1036409 different iterations. Found a > solution that moves 161926 regions; Going from a computed cost of > 118.75715593924485 to a new cost of 1.5509126920967042 > The time to compute the plan is also printed in a format that is not human > readable. we also need to let operator understand that balancer is just > submitting the plan and it be up to execution to complete the move. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25666) Explain why balancer is skipping runs
[ https://issues.apache.org/jira/browse/HBASE-25666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang resolved HBASE-25666. --- Resolution: Fixed > Explain why balancer is skipping runs > - > > Key: HBASE-25666 > URL: https://issues.apache.org/jira/browse/HBASE-25666 > Project: HBase > Issue Type: Improvement > Components: Balancer, master, UI >Reporter: Clara Xiong >Assignee: Zhuoyue Huang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.4 > > > Balancer needs to find the balance of keeping cluster balanced and stable. > There is a configuration minCostNeedBalance as the threshold of imbalance to > try to balance. Since we use a single score to combine all the factors we > consider, it is hard for operation to understand why balancer is "stuck". We > should add to master-status UI to show balancer is skipping runs and explain > all factors considered, such as the weight and cost of all cost functions. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356283#comment-17356283 ] Zhuoyue Huang commented on HBASE-25914: --- The latest version was pushed to [GitHub Pull Request #3319|https://github.com/apache/hbase/pull/3319]. see two new screenshots 'opeartionDetails1.png' and 'opeartionDetails2.png' > Provide slow/large logs on RegionServer UI > -- > > Key: HBASE-25914 > URL: https://issues.apache.org/jira/browse/HBASE-25914 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Attachments: callDetails.png, largeLog.png, opeartionDetails1.png, > operationDetails2.png, slowLog.png > > > Pulling slow/large log from in-memory queues on RegionServer then display > details info in RegionServer status UI -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25914) Provide slow/large logs on RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-25914: -- Attachment: opeartionDetails1.png > Provide slow/large logs on RegionServer UI > -- > > Key: HBASE-25914 > URL: https://issues.apache.org/jira/browse/HBASE-25914 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Attachments: callDetails.png, largeLog.png, opeartionDetails1.png, > operationDetails2.png, slowLog.png > > > Pulling slow/large log from in-memory queues on RegionServer then display > details info in RegionServer status UI -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25914) Provide slow/large logs on RegionServer UI
[ https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhuoyue Huang updated HBASE-25914: -- Attachment: operationDetails2.png > Provide slow/large logs on RegionServer UI > -- > > Key: HBASE-25914 > URL: https://issues.apache.org/jira/browse/HBASE-25914 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Zhuoyue Huang >Assignee: Zhuoyue Huang >Priority: Major > Attachments: callDetails.png, largeLog.png, opeartionDetails1.png, > operationDetails2.png, slowLog.png > > > Pulling slow/large log from in-memory queues on RegionServer then display > details info in RegionServer status UI -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356091#comment-17356091
]
Zhuoyue Huang edited comment on HBASE-25914 at 6/3/21, 4:00 AM:
[~stack] [~vjasani] Thanks for the comments. If there aren't any new
suggestion, I will push a new commit following the modification below
h1. Rename text
# Tab 'Named Queue Logs' rename to 'Operation Details'(Every record saved in
named queue framework ring buffers are about 'opeartion' information)
# sub-tab 'Slow Logs' rename to 'Slow RPCs '
# sub-tab 'Large Logs' rename to 'Large Response RPCs'
h1. Explanation
# Explanation of the main tab 'Operation Details'
{code:java}
HBase uses some fixed-size ring buffers to maintain rolling window history of
specific server-side operation details. This page list all operation details
retrieve from these ring buffers
{code}
2. Explanation of sub-tab 'Slow RPCs'
{code:java}
Slow RPCs record those RPCs whose processing time is greater than the threshold
(see the setting 'hbase.ipc.warn.response.time' for details)
{code}
3. Explanation of sub-tab 'Large Response RPCs'
{code:java}
Large response RPCs record those RPCs whose returned data size is greater than
the threshold (see the setting'hbase.ipc.warn.response.size' for details)
{code}
h1. RPC full display opt-in switch
The setting 'hbase.regionserver.slowlog.buffer.enabled' has determined whether
to record slow/large log on RegionServer, and this setting also determines
whether the UI is displayed. If set 'hbase.regionserver.slowlog.buffer.enabled'
as false. the slow log UI on RegionServer does not show any rpc records
was (Author: georryhuang):
[~stack] [~vjasani] Thanks for the comments. If there aren't any new
suggestion, I will push a new commit following the modification below
h1. Rename text
# Tab 'Named Queue Logs' rename to 'Operation Details'(Every record saved in
named queue framework ring buffers are about 'opeartion' information)
# sub-tab 'Slow Logs' rename to 'Slow RPCs '
# sub-tab 'Large Logs' rename to 'Large Response RPCs'
h1. Explanation
# Explanation of the main tab 'Operation Details'
{code:java}
HBase uses some fixed-size ring buffers to maintain rolling window history of
specific server-side operation details. This page list all operation details
retrieve from these ring buffers
{code}
2. Explanation of sub-tab 'Slow RPCs'
{code:java}
Slow RPCs record those RPCs whose processing time is greater than the threshold
(see the setting 'hbase.ipc.warn.response.time' for details)
{code}
3. Explanation of sub-tab 'Large Response RPCs'
{code:java}
Large response RPCs record those RPCs whose returned data size is greater than
the threshold (see the setting'hbase.ipc.warn.response.size' for details)
{code}
h1. RPC full display opt-in switch
The setting 'hbase.regionserver.slowlog.buffer.enabled' has determined whether
to record slow/large log on RegionServer, and this setting also determines
whether the UI is displayed. If set 'hbase.regionserver.slowlog.buffer.enabled'
as false. the slow log UI on RegionServer does not show up
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356091#comment-17356091
]
Zhuoyue Huang commented on HBASE-25914:
---
[~stack] [~vjasani] Thanks for the comments. If there aren't any new
suggestion, I will push a new commit following the modification below
h1. Rename text
# Tab 'Named Queue Logs' rename to 'Operation Details'(Every record saved in
named queue framework ring buffers are about 'opeartion' information)
# sub-tab 'Slow Logs' rename to 'Slow RPCs '
# sub-tab 'Large Logs' rename to 'Large Response RPCs'
h1. Explanation
# Explanation of the main tab 'Operation Details'
{code:java}
HBase uses some fixed-size ring buffers to maintain rolling window history of
specific server-side operation details. This page list all operation details
retrieve from these ring buffers
{code}
2. Explanation of sub-tab 'Slow RPCs'
{code:java}
Slow RPCs record those RPCs whose processing time is greater than the threshold
(see the setting 'hbase.ipc.warn.response.time' for details)
{code}
3. Explanation of sub-tab 'Large Response RPCs'
{code:java}
Large response RPCs record those RPCs whose returned data size is greater than
the threshold (see the setting'hbase.ipc.warn.response.size' for details)
{code}
h1. RPC full display opt-in switch
The setting 'hbase.regionserver.slowlog.buffer.enabled' has determined whether
to record slow/large log on RegionServer, and this setting also determines
whether the UI is displayed. If set 'hbase.regionserver.slowlog.buffer.enabled'
as false. the slow log UI on RegionServer does not show up
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354389#comment-17354389
]
Zhuoyue Huang commented on HBASE-25914:
---
{quote} In the current implementation we directly use the max queue size as
param to retrieve all log records
{quote}
Not max size, but default size
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354386#comment-17354386
]
Zhuoyue Huang commented on HBASE-25914:
---
{quote}In the past, folks have been worried about showing full RPC payload in
logs... privacy/security concerns. The full display of rpc payload should
probably be an opt-in switch.{quote}
Agree! 'hbase.regionserver.slowlog.buffer.enabled' already set whether the RS
should record 'Slow Logs/Large Logs'. I think we can use this conf to decide
whether UI should display the so-called 'Slow Logs' page in UI. any other
suggestions?
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354385#comment-17354385
]
Zhuoyue Huang commented on HBASE-25914:
---
{quote}I think so and hopefully it will be scrollable. [Zhuoyue
Huang|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=GeorryHuang]
is my assumption correct? As API supports limit param, let's have a drop down
on UI to limit no of recent records in UI?{quote}
[~vjasani] Yes, you are right. But should we really need a drop-down on UI? In
the current implementation we directly use the max queue size as param to
retrieve all log records(Now, using admin API to retrieve RS log entries maybe
not a good way.)
Using page buttons to switch pages, and each page showing fixed-line records is
another good way(just like the UI of user tables page)?
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25914) Provide slow/large logs on RegionServer UI
[
https://issues.apache.org/jira/browse/HBASE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354378#comment-17354378
]
Zhuoyue Huang commented on HBASE-25914:
---
{quote}I'd expect some text on the page explaining what is being
displayed,{quote}
{quote}Would suggest a version of your explanation be added to the head of the
'Named Queue Log' tab to explain what it is.{quote}
[~stack] Ok, going to add explanation.
{quote}My concern w/ 'Name Queue Log' is that it indeed is generic, so generic,
the operator will be confounded by what it is they are looking at; the tab
needs explanatory text and a better name I'd suggest (HBase has many queues.
'Name'? 'Log' is usually a file on disk ).{quote}
{quote}On naming, instead of 'Name Queue Log' , which exposes the
implementation, something like 'Detail' or 'Operation Detail' with explanation
on the page of why the particular set of samples are displayed and of what the
samples are? Perhaps others have better input here. They are not 'Large Logs'
or 'Slow Logs', they are slow queries or requests or rpcs? And ditto for large,
no?{quote}
Let me think about how to name it. Thanks for your suggestion.
> Provide slow/large logs on RegionServer UI
> --
>
> Key: HBASE-25914
> URL: https://issues.apache.org/jira/browse/HBASE-25914
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Zhuoyue Huang
>Assignee: Zhuoyue Huang
>Priority: Major
> Attachments: callDetails.png, largeLog.png, slowLog.png
>
>
> Pulling slow/large log from in-memory queues on RegionServer then display
> details info in RegionServer status UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
