[
https://issues.apache.org/jira/browse/HBASE-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
churro morales updated HBASE-10528:
-----------------------------------
Description:
We have quite a large cluster > 100k regions, and we needed to isolate a region
was very hot until we could push a patch. We put this region on its own
regionserver and set it in the draining state. The default balancer was
selecting regions to move to this cluster for its region plans.
It just so happened for other tables, the default load balancer was creating
plans for the draining servers, even though they were not available to move
regions to. Thus we were closing regions, then attempting to move them to the
draining server then finding out its draining.
We had to disable the balancer to resolve this issue.
There are some approaches we can take here.
1. Exclude draining servers altogether, don't even pass those into the load
balancer from HMaster.
2. We could exclude draining servers from ceiling and floor calculations where
we could potentially skip load balancing because those draining servers wont be
represented when deciding whether to balance.
3. Along with #2 when assigning regions, we would skip plans to assign regions
to those draining servers.
I am in favor of #1 which is simply removes servers as candidates for balancing
if they are in the draining state.
But I would love to hear what everyone else thinks.
was:
We have quite a large cluster > 100k regions, and we needed to isolate a region
was very hot until we could push a patch. We put this region on its own
regionserver and set it in the draining state. The default balancer was
selecting regions to move to this cluster for its region plans.
It just so happened that there were very small regions on the draining servers,
which constantly triggered balancing. Thus we were closing regions, then
attempting to move them to the draining server then finding out its draining.
We had to disable the balancer to resolve this issue.
There are some approaches we can take here.
1. Exclude draining servers altogether, don't even pass those into the load
balancer from HMaster.
2. We could exclude draining servers from ceiling and floor calculations where
we could potentially skip load balancing because those draining servers wont be
represented when deciding whether to balance.
3. Along with #2 when assigning regions, we would skip plans to assign regions
to those draining servers.
I am in favor of #1 which is simply removes servers as candidates for balancing
if they are in the draining state.
But I would love to hear what everyone else thinks.
> DefaultBalancer selects plans to move regions onto draining nodes
> -----------------------------------------------------------------
>
> Key: HBASE-10528
> URL: https://issues.apache.org/jira/browse/HBASE-10528
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.5
> Reporter: churro morales
>
> We have quite a large cluster > 100k regions, and we needed to isolate a
> region was very hot until we could push a patch. We put this region on its
> own regionserver and set it in the draining state. The default balancer was
> selecting regions to move to this cluster for its region plans.
> It just so happened for other tables, the default load balancer was creating
> plans for the draining servers, even though they were not available to move
> regions to. Thus we were closing regions, then attempting to move them to
> the draining server then finding out its draining.
> We had to disable the balancer to resolve this issue.
> There are some approaches we can take here.
> 1. Exclude draining servers altogether, don't even pass those into the load
> balancer from HMaster.
> 2. We could exclude draining servers from ceiling and floor calculations
> where we could potentially skip load balancing because those draining servers
> wont be represented when deciding whether to balance.
> 3. Along with #2 when assigning regions, we would skip plans to assign
> regions to those draining servers.
> I am in favor of #1 which is simply removes servers as candidates for
> balancing if they are in the draining state.
> But I would love to hear what everyone else thinks.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
