[
https://issues.apache.org/jira/browse/HBASE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882034#action_12882034
]
Jonathan Gray commented on HBASE-2697:
--------------------------------------
Re: metascanner, the discovery of something being unassigned would no longer
come for metascanner.
These are the reasons a region is ever unassigned:
1) Cluster Startup
Master knows this condition, we can do a full assignment algorithm that takes
into account block locality.
Use metascanner to get list of all regions but don't need the
metascanner-triggered/heartbeat-based assignment.
Master will hold the assignment "plan" in memory, send RPCs out to the RSs, and
track the transitions regions via zk.
2) Node Failure
Master knows this condition, generates an assignment plan for the regions on
that node.
This could use metascanner to get list of regions on that node.
Another option is to have the master keep an up-to-date map of rs to all of its
regions rather than "most loaded" as it is today (now master needs more memory?
need to see how many regions we could track)
Or each RS could store its hosted regions in zk, or if we ever move META itself
into zk, the info would be available from zk.
(I lean towards using metascanner to get list of regions, we'll know if any of
them were in transition from zk... can always do other stuff later but not
necessary for these jiras)
3) Load Balancing
Current plan is that this becomes a background thread rather than being
calculated on heartbeats.
Master generates a plan for all the movement of regions, currently we have
mostLoadedRegions for each server which serves as candidates, some of the stuff
above would make it so we could cheaply know all the regions. There's plenty
more to do here by taking per-region load info into account but moving to a
separate class and background thread makes it easier to get here eventually.
4) Master Failover
Master knows this condition. There's some discussion on how this would work
over in HBASE-2700.
Doesn't need to use metascanner, only needs to make sure to take care of any
regions that are in transition and might not get assigned out if it doesn't
take action.
So we'll still use the metascanner but it will be triggered as needed rather
than the periodic operation it is today.
> Implement new open/close logic in handlers and stop using heartbeats for
> open/close messages
> --------------------------------------------------------------------------------------------
>
> Key: HBASE-2697
> URL: https://issues.apache.org/jira/browse/HBASE-2697
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, regionserver
> Reporter: Jonathan Gray
> Priority: Critical
> Fix For: 0.21.0
>
>
> This issue is doing the meat of what HBASE-2485 is about and continues what
> was started in HBASE-2694 after some code cleanup to make life easier.
> This deals with no longer piggybacking messages from Master to RegionServers
> on heartbeat responses and instead sending direct unsolicited messages. This
> also deals with moving the open/close logic fully into handlers and removing
> the existing open/close code on both the RS and M sides. There may also be
> some changes to the master in-memory state of regions in transition. The new
> load balancer will probably be introduced with this issue but not fully
> integrated yet.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
