[
https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492669#comment-13492669
]
stack commented on HBASE-5258:
------------------------------
[~sershe] Currently the RegionLoad in ServerLoad is badly overloaded carrying
both load and attributes. Historically it was just easiest adding attributes
to the message the RS was passing the Master on a heartbeat. Now we are pb'ing
everywhere, we can entertain breaking load and attributes apart. Or, as below
outlines, we'd not put load nor attributes on the heartbeat at all.
Load is currently sent by the regionservers to the master on the old heartbeat
channel that has been around since the early days. The only reason the master
gets the load is so its UI can give a bit of a clue as to what is going on
clusterwise. The load the regionservers put on the heartbeat is mostly a
duplicate of what the regionserver emits over its metrics channel. It would be
good if we did not duplicate this load composition. It would be good if
instead the master listened to the regionserver metrics emissions rather than
get load off the heartbeat (or, RS wrote their load/metrics to a table and
master read that whenever it wanted to show a page on cluster state). That
would make it so the heartbeat now only carries attributes. But maybe it
should not carry this even. Most of the time the attributes do not change so
it is almost as though they should be written once and then the master would
consult them when it needs them. The master could, say, write them to zk into
the body of the RS ephemeral znode. Master can read them when it joins the
cluster or when they are updated by the RS.
So, then the heartbeat would seem to have no purpose given zk client in the RS
is already heartbeating so we could drop it.
But maybe we shouldn't and rather, we should rethread it. FB are starting to
go down the path where they are starting to rely on this RS heartbeat rather
than the ephemeral node in zk and letting master make the call on whether a RS
should be let go or not. They are tending this way because the zk ephemeral
node is dumb -- no logic other than timeout -- and because if master is allowed
say what is dead and what is alive, it has the opportunity of applying rules;
e.g. if all regionservers in the cluster are unresponsive all at the same time,
its probably a network issue so hold tight till the network comes back (as
opposed to zk which would just expire all sessions and all of the cluster would
go down).
Hope this helps.
> Move coprocessors set out of RegionLoad
> ---------------------------------------
>
> Key: HBASE-5258
> URL: https://issues.apache.org/jira/browse/HBASE-5258
> Project: HBase
> Issue Type: Task
> Reporter: Ted Yu
> Priority: Critical
>
> When I worked on HBASE-5256, I revisited the code related to Ser/De of
> coprocessors set in RegionLoad.
> I think the rationale for embedding coprocessors set is for maximum
> flexibility where each region can load different coprocessors.
> This flexibility is causing extra cost in the region server to Master
> communication and increasing the footprint of Master heap.
> Would HServerLoad be a better place for this set ?
> If required, region server should calculate disparity of loaded coprocessors
> among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
