[
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134021#comment-17134021
]
Viraj Jasani commented on HBASE-24528:
--------------------------------------
{color:#172b4d}While I am not fully aware of all advantages local region
provides, I believe rolling/sliding window based use-cases are best served by
concurrent queues. But again I don't know all pros and maybe local region can
turn out to be suitable (worth spending time for).{color}
However, I agree on region growth and assignment points(need further
information on how well they are handled) and the feature availability on
releases other than 3.0 (specifically it should be at least on 2.3/2.4 at this
point).
Yesterday I was planning to start working on refactor and creating framework
for namedQueues, and if it can be checked-in in next couple of days, we won't
have to worry about reverting HBASE-22978 and it's sub-tasks from branch-2.3.
However, I believe this will put more burden on everyone including reviewers
and also [~ndimiduk] for delaying RC creation for 2.3. branch-2.3 is already
under moratorium and I don't want to break it. Still, just wanted to put this
as a thought.
[~apurtell] let me revert the relevant commits from branch-2.3 in a couple of
days, unless there is any objections or change of thought w.r.t ring buffer
framework implementation.
> Improve balancer decision observability
> ---------------------------------------
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Balancer, Operability, shell, UI
> Reporter: Andrew Kyle Purtell
> Priority: Major
>
> We provide detailed INFO and DEBUG level logging of balancer decision
> factors, outcome, and reassignment planning, as well as similarly detailed
> logging of the resulting assignment manager activity. However, an operator
> may need to perform online and interactive observation, debugging, or
> performance analysis of current balancer activity. Scraping and correlating
> the many log lines resulting from a balancer execution is labor intensive and
> has a lot of latency (order of ~minutes to acquire and index, order of
> ~minutes to correlate).
> The balancer should maintain a rolling window of history, e.g. the last 100
> region move plans, or last 1000 region move plans submitted to the assignment
> manager. This history should include decision factor details and weights and
> costs. The rsgroups balancer may be able to provide fairly simple decision
> factors, like for example "this table was reassigned to that regionserver
> group". The underlying or vanilla stochastic balancer on the other hand,
> after a walk over random assignment plans, will have considered a number of
> cost functions with various inputs (locality, load, etc.) and multipliers,
> including custom cost functions. We can devise an extensible class structure
> that represents explanations for balancer decisions, and for each region move
> plan that is actually submitted to the assignment manager, we can keep the
> explanations of all relevant decision factors alongside the other details of
> the assignment plan like the region name, and the source and destination
> regionservers.
> This history should be available via API for use by new shell commands and
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of
> balancer decision components into human readable output.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
