[
https://issues.apache.org/jira/browse/HBASE-24527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-24527:
----------------------------------------
Description:
We provide a coarse grained admin API and associated shell command for
determining the compaction status of a table:
{noformat}
hbase(main):001:0> help "compaction_state"
Here is some help for this command:
Gets compaction status (MAJOR, MAJOR_AND_MINOR, MINOR, NONE) for a table:
hbase> compaction_state 'ns1:t1'
hbase> compaction_state 't1'
{noformat}
We also log compaction activity, including a compaction journal at completion,
via log4j to whatever log aggregation solution is available in production.
This is not sufficient for online and interactive observation, debugging, or
performance analysis of current compaction activity. In this kind of activity
an operator is attempting to observe and analyze compaction activity in real
time. Log aggregation and presentation solutions have typical latencies (end to
end visibility of log lines on the order of ~minutes) which make that not
possible today.
We don't offer any API or tools for directly interrogating split and merge
activity in real time. Some indirect knowledge of split or merge activity can
be inferred from RIT information via ClusterStatus. It can also be scraped,
with some difficulty, from the debug servlet.
We should have new APIs and shell commands, and perhaps also new admin UI
views, for
at regionserver scope:
* listing the current state of a regionserver's compaction, split, and merge
tasks and threads
* counting (simple view) and listing (detailed view) a regionserver's
compaction queues
* listing a region's currently compacting, splitting, or merging status
at master scope, aggregations of the above detailed information into:
* listing the active compaction tasks and threads for a given table, the
extension of _compaction_state_ with a new detailed view
* listing the active split or merge tasks and threads for a given table's
regions
was:
We provide a coarse grained admin API and associated shell command for
determining the compaction status of a table:
{noformat}
hbase(main):001:0> help "compaction_state"
Here is some help for this command:
Gets compaction status (MAJOR, MAJOR_AND_MINOR, MINOR, NONE) for a table:
hbase> compaction_state 'ns1:t1'
hbase> compaction_state 't1'
{noformat}
We also log compaction activity, including a compaction journal at completion,
via log4j to whatever log aggregation solution is available in production.
This is not sufficient for online and interactive observation, debugging, or
performance analysis of current compaction activity. In this kind of activity
an operator is attempting to observe and analyze compaction activity in real
time. Log aggregation and presentation solutions have typical latencies (end to
end visibility of log lines on the order of ~minutes) which make that not
possible today.
We don't offer any API or tools for directly interrogating split and merge
activity in real time. Some indirect knowledge of split or merge activity can
be inferred from RIT information via ClusterStatus.
We should have new APIs and shell commands, and perhaps also new admin UI
views, for
at regionserver scope:
* listing the current state of a regionserver's compaction, split, and merge
tasks and threads
* counting (simple view) and listing (detailed view) a regionserver's
compaction queues
* listing a region's currently compacting, splitting, or merging status
at master scope, aggregations of the above detailed information into:
* listing the active compaction tasks and threads for a given table, the
extension of _compaction_state_ with a new detailed view
* listing the active split or merge tasks and threads for a given table's
regions
> Improve region housekeeping status observability
> ------------------------------------------------
>
> Key: HBASE-24527
> URL: https://issues.apache.org/jira/browse/HBASE-24527
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Compaction, shell, UI
> Reporter: Andrew Kyle Purtell
> Priority: Major
>
> We provide a coarse grained admin API and associated shell command for
> determining the compaction status of a table:
> {noformat}
> hbase(main):001:0> help "compaction_state"
> Here is some help for this command:
> Gets compaction status (MAJOR, MAJOR_AND_MINOR, MINOR, NONE) for a table:
> hbase> compaction_state 'ns1:t1'
> hbase> compaction_state 't1'
> {noformat}
> We also log compaction activity, including a compaction journal at
> completion, via log4j to whatever log aggregation solution is available in
> production.
> This is not sufficient for online and interactive observation, debugging, or
> performance analysis of current compaction activity. In this kind of activity
> an operator is attempting to observe and analyze compaction activity in real
> time. Log aggregation and presentation solutions have typical latencies (end
> to end visibility of log lines on the order of ~minutes) which make that not
> possible today.
> We don't offer any API or tools for directly interrogating split and merge
> activity in real time. Some indirect knowledge of split or merge activity can
> be inferred from RIT information via ClusterStatus. It can also be scraped,
> with some difficulty, from the debug servlet.
> We should have new APIs and shell commands, and perhaps also new admin UI
> views, for
> at regionserver scope:
> * listing the current state of a regionserver's compaction, split, and merge
> tasks and threads
> * counting (simple view) and listing (detailed view) a regionserver's
> compaction queues
> * listing a region's currently compacting, splitting, or merging status
> at master scope, aggregations of the above detailed information into:
> * listing the active compaction tasks and threads for a given table, the
> extension of _compaction_state_ with a new detailed view
> * listing the active split or merge tasks and threads for a given table's
> regions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
