[
https://issues.apache.org/jira/browse/HBASE-20945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sekhon updated HBASE-20945:
--------------------------------
Description:
Request that the timestamp of the last major compaction be stored in JMX API
available at /jmx.
Major Compactions may be disabled to better control scheduling to trigger off
peak (this is an old school recommendation), but there is a risk that the major
compaction doesn't happen in that case. Also people may trigger major
compactions manually and it's hard to see that (I've looked at graphs of
storefile counts where it's not obvious but I can infer it from spikes in
compaction queue length). Storing the last timestamps would allow all sorts of
scripting checks against the API much more simply than trying to infer it from
changes in graphs. Also with recent changes to allow compactions to be
cancelled in HBASE-6028, the queue length doesn't tell the whole story as the
compaction may not have happened if it got cancelled, so the compaction queue
spike will be there even though major compaction did not in fact
happen/complete.
Since major compactions may take hours and can also now be cancelled in the
latest versions of HBase, we need a few different fields added to JMX:
* HBase Master JMX:
** timestamp that last major compaction was triggered, either manually via
major_compact command or via schedule
** timestamp that last major compaction completed successfully (since
timestamp above could have been started and then later cancelled manually if
load was too high)
* HBase Regionserver JMX:
** timestamp per region that last major compaction was triggered (there are
already compcationsCompletedCount, numBytesCompactedCount and
numFilesCompactedCount so it makes sense to add this next to those for each
region)
** timestamp per region that last major compaction completed successfully
was:
Request that the timestamp of the last major compaction be stored in JMX API
available at /jmx.
Major Compactions may be disabled to better control scheduling to trigger off
peak (this is an old school recommendation), but there is a risk that the major
compaction doesn't happen in that case. Also people may trigger major
compactions manually and it's hard to see that (I've looked at graphs of
storefile counts where it's not obvious but I can infer it from spikes in
compaction queue length). Storing the last timestamps would allow all sorts of
scripting checks against the API much more simply than trying to infer it from
changes in graphs. Also with recent changes to allow compactions to be
cancelled in HBASE-6028, the queue length doesn't tell the whole story as the
compaction may not have happened if it got cancelled, so the compaction queue
spike will be there even though major compaction did not in fact
happen/complete.
Since major compactions may take hours and can also now be cancelled in the
latest versions of HBase, we need a few different fields added to JMX:
* HBase Master JMX:
** timestamp that last major compaction was triggered, either manually via
major_compact command or via schedule
** timestamp that last major compaction completed successfully (since
timestamp above could have been started and then later cancelled manually if
load was too high)
* HBase Regionserver JMX:
** timestamp per region that last major compaction was triggered (there are
already compcationsCompletedCount, numBytesCompactedCount and
numFilesCompactedCount so it makes sense to add this next to those for each
region)
** timestamp per region that last major compaction completed successfully
Ideally expose this JMX info in the HMaster UI as well.
> HBase JMX - timestamp of last Major Compaction (started, completed
> successfully)
> --------------------------------------------------------------------------------
>
> Key: HBASE-20945
> URL: https://issues.apache.org/jira/browse/HBASE-20945
> Project: HBase
> Issue Type: Improvement
> Components: API, Compaction, master, metrics, monitoring,
> regionserver, tooling
> Affects Versions: 1.1.2
> Reporter: Hari Sekhon
> Priority: Major
>
> Request that the timestamp of the last major compaction be stored in JMX API
> available at /jmx.
> Major Compactions may be disabled to better control scheduling to trigger off
> peak (this is an old school recommendation), but there is a risk that the
> major compaction doesn't happen in that case. Also people may trigger major
> compactions manually and it's hard to see that (I've looked at graphs of
> storefile counts where it's not obvious but I can infer it from spikes in
> compaction queue length). Storing the last timestamps would allow all sorts
> of scripting checks against the API much more simply than trying to infer it
> from changes in graphs. Also with recent changes to allow compactions to be
> cancelled in HBASE-6028, the queue length doesn't tell the whole story as the
> compaction may not have happened if it got cancelled, so the compaction queue
> spike will be there even though major compaction did not in fact
> happen/complete.
> Since major compactions may take hours and can also now be cancelled in the
> latest versions of HBase, we need a few different fields added to JMX:
> * HBase Master JMX:
> ** timestamp that last major compaction was triggered, either manually via
> major_compact command or via schedule
> ** timestamp that last major compaction completed successfully (since
> timestamp above could have been started and then later cancelled manually if
> load was too high)
> * HBase Regionserver JMX:
> ** timestamp per region that last major compaction was triggered (there are
> already compcationsCompletedCount, numBytesCompactedCount and
> numFilesCompactedCount so it makes sense to add this next to those for each
> region)
> ** timestamp per region that last major compaction completed successfully
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
