[
https://issues.apache.org/jira/browse/HBASE-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192491#comment-13192491
]
Mikhail Bautin commented on HBASE-5262:
---------------------------------------
Some of the information needed for compaction decision making:
* Compaction start time
* For each compaction input file:
* Size
* Number of key/value pairs
* Average key/value size
* other metadata
* Compaction end time
* Compaction status (success, failure)
* The same metadata as above for the compaction output file
I am not yet sure what information we would like to collect for caching
decision making—that needs more thinking.
We could also collect "region history", e.g. region open / close events:
* Region name
* Event type
* Server where the region was opened or closed
* Reason
That would allow to detect problematic regions that move from machine to
machine automatically.
I agree that it would make sense to isolate information collection logic from
decision making logic, so that external adaptive cluster tuning tools and/or
external sources of information could be plugged in.
> Structured event log for HBase for monitoring and auto-tuning performance
> -------------------------------------------------------------------------
>
> Key: HBASE-5262
> URL: https://issues.apache.org/jira/browse/HBASE-5262
> Project: HBase
> Issue Type: Improvement
> Reporter: Mikhail Bautin
>
> Creating this JIRA to open a discussion about a structured (machine-readable)
> log that will record events such as compaction start/end times, compaction
> input/output files, their sizes, the same for flushes, etc. This can be
> stored e.g. in a new system table in HBase itself. The data from this log can
> then be analyzed and used to optimize compactions at run time, or otherwise
> auto-tune HBase configuration to reduce the number of knobs the user has to
> configure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
