[
https://issues.apache.org/jira/browse/HBASE-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars George updated HBASE-2021:
-------------------------------
Attachment: HBASE-2021.patch
HBASE-2021.patch adds
- compaction queue size to region server metrics, can be seen on
regionserver.jsp
|Metrics | request=0.0, regions=3, stores=6, storefiles=3,
storefileIndexSize=0, memstoreSize=0, compactionQueueSize=0, usedHeap=35,
maxHeap=994, blockCacheSize=1713760, blockCacheFree=206913552,
blockCacheCount=1, blockCacheHitRatio=95 | RegionServer Metrics; file and heap
sizes are in megabytes |
The "compactionQueueSize" reflects the number of regions this regionserver has
in the queue to be processed and allows to "determine" if compactions are in
progress from the UI.
I would have preferred to have a global indicator in the master UI but that
value cannot be bubbled up unless I for example add it to HServerInfo - which I
have been told is not a good idea to change. Right?
- Added fragmentation details to UI
Added a total fragmentation precentage to master.jsp in top table, for example:
| Regions On FS | 3 | Number of regions on FileSystem. Rough count. |
| Fragmentation | 17% | Overall fragmentation of all tables, including .META.
and -ROOT-.|
|Zookeeper Quorum | localhost:2181 | Addresses of all registered ZK servers.
For more, see zk dump.|
Then in the tables below for .META., -ROOT- and user table I added it as a
column with an abbreviated title but added a pop-up explanation when you hover
over it.
> Add compaction details to master UI
> -----------------------------------
>
> Key: HBASE-2021
> URL: https://issues.apache.org/jira/browse/HBASE-2021
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: Lars George
> Assignee: Lars George
> Priority: Minor
> Attachments: HBASE-2021.patch
>
>
> There are two issues with this, first to detect that there is a compaction
> needed. You can currently use the little helper util that checks if a table
> has at least one colfam with more than one store file. I though about
> scanning all tables and all colfams in each and then compute the
> "fragmentation" ratio as a percentage of colfams with more than one store to
> the total number of colfams. That gives a "Table xyz is 33% fragmented"
> output. While minor percentage are normal under insert operations it is still
> important to know how bad the fragmentation is overall.
> Another idea is to weigh the number of files per store too, so that if you
> have two per colfam it is considered "low" and if you have more, for example
> 6-8 it is considered "high". Not sure how that can be done yet but noting the
> idea down here.
> Of course seeing the .META. fragmentation is useful to quickly debug
> performance issues (as JD told me on IRC).
> The other issue is that when you have started a compaction you have no idea
> how far it is and if it is still in progress. One indication of course is the
> above value. If it is 0% then all is done. But if you are at say 23%, is it
> still compacting? We could have a simple status that compactions are still in
> progress.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
