[
https://issues.apache.org/jira/browse/HBASE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011444#comment-13011444
]
Ted Yu commented on HBASE-3694:
-------------------------------
memstoreSizeMB is a member of RegionServerMetrics and is set at
hbase.regionserver.msginterval
See line 1162 in HRegionServer.java:
{code}
this.metrics.memstoreSizeMB.set((int) (memstoreSize / (1024 * 1024)));
{code}
memstoreSizeMB is of type MetricsIntValue which is a subclass of MetricsBase
and stores value in:
{code}
private int value;
{code}
We can create MetricsAtomicLongValue class with following signature:
{code}
public class MetricsAtomicLongValue extends MetricsBase{
private AtomicLong value;
private boolean changed;
{code}
If we reach agreement on adding this method to RegionServerServices (which is
available in HRegionServer and being used by MemStoreFlusher):
{code}
/**
* @return Region server metrics instance.
*/
public RegionServerMetrics getMetrics() {
{code}
then we can change memstoreSizeMB to memstoreSize which is of type
MetricsAtomicLongValue and blend Liyin's changes onto memstoreSize.
> high multiput latency due to checking global mem store size in a synchronized
> function
> --------------------------------------------------------------------------------------
>
> Key: HBASE-3694
> URL: https://issues.apache.org/jira/browse/HBASE-3694
> Project: HBase
> Issue Type: Improvement
> Reporter: Liyin Tang
> Assignee: Liyin Tang
> Attachments: Hbase-3694[r1085306], Hbase-3694[r1085306]_2.patch,
> Hbase-3694[r1085306]_3.patch, Hbase-3694[r1085508]_4.patch
>
>
> The problem is we found the multiput latency is very high.
> In our case, we have almost 22 Regions in each RS and there are no flush
> happened during these puts.
> After investigation, we believe that the root cause is the function
> getGlobalMemStoreSize, which is to check the high water mark of mem store.
> This function takes almost 40% of total execution time of multiput when
> instrumenting some metrics in the code.
> The actual percentage may be more higher. The execution time is spent on
> synchronize contention.
> One solution is to keep a static var in HRegion to keep the global MemStore
> size instead of calculating them every time.
> Why using static variable?
> Since all the HRegion objects in the same JVM share the same memory heap,
> they need to share fate as well.
> The static variable, globalMemStroeSize, naturally shows the total mem usage
> in this shared memory heap for this JVM.
> If multiple RS need to run in the same JVM, they still need only one
> globalMemStroeSize.
> If multiple RS run on different JVMs, everything is fine.
> After changing, in our cases, the avg multiput latency decrease from 60ms to
> 10ms.
> I will submit a patch based on the current trunk.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
