[
https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580793#comment-14580793
]
Ted Yu commented on HBASE-13876:
--------------------------------
{code}
76 || context.getCurMemStoreUsed()*2 <
context.getCurMemStoreSize();
{code}
How is 50% determined ? In other words, if 51% of memstore is used, why would
we consider memstore size insufficient ?
When stepDirectionIndicator is true, block cache size is to be increased. Would
using an enum make the code more readable ?
{code}
94 float percentChangeInEvictCount =
(float)(evictCount-prevEvictCount)/(float)(evictCount);
{code}
Should the right hand side be
'(float)(evictCount-prevEvictCount)/(float)(prevEvictCount)' ?
Similar comment applies to the calculation of percentChangeInFlushes and
percentChangeInMisses.
{code}
110 // try to balance changes in number of cache misses and
flushes
111 stepDirectionIndicator = percentChangeInMisses >
percentChangeInFlushes;
{code}
What if percentChangeInMisses and percentChangeInFlushes are both negative ? In
that case there is no need to change direction, right ?
Would it make sense to introduce NEUTRAL direction which doesn't require tuning
for the current iteration ?
Putting the patch on review board would make reviewing easier.
> Improving performance of HeapMemoryManager
> ------------------------------------------
>
> Key: HBASE-13876
> URL: https://issues.apache.org/jira/browse/HBASE-13876
> Project: HBase
> Issue Type: Improvement
> Components: hbase, regionserver
> Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
> Reporter: Abhilash
> Assignee: Abhilash
> Priority: Minor
> Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch,
> HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by
> introducing some more checks. The current checks under which the
> DefaultHeapMemoryTuner works are very rare so I am trying to weaken these
> checks to improve its performance.
> Check current memstore size and current block cache size. If we are using
> less than 50% of currently available block cache size we say block cache is
> sufficient and same for memstore. This check will be very effective when
> server is either load heavy or write heavy. Earlier version just waited for
> number of evictions / number of flushes to be zero which are very rare.
> Otherwise based on percent change in number of cache misses and number of
> flushes we increase / decrease memory provided for caching / memstore. After
> doing so, on next call of HeapMemoryTuner we verify that last change has
> indeed decreased number of evictions / flush ( combined). I am doing this
> analysis by comparing percent change (which is basically nothing but
> normalized derivative) of number of evictions and number of flushes during
> last two periods. The main motive for doing this was that if we have random
> reads then we will be having a lot of cache misses. But even after increasing
> block cache we wont be able to decrease number of cache misses and we will
> revert back and eventually we will not waste memory on block caches. This
> will also help us ignore random short term spikes in reads / writes.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
