[
https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603789#comment-14603789
]
Abhilash commented on HBASE-13876:
----------------------------------
I was thinking about that too. Rather than giving more weightage to
blockedFlushCount, can we just directly increase(if favorable) memstore size
when we observe blocked flushes ? As even a single blockedFlushCount very
strongly indicates that current upper limit for memstore is not sufficient and
its highly undesirable ?
> Improving performance of HeapMemoryManager
> ------------------------------------------
>
> Key: HBASE-13876
> URL: https://issues.apache.org/jira/browse/HBASE-13876
> Project: HBase
> Issue Type: Improvement
> Components: hbase, regionserver
> Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
> Reporter: Abhilash
> Assignee: Abhilash
> Priority: Minor
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch,
> HBASE-13876-v4.patch, HBASE-13876-v5.patch, HBASE-13876-v6.patch,
> HBASE-13876-v7.patch, HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by
> introducing some more checks. The current checks under which the
> DefaultHeapMemoryTuner works are very rare so I am trying to weaken these
> checks to improve its performance.
> Check current memstore size and current block cache size. For say if we are
> using less than 50% of currently available block cache size we say block
> cache is sufficient and same for memstore. This check will be very effective
> when server is either load heavy or write heavy. Earlier version just waited
> for number of evictions / number of flushes to be zero which are very rare.
> Otherwise based on percent change in number of cache misses and number of
> flushes we increase / decrease memory provided for caching / memstore. After
> doing so, on next call of HeapMemoryTuner we verify that last change has
> indeed decreased number of evictions / flush either of which it was expected
> to do. We also check that it does not make the other (evictions / flush)
> increase much. I am doing this analysis by comparing percent change (which is
> basically nothing but normalized derivative) of number of evictions and
> number of flushes during last two periods. The main motive for doing this was
> that if we have random reads then we will be having a lot of cache misses.
> But even after increasing block cache we wont be able to decrease number of
> cache misses and we will revert back and eventually we will not waste memory
> on block caches. This will also help us ignore random short term spikes in
> reads / writes. I have also tried to take care not to tune memory if do do
> not have enough hints as unnecessary tuning my slow down the system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
