[
https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581207#comment-14581207
]
Abhilash commented on HBASE-13876:
----------------------------------
Thanks a lot for the reviews.
Point of adding up percent changes was that in most of cases when we try to
decrease evictCounts by increasing blockCache size the number of flushes
increases so that would help to create balance. For example suppose after
increasing block cache size we have significant increase in number of flushes (
more than tolerance level ) but at the same time we have very huge decrease in
cache misses ( maybe we were using cache size just less than working set size )
separating them would lead to no tuner operation in such cases.
Trying to implement other suggestions.
> Improving performance of HeapMemoryManager
> ------------------------------------------
>
> Key: HBASE-13876
> URL: https://issues.apache.org/jira/browse/HBASE-13876
> Project: HBase
> Issue Type: Improvement
> Components: hbase, regionserver
> Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
> Reporter: Abhilash
> Assignee: Abhilash
> Priority: Minor
> Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch,
> HBASE-13876-v4.patch, HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by
> introducing some more checks. The current checks under which the
> DefaultHeapMemoryTuner works are very rare so I am trying to weaken these
> checks to improve its performance.
> Check current memstore size and current block cache size. If we are using
> less than 50% of currently available block cache size we say block cache is
> sufficient and same for memstore. This check will be very effective when
> server is either load heavy or write heavy. Earlier version just waited for
> number of evictions / number of flushes to be zero which are very rare.
> Otherwise based on percent change in number of cache misses and number of
> flushes we increase / decrease memory provided for caching / memstore. After
> doing so, on next call of HeapMemoryTuner we verify that last change has
> indeed decreased number of evictions / flush ( combined). I am doing this
> analysis by comparing percent change (which is basically nothing but
> normalized derivative) of number of evictions and number of flushes during
> last two periods. The main motive for doing this was that if we have random
> reads then we will be having a lot of cache misses. But even after increasing
> block cache we wont be able to decrease number of cache misses and we will
> revert back and eventually we will not waste memory on block caches. This
> will also help us ignore random short term spikes in reads / writes.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
