[
https://issues.apache.org/jira/browse/HBASE-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606617#comment-14606617
]
Abhilash commented on HBASE-13980:
----------------------------------
In my opinion rather than giving more weight (it would be difficult in itself
to reason about their relative weights) to blockedFlushCount. We should
directly take tuning decision if we observe blocked flushes in steady state.
Because they are rare and very undesirable.
One complication is that when memory tuner increases block cache size and
decreases memstore size. It might force many blocked flushes but that does not
mean we have to abruptly increase memstore size. So we should do that only in
steady state.
If after reading this you have doubt about how tuner would works then ?
Ans : Suppose in a certain period tuner(1st tuning) increased block cache by 8%
and decreased memstore by same amount. Then in next step(if server is not
completely read heavy, if it is ready heavy then we are good anyways) we will
observe many blocked / unblocked flushes. Then if this is number is really very
high compared to past trend we revert our tuning. But not completely, we
partially revert previous tuning operation(2nd tuning). Overall we have
increased block cache by 4% which tuner wanted to do in 1st tuning.
> Distinguish blockedFlushCount vs unblockedFlushCount when tuning heap memory
> ----------------------------------------------------------------------------
>
> Key: HBASE-13980
> URL: https://issues.apache.org/jira/browse/HBASE-13980
> Project: HBase
> Issue Type: Improvement
> Reporter: Ted Yu
> Assignee: Abhilash
> Priority: Minor
>
> Currently DefaultHeapMemoryTuner doesn't distinguish blockedFlushCount vs
> unblockedFlushCount.
> In its tune() method:
> {code}
> long totalFlushCount = blockedFlushCount+unblockedFlushCount;
> rollingStatsForCacheMisses.insertDataValue(cacheMissCount);
> rollingStatsForFlushes.insertDataValue(totalFlushCount);
> {code}
> Occurrence of blocked flush indicates that upper limit for memstore is not
> sufficient.
> We should either give blockedFlushCount more weight or, take tuning action
> based on blockedFlushCount directly.
> See discussion from tail of HBASE-13876.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
