[
https://issues.apache.org/jira/browse/HBASE-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762117#comment-13762117
]
rahul gidwani commented on HBASE-9472:
--------------------------------------
We have a 20GB heap for these regionservers which are used for a cluster which
is primarily doing random reads. We are trying to hold everything in the block
cache. We don't do a lot of writes in comparison to reads in the steady state.
When we ran a job to write quite a few entries to this cluster we noticed the
memstore was greater than the expected size: of .8GB.
With respect to the configuration, our expected block cache upper limit is ~
15GB and our memstore is ~ .8GB
If the memstore is not allowed to be less than .1, we use the default value
which is .4, combined our blockcache + memstore upper limits are: 23GB which
is larger than our total heap of 20GB.
> If the memstore size is under .1 or greater than .9 the memstore size
> defaults to the default memstore size
> -----------------------------------------------------------------------------------------------------------
>
> Key: HBASE-9472
> URL: https://issues.apache.org/jira/browse/HBASE-9472
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.5
> Reporter: rahul gidwani
>
> In HbaseConfiguration.checkForClusterFreeMemoryLimit it does a check to see
> if the blockCache + memstore > .8 this threshold ensures we do not run out of
> memory.
> But MemStoreFlusher.getMemStoreLimit does this check:
> {code}
> if (limit >= 0.9f || limit < 0.1f) {
> LOG.warn("Setting global memstore limit to default of " + defaultLimit +
> " because supplied value outside allowed range of 0.1 -> 0.9");
> effectiveLimit = defaultLimit;
> }
> {code}
> In our cluster we had the block cache set to an upper limit of 0.76 and the
> memstore upper limit was set to 0.04. We noticed the memstore size was
> exceeding the limit we had set and after looking at the getMemStoreLimit code
> it seems that the memstore upper limit is sized to the default value if the
> configuration value is less than .1 or greater than .9. This now makes the
> block cache and memstore greater than our available heap.
> We can remove the check for the greater than 90% of the heap as this can
> never happen due to the check in
> HbaseConfiguration.checkForClusterFreeMemoryLimit()
> This check doesn't seem necessary anymore as we have the HbaseConfiguration
> class checking for the cluster free limit. Am I correct in this assumption?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
