[
https://issues.apache.org/jira/browse/ACCUMULO-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219861#comment-14219861
]
John Vines edited comment on ACCUMULO-3349 at 11/21/14 1:39 AM:
----------------------------------------------------------------
So I think if we something like {code}public CacheEntry cacheBlock(String
blockName, byte buf[], boolean inMemory) {
CachedBlock cb = new CachedBlock(blockName, buf, count.incrementAndGet(),
inMemory);
CachedBlock currCb = map.putIfAbsent(blockName, cb);
if (currCb != null) {
stats.duplicateReads();
currCb.access(count.incrementAndGet());
} else {
long newSize = size.addAndGet(cb.heapSize());
elements.incrementAndGet();
if (newSize > acceptableSize() && !evictionInProgress) {
runEviction();
}
}
return cb;
}
{code} it can prevent the size from going off the rails
was (Author: vines):
So I think if we something like {code} public CacheEntry cacheBlock(String
blockName, byte buf[], boolean inMemory) {
CachedBlock cb = new CachedBlock(blockName, buf, count.get(), inMemory);
CachedBlock currCb = map.putIfAbsent(blockName, cb);
if (currCb != null) {
stats.duplicateReads();
currCb.access(count.incrementAndGet());
} else {
cb.access(count.incrementAndGet());
long newSize = size.addAndGet(cb.heapSize());
elements.incrementAndGet();
if (newSize > acceptableSize() && !evictionInProgress) {
runEviction();
}
}
return cb;
}
{code} it can prevent the size from going off the rails
> LruBlockCache is miscalculating sizes
> -------------------------------------
>
> Key: ACCUMULO-3349
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3349
> Project: Accumulo
> Issue Type: Bug
> Affects Versions: 1.6.0
> Reporter: John Vines
>
> On a system, we have index configured to 512MB and data block caches set to
> 1GB. I'm seeing the total size seemingly horrendously miscalculated,
> resulting in a LOT of eviction calls.
> {code}2014-11-17 22:07:54,814 [cache.LruBlockCache] DEBUG: Block cache LRU
> eviction started. Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,817 [cache.LruBlockCache] DEBUG: Block cache LRU eviction
> completed. Freed 102696 bytes. Priority Sizes: Single=186.14093MB
> (195182912), Multi=384.03125MB (402685944),Memory=0.0MB (0)
> 2014-11-17 22:07:54,820 [cache.LruBlockCache] DEBUG: Block cache LRU eviction
> started. Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,823 [cache.LruBlockCache] DEBUG: Block cache LRU eviction
> completed. Freed 102680 bytes. Priority Sizes: Single=186.14093MB
> (195182920), Multi=384.03125MB (402685936),Memory=0.0MB (0)
> 2014-11-17 22:07:54,829 [cache.LruBlockCache] DEBUG: Block cache LRU eviction
> started. Attempting to free 58954734984 bytes
> 2014-11-17 22:07:54,832 [cache.LruBlockCache] DEBUG: Block cache LRU eviction
> completed. Freed 205384 bytes. Priority Sizes: Single=186.043MB (195080224),
> Multi=384.12918MB (402788640),Memory=0.0MB (0)
> {code}
> {code}2014-11-17 22:06:39,439 [cache.LruBlockCache] DEBUG: Cache Stats:
> Sizes: Total=56991.715MB (59760144624), Free=-55967.715MB (-58686402800),
> Max=1024.0MB
> (1073741824), Counts: Blocks=581902, Access=261753272, Hit=202812039,
> Miss=58941233, Evictions=45047347, Evicted=53777417, Ratios: Hit
> Ratio=77.4821341
> 0377502%, Miss Ratio=22.51785844564438%, Evicted/Run=1.1937975883483887,
> Duplicate Reads=4581914
> 2014-11-17 22:07:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes:
> Total=835.4461MB (876028744), Free=-323.4461MB (-339157832), Max=512.0MB
> (536870912), Counts: Blocks=19881, Access=171866619, Hit=171174512,
> Miss=692107, Evictions=300619, Evicted=595529, Ratios: Hit
> Ratio=99.5972990989685%, Miss Ratio=0.402700062841177%,
> Evicted/Run=1.9810091257095337, Duplicate Reads=76697
> 2014-11-17 22:07:39,439 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes:
> Total=56991.418MB (59759831656), Free=-55967.418MB (-58686089832),
> Max=1024.0MB (1073741824), Counts: Blocks=581901, Access=261755962,
> Hit=202813386, Miss=58942576, Evictions=45048521, Evicted=53778759, Ratios:
> Hit Ratio=77.48185992240906%, Miss Ratio=22.518140077590942%,
> Evicted/Run=1.1937963962554932, Duplicate Reads=4581916
> 2014-11-17 22:08:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes:
> Total=835.1727MB (875742096), Free=-323.17273MB (-338871184), Max=512.0MB
> (536870912), Counts: Blocks=19762, Access=171879228, Hit=171187080,
> Miss=692148, Evictions=300642, Evicted=595689, Ratios: Hit
> Ratio=99.5972990989685%, Miss Ratio=0.40269438177347183%,
> Evicted/Run=1.9813898801803589, Duplicate Reads=76697
> {code}
> Currently do not have log history to pinpoint where/where/if there was a slow
> incline to get to 55GB or if it was spontaneous at some discernible point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
