Hi Seems you have write ops happening as you mentioned abt minor compactions. When the compaction happens, the compacted file's blocks will get evicted. Whatever be the value of 'hbase.rs.evictblocksonclose'. This config comes to play when the Store is closed. Means the region movement is happening or split and so a close on stores. Also the table might get disabled or deleted. All such store close cases this config comes to picture. But minor compactions means there will be evictions. These are not via the eviction threads which monitor the less spaces and select LRU blocks for eviction. This is done by the compaction threads. That is why you can see the evict ops (done by Eviction thread) is zero but the #evicted blocks are there. Those might be the blocks of the compacted away files. Hope this helps you to understand what is going on.
-Anoop- On Mon, Feb 19, 2018 at 5:25 AM, Saad Mufti <saad.mu...@gmail.com> wrote: > Sorry I meant BLOCKCACHE => 'false' on the one column family we don't want > getting cached. > > Cheers. > > ---- > Saad > > > On Sun, Feb 18, 2018 at 6:51 PM, Saad Mufti <saad.mu...@gmail.com> wrote: > >> Hi, >> >> We have an HBase system running HBase 1.3.1 on an AWS EMR service. Our >> BucketCache is configured for 400 GB on a set of attached EBS disk volumes, >> with all column families marked for in-memory in their column family >> schemas using INMEMORY => 'true' (except for one column family we only ever >> write to, so we set BUCKETCACHE => 'false' on that one). >> >> Even though all column families are marked INMEMORY, we have the following >> ratios set: >> >> "hbase.bucketcache.memory.factor":"0.8", >> >> "hbase.bucketcache.single.factor":"0.1", >> >> >> "hbase.bucketcache.multi.factor":"0.1", >> >> Currently the bucket cache shows evictions even though it has tons of free >> space. I am trying to understand why we get any evictions at all? We do >> have minor compactions going on, but we have not set >> hbase.rs.evictblocksonclose >> to any value and from looking at the code, it defaults to false. The total >> bucket cache size is nowhere near any of the above limits, in fact on some >> long running servers where we stopped traffic, the cache size went down to >> 0. Which makes me think something is evicting blocks from the bucket cache >> in the background. >> >> You can see a screenshot from one of the regionserver L2 stats UI pages at >> https://imgur.com/a/2ZUSv . Another interesting thing to me on this page >> is that it has non-zero evicted blocks but says Evictions: 0 >> >> Any help understanding this would be appreciated. >> >> ---- >> Saad >> >>