Hi, Stack Thanks for your reply and also thanks very much for your reply to my previous mail "Questions about BucketCache".
The hbase.bucketcache.bucket.sizes property takes effect. I did not notice that in your first mail you have told me the name should be hbase.bucketcache.bucket.sizes, instead of hbase.bucketcache.sizes. I noticed this point until your last mail. It's my fault. Thanks for your patient. As to the relationship between HBASE_OFFHEAPSIZE and -XX:MaxDirectMemorySize, I followed your advice to find in the bin/hbase the statements that contain HBASE_OFFHEAPSIZE, but I found that there isn't any statement which contains HBASE_OFFHEAPSIZE. I also tried "bin/hbase-daemon.sh" and "bin/hbase-config.sh", they don't contain HBASE_OFFHEAPSIZE either. So I still don't know their relationship. My HBase version is 0.98.10. Is HBASE_OFFHEAPSIZE not used in this version ? As to "the pure secondary cache" or "bypass CBC", I mean use BucketCache as a strict L2 cache to the L1 LruBlockCache, ie the Raw L1+L2. The point comes from the reference guide of Apache HBase, which says : "It is possible to deploy an L1+L2 setup where we bypass the CombinedBlockCache policy and have BucketCache working as a strict L2 cache to the L1 LruBlockCache. For such a setup, set CacheConfig.BUCKET_CACHE_COMBINED_KEY to false. In this mode, on eviction from L1, blocks go to L2. When a block is cached, it is cached first in L1. When we go to look for a cached block, we look first in L1 and if none found, then search L2. Let us call this deploy format, Raw L1+L2." I want to configure this kind of cache not because the CBC poliy is not good, but because I am a tech-leader in a bank. I need to compare these two kinds of cache to make a decision for our differenct kinds of apps. The reference guide said I can configure in "CacheConfig.BUCKET_CACHE_COMBINED_KEY", but is there anyway I can configure in hbase-site.xml? Many Thanks! ------------------ 原始邮件 ------------------ 发件人: "Stack";<[email protected]>; 发送时间: 2015年3月2日(星期一) 下午2:30 收件人: "Hbase-User"<[email protected]>; 主题: Re: BucketCache Configuration Problem On Sun, Mar 1, 2015 at 12:57 AM, donhoff_h <[email protected]> wrote: > Hi, experts > > I am using HBase0.98.10 and have 3 problems about BucketCache > configuration. > > First: > I read the reference guide of Apache HBase to learn how to config > BucketCache. I find that when using offheap BucketCache, the reference > guide says that I should config the HBASE_OFFHEAPSIZE , it also says that > I should config -XX:MaxDirectMemorySize. Since these two parameters are > both related to the DirectMemory, I am confused which one should I > configure? > > See bin/hbase how HBASE_OFFHEAPSIZE gets interpolated as value of the -XX:MaxDirectMemorySize passed to java (so set HBASE_OFFHEAPSIZE) (will fix the doc so more clear) > Second: > I want to know how to configure the BucketCache as a pure secondary > cache, which I mean to bypass the CombinedBlockCache policy. I tried to > configure as following , but when I go to the regionserver's webUI, I > found it says "No L2 deployed" > > hbase.bucketcache.ioengine=offheap > hbase.bucketcache.size=200 > hbase.bucketcache.combinedcache.enabled=false > > What do you mean by pure secondary cache? Which block types do you want in the bucketcache? Why bypass CBC? We've been trying to simplify bucketcache deploy. Part of this streamlining has been removing the myriad options because they tend to confuse and give user a few simple choices. Do the options not work for you? > Third: > I made the following configuration to set the Bucket Sizes. But from > regionserver's WebUI, I found that (4+1)K and (8+1)K sizes are used, > (64+1)K sizes are not used. What's wrong with my configuration? > hbase.bucketcache.ioengine=offheap > hbase.bucketcache.size=200 > hbase.bucketcache.combinedcache.enabled=true > hbase.bucketcache.sizes=65536 hfile.block.cache.sizes=65536 I configured > these two both for I don't know which one is in use now. > > As per previous mail (and HBASE-13125), hfile.block.cache.sizes has no effect in 0.98.x. Also per our previous mail "Questions about BucketCache", isn't it hbase.bucketcache.bucket.sizes that you want? You have tried the defaults and they do not fit your access patten? Yours, St.Ack > Many Thanks!
