Ram: As Arul indicated, HDP is being used where there're a lot of backports to 1.1.2 baseline.
Arul: I suggest going thru vendor's channel first. Cheers On Tue, Sep 12, 2017 at 10:11 AM, ramkrishna vasudevan < [email protected]> wrote: > Why Master also accepts MaxDirectMemory config is probably because in 1.1 > onwards we treat HMaster as HregionServer but it does the region managment > also. Am not very sure in 1.1.2 is HMaster allowed to host regions ? If so > you need to configure MaxDirectMemory, if not probably we can see how we > can avoid it. We need to raise a JIRA for that. > > Coming to the size of MaxDirectMemory less than bucket cache - I am > wondering was there a bug previously? Because assume you need 25G offheap > bucket cache then atleast 25G MaxDirectMemory is a must. Ideally you may > need some delta more than 25G. > > 0.98 is obsolete now so its better we go with how 1.1.2 works. But if you > feel there is a documentation that could help I think it is better we > provide one so that users like you are not affected. > > Regards > Ram > > > > > On Tue, Sep 12, 2017 at 10:22 PM, Arul Ramachandran <[email protected]> > wrote: > > > Thank you, Ram. > > > > >> So are you trying to use bucket cache feature in offheap mode with > > 1.1.2? > > > > Yes. > > > > >> So even in 0.98 you were using bucket cache in offheap mode? > > > > Yes, but it is a different hbase cluster and it run 0.98. The one I am > > trying to setup offheap cache is hbase 1.1.2 > > > > > > -arul > > > > > > On Tue, Sep 12, 2017 at 9:34 AM, ramkrishna vasudevan < > > [email protected]> wrote: > > > > > Hi > > > > > > So are you trying to use bucket cache feature in offheap mode with > 1.1.2? > > > If so then it is needed that the MaxDirectMemory is greater than the > > > offheap bucket cache size. > > > > > > If you are not using in offheap mode then probably there is no need for > > > MaxDirectMemory to be greater than bucket cache size. > > > > > > >>in Hbase 0.98, I had to set -XX:MaxDirectMemorySize less than > > > hbase.bucket.cache.size > > > So even in 0.98 you were using bucket cache in offheap mode? > > > > > > Regards > > > Ram > > > > > > On Tue, Sep 12, 2017 at 9:40 PM, Ted Yu <[email protected]> wrote: > > > > > > > Looks like the config you meant should be hbase.bucketcache.size > > > > > > > > As the refguide says: > > > > > > > > A float that EITHER represents a percentage of total heap memory size > > to > > > > give to the cache (if < 1.0) OR, it is the total capacity in > megabytes > > of > > > > BucketCache. Default: 0.0 > > > > > > > > If you specify the size as capacity, -XX:MaxDirectMemorySize should > be > > > > bigger than the capacity. > > > > > > > > For #2, did you encounter some error ? > > > > > > > > Cheers > > > > > > > > On Tue, Sep 12, 2017 at 8:52 AM, Arul Ramachandran < > [email protected] > > > > > > > wrote: > > > > > > > > > In HBase 1.1.2, I am setting up bucket cache. I set MaxDirectMemory > > > size > > > > > greater than hbase.bucket.cache.size - only then it would work. > > > > > > > > > > 1) Does HBASE_REGIONSERVER_OPTS -XX:MaxDirectMemorySize needs to be > > > > greater > > > > > than hbase.bucket.cache.size? > > > > > 2) It seems with hbase 1.1.2, HBASE_MASTER_OPTS also needs the > > > > > -XX:MaxDirectMemorySize setting? > > > > > > > > > > IIRC, in Hbase 0.98, I had to set -XX:MaxDirectMemorySize less than > > > > > hbase.bucket.cache.size --and-- I did not have to set > > > > > -XX:MaxDirectMemorySize for HBASE_MASTER_OPTS. > > > > > > > > > > > > > > > Thanks, > > > > > Arul > > > > > > > > > > > > > > >
