The HBASE-21879 has lots of changes: 123 files changed, 5833 insertions(+), 3015 deletions(-). Currently we developed this issue based on master branch, and expect to release it in future HBase3.x. Of course, if branch-2 want this feature we can do the backport, should have some conflicts now but I don't think it would be hard to fix because I believe the branch-2 shouldn't have so much diff with master now (at least in read path). The first priority thing for now, I think it would be merging the HBASE-21879 branch to master branch before diverging. After that, I can do the backport.
Thanks for your suggestion, Guanghao ! On Tue, Jun 18, 2019 at 11:39 AM Guanghao Zhang <zghao...@gmail.com> wrote: > This is a improvement not a new feature? So backport to branch-2, too? > > OpenInx <open...@gmail.com> 于2019年6月17日周一 下午2:45写道： > > > Dear HBase dev: > > > > In HBASE-21879[1], we redesigned the offheap read path: read the > HFileBlock > > from HDFS to pooled offheap > > ByteBuffers directly, while before HBASE-21879 we just read the > HFileBlock > > to heap which would still lead > > to high GC pressure. > > > > After few months of development and testing, all subtasks have been > > resovled now except the HBASE-21946[2] > > (It depends on HDFS-14483[3] and our HDFS teams are working on this, we > > expect the HDFS-14483 to be included > > in hadoop 2.8.6 and after that the HBASE-21946 will get resolved). we > think > > the feature is stable enough now and it's > > time to merge branch HBASE-21879 back to master now. > > > > We have designed 3 test cases to prove the performance improvment with > > HBASE-21879: > > 1. Disabled BlockCache, which means the cacheHitRatio is 0%; > > 2. CacheHitRatio~65%; > > 3. CachehHitRatio~100%; > > > > In our performance results[4], we can see that: the case#1 have an great > > performance improvement > > ( > > *throughput increased about 17%, heap allocation decreased about 95%, > Young > > generaion size decreased about 81.7%*), that's because after HBASE-21879 > > all reads will allocate from pooled offheap bytebuffers > > and almost no heap allocation, while before HBASE-21879 the read path > will > > create so many heap allocations. > > On the other hand, from the testing results of case#2 and case#3 we can > > also see that: > > > > *As the cacheHitRatioincreasing, the difference between > before-HBASE-21879 > > and after-HBASE-21879 will decrease, when cacheHitRatio is 100%, they > > almost have no much difference in both throughput and latency.* > > > > For more details please see the document[4]. Thanks > > Anoop/Ram/DuoZhang/Stack/GuanghaoZhang very much > > for your meticulous work (Suggession, discussion, patch reviewing, doc > > reviewing etc). > > > > Please vote > > > > [] +1 > > [] +0/-0 > > [] -1 Do not merge the branch back because... > > > > Thanks. Any suggestions are welcomed. > > > > [1] https://issues.apache.org/jira/browse/HBASE-21879 > > [2] https://issues.apache.org/jira/browse/HBASE-21946 > > [3] https://issues.apache.org/jira/browse/HDFS-14483 > > [4] > > > > > https://docs.google.com/document/d/1xSy9axGxafoH-Qc17zbD2Bd--rWjjI00xTWQZ8ZwI_E > > >
