Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-23 Thread OpenInx
Merged the HBASE-21879 back to master branch, Thanks all. On Sun, Jun 23, 2019 at 10:40 AM OpenInx wrote: > Now we have four +1 votes ( all binding). > Will wait a day before the merge, if no objections I will do the merge. > Thanks all for reviewing & checking. > > On Sat, Jun 22, 2019 at

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-22 Thread OpenInx
Now we have four +1 votes ( all binding). Will wait a day before the merge, if no objections I will do the merge. Thanks all for reviewing & checking. On Sat, Jun 22, 2019 at 11:05 PM Anoop John wrote: > +1 for master merge > > Anoop > > On Sat, Jun 22, 2019 at 9:47 AM ramkrishna vasudevan < >

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-22 Thread Anoop John
+1 for master merge Anoop On Sat, Jun 22, 2019 at 9:47 AM ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > +1 to merge to master. Great job Zheng > > On Sat, Jun 22, 2019, 8:41 AM Guanghao Zhang wrote: > > > +1 for merge this to master. > > > > OpenInx 于2019年6月21日周五 下午2:56写道:

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-21 Thread ramkrishna vasudevan
+1 to merge to master. Great job Zheng On Sat, Jun 22, 2019, 8:41 AM Guanghao Zhang wrote: > +1 for merge this to master. > > OpenInx 于2019年6月21日周五 下午2:56写道: > > > Update: > > > > The ByteBuffer pread backport is under reviewing now. > > https://github.com/apache/hadoop/pull/997 > > > > As

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-21 Thread Guanghao Zhang
+1 for merge this to master. OpenInx 于2019年6月21日周五 下午2:56写道: > Update: > > The ByteBuffer pread backport is under reviewing now. > https://github.com/apache/hadoop/pull/997 > > As Hadoop team said, the Hadoop 2.8 will be EOL soon, so our HDFS team > will backport this patch to > branch-2 &

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-21 Thread OpenInx
Update: The ByteBuffer pread backport is under reviewing now. https://github.com/apache/hadoop/pull/997 As Hadoop team said, the Hadoop 2.8 will be EOL soon, so our HDFS team will backport this patch to branch-2 & branch-2.9, we may need to upgrade the hadoop dependencies from 2.8.5 to 2.9.3

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-19 Thread OpenInx
Thanks for your reviewing and flaky test checking, Duo. Will file a separate issue to address your comment if necessary. Thanks. On Wed, Jun 19, 2019 at 9:55 PM 张铎(Duo Zhang) wrote: > +1 from me. > > Left a few comments on github PR, not big problems. And the flaky dashboard > is pretty good.

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-18 Thread Duo Zhang
Good. Will take a look soon. OpenInx 于2019年6月18日周二 下午9:41写道: > > Could please open a PR, just like what I have done for HBASE-21512, so > that others could have a overall view on the modified code? > > OK, created a PR for this: https://github.com/apache/hbase/pull/320 > Thanks for your

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-18 Thread OpenInx
> Could please open a PR, just like what I have done for HBASE-21512, so that others could have a overall view on the modified code? OK, created a PR for this: https://github.com/apache/hbase/pull/320 Thanks for your suggestion, Duo. On Tue, Jun 18, 2019 at 9:24 PM 张铎(Duo Zhang) wrote: > The

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-18 Thread Duo Zhang
The performance number is great. Could please open a PR, just like what I have done for HBASE-21512, so that others could have a overall view on the modified code? Thanks. OpenInx 于2019年6月18日周二 下午6:58写道: > BTW, when testing this branch, we found some performance issues about > HDFS Client:

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-18 Thread OpenInx
BTW, when testing this branch, we found some performance issues about HDFS Client: 1. we reduced the DFS client's heap allocation from 45% to 27% in HDFS-14535 [1]; 2. we also increased get throughput by 17.8% in disabled block cache case in HDFS-14541[2]. In theory, it should also helps

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-17 Thread OpenInx
The HBASE-21879 has lots of changes: 123 files changed, 5833 insertions(+), 3015 deletions(-). Currently we developed this issue based on master branch, and expect to release it in future HBase3.x. Of course, if branch-2 want this feature we can do the backport, should have some conflicts now but

Re: [VOTE] Merge branch HBASE-21879 (Reading HFile's Block to ByteBuffer directly) back to master.

2019-06-17 Thread Guanghao Zhang
This is a improvement not a new feature? So backport to branch-2, too? OpenInx 于2019年6月17日周一 下午2:45写道: > Dear HBase dev: > > In HBASE-21879[1], we redesigned the offheap read path: read the HFileBlock > from HDFS to pooled offheap > ByteBuffers directly, while before HBASE-21879 we just read