[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370564#comment-17370564 ] Zheng Hu commented on HBASE-21946: -- [~weichiu] Please just go ahead, i would like to review your patch for this. Thanks. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370485#comment-17370485 ] Wei-Chiu Chuang commented on HBASE-21946: - I have a patch that uses DFSInputStream.hasCapability() API to detect which read() API to use. Using this approach, I can come up with a solution that is compatible with any Hadoop version >= 2.9.1. [~openinx] can i take over this jira? > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-2 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237623#comment-17237623 ] Andrew Kyle Purtell commented on HBASE-21946: - Moving out of 2.4. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.5.0 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167841#comment-17167841 ] Reid Chan commented on HBASE-21946: --- Looks like it gets blocked by HDFS-3246 which applied to hadoop 3+ only > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145715#comment-17145715 ] Michael Stack commented on HBASE-21946: --- [~openinx] Any more feedback on this feature now you have been running it for a while? If it is working nicely for you, then all the more reason to make it available to others. Thanks. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0 > > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145713#comment-17145713 ] Michael Stack commented on HBASE-21946: --- HDFS-3246 was backported to branch-2.9: 4f09a6bcfd6 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9 HDFS-3246 is only in branch-3.3, not in branch-3.1, our current hadoop version for master branch. I think we should get this in as an option even if it means hackery otherwise the work over in HBASE-21879 is only for those who have patched their own HDFS. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Improvement > Components: Offheaping >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-21946.HBASE-21879.v01.patch, > HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch, > HBASE-21946.HBASE-21879.v04.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[
https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855290#comment-16855290
]
Anoop Sam John commented on HBASE-21946:
pom changes ok?
{code}
return preadWithoutHeapBuffer(buff, dis, position, necessaryLen, extraLen);
212 } catch (UnsupportedOperationException e) {
{code}
Can there be a way with out catch Exception and then go with old way of read.
Like isBBSupported kind API or so available?
> Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once
> HDFS-3246 prepared
> --
>
> Key: HBASE-21946
> URL: https://issues.apache.org/jira/browse/HBASE-21946
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-21946.HBASE-21879.v01.patch,
> HBASE-21946.HBASE-21879.v02.patch, HBASE-21946.HBASE-21879.v03.patch,
> HBASE-21946.HBASE-21879.v04.patch
>
>
> [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread
> in HFileBlock reading. Once it get resolved, we can upgrade the hadoop
> version and do the replacement.
> I think it will be a great p999 latency improvement in 100% Get case, anyway
> file a issue address this firstly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21946) Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once HDFS-3246 prepared
[ https://issues.apache.org/jira/browse/HBASE-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827816#comment-16827816 ] Zheng Hu commented on HBASE-21946: -- We've ported the patch of HDFS-3246 into our HDFS branch, so upload the ByteBuffer pread patch for HBase here. Cann't trigger the hadoop QA now, becauseour hadoop release can not be accessed in public repo now. > Replace the byte[] pread by ByteBuffer pread in HFileBlock reading once > HDFS-3246 prepared > -- > > Key: HBASE-21946 > URL: https://issues.apache.org/jira/browse/HBASE-21946 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-21946.HBASE-21879.v01.patch > > > [~stakiar] is working on HDFS-3246, so now we have to keep the byte[] pread > in HFileBlock reading. Once it get resolved, we can upgrade the hadoop > version and do the replacement. > I think it will be a great p999 latency improvement in 100% Get case, anyway > file a issue address this firstly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
