[
https://issues.apache.org/jira/browse/HDFS-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henry Robinson updated HDFS-3110:
---------------------------------
Attachment: HDFS-3110.2.patch
Patch addressing Todd's concerns.
I added a 'flags' field to hdfsFile that has a bit set if a direct read is
supported. I detect that by trying to issue a 0-byte read when the file is
created. If an exception is thrown, the flag is cleared, otherwise it is set.
Once the flag is set, all subsequent hdfsRead calls will be diverted to
hdfsReadDirect.
An alternative is to use reflection to grab the input stream inside
FsDataInputStream and use reflection to look for ByteBufferReadable, but that
feels a little fragile (and complex to do in C); plus if some FS implements
read(ByteBuffer) only to stub it out with a UnsupportedOperationException or
similar, reads would never work correctly.
> libhdfs implementation of direct read API
> -----------------------------------------
>
> Key: HDFS-3110
> URL: https://issues.apache.org/jira/browse/HDFS-3110
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: libhdfs
> Reporter: Henry Robinson
> Assignee: Henry Robinson
> Fix For: 0.24.0
>
> Attachments: HDFS-3110.0.patch, HDFS-3110.1.patch, HDFS-3110.2.patch
>
>
> Once HDFS-2834 gets committed, we can add support for the new API to libhdfs,
> which leads to significant performance increases when reading local data from
> C.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
