[
https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782101#comment-16782101
]
Sahil Takiar commented on HDFS-3246:
------------------------------------
[~anoop.hbase] {quote} When calling this API with buf remaining size of n and
the file is having data size > n after given position, is it guaranteed to read
the whole n bytes into BB in one go? Just wanted to confirm. Thanks. {quote}
Unfortunately, the existing APIs aren't clear on this behavior. The
{{ByteBufferPositionedReadable}} interface is meant to follow the same
semantics as {{PositionedReadable}} and {{ByteBufferReadable}}.
{{PositionedReadable}} says it "Read[s] up to the specified number of bytes"
and {{ByteBufferReadable}} says it "Reads up to buf.remaining() bytes". In
practice, it looks like pread in {{DFSInputStream}} follows the behavior you
have described, e.g. it either reads until {{ByteBuffer#hasRemaining()}}
returns false, or there are no more bytes in the file.
{{ByteBufferPositionedReadable}} should follow the same behavior for
{{DFSInputStream}}.
[~jojochuang] thanks for the review comments. I've got this implemented for
{{CryptoInputStream}} as well and will post a patch soon.
As far as testing goes. I've tested the libhdfs path via Impala on a real
cluster and everything seemed to be working as expected (have not tested
against an encrypted HDFS cluster).
Will post an updated patch shortly.
> pRead equivalent for direct read path
> -------------------------------------
>
> Key: HDFS-3246
> URL: https://issues.apache.org/jira/browse/HDFS-3246
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, performance
> Affects Versions: 3.0.0-alpha1
> Reporter: Henry Robinson
> Assignee: Sahil Takiar
> Priority: Major
> Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch,
> HDFS-3246.003.patch, HDFS-3246.004.patch
>
>
> There is no pread equivalent in ByteBufferReadable. We should consider adding
> one. It would be relatively easy to implement for the distributed case
> (certainly compared to HDFS-2834), since DFSInputStream does most of the
> heavy lifting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
