[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782101#comment-16782101 ]
Sahil Takiar commented on HDFS-3246: ------------------------------------ [~anoop.hbase] {quote} When calling this API with buf remaining size of n and the file is having data size > n after given position, is it guaranteed to read the whole n bytes into BB in one go? Just wanted to confirm. Thanks. {quote} Unfortunately, the existing APIs aren't clear on this behavior. The {{ByteBufferPositionedReadable}} interface is meant to follow the same semantics as {{PositionedReadable}} and {{ByteBufferReadable}}. {{PositionedReadable}} says it "Read[s] up to the specified number of bytes" and {{ByteBufferReadable}} says it "Reads up to buf.remaining() bytes". In practice, it looks like pread in {{DFSInputStream}} follows the behavior you have described, e.g. it either reads until {{ByteBuffer#hasRemaining()}} returns false, or there are no more bytes in the file. {{ByteBufferPositionedReadable}} should follow the same behavior for {{DFSInputStream}}. [~jojochuang] thanks for the review comments. I've got this implemented for {{CryptoInputStream}} as well and will post a patch soon. As far as testing goes. I've tested the libhdfs path via Impala on a real cluster and everything seemed to be working as expected (have not tested against an encrypted HDFS cluster). Will post an updated patch shortly. > pRead equivalent for direct read path > ------------------------------------- > > Key: HDFS-3246 > URL: https://issues.apache.org/jira/browse/HDFS-3246 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, performance > Affects Versions: 3.0.0-alpha1 > Reporter: Henry Robinson > Assignee: Sahil Takiar > Priority: Major > Attachments: HDFS-3246.001.patch, HDFS-3246.002.patch, > HDFS-3246.003.patch, HDFS-3246.004.patch > > > There is no pread equivalent in ByteBufferReadable. We should consider adding > one. It would be relatively easy to implement for the distributed case > (certainly compared to HDFS-2834), since DFSInputStream does most of the > heavy lifting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org