[
https://issues.apache.org/jira/browse/HADOOP-14603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16771190#comment-16771190
]
Sahil Takiar commented on HADOOP-14603:
---------------------------------------
Looked into this a bit while working on HDFS-3246, as far as I can tell, the
AWS SDK does not support {{ByteBuffer}} only {{byte[]}}, which will makes this
optimization less useful since passing in a {{DirectByteBuffer}} would require
copying the data into a {{byte []}} before it can be passed to the AWS SDK. It
looks like the AWS SDK is just reading data from a {{SocketInputStream}} rather
than a {{SocketChannel}}. Whereas HDFS reads from either a
{{ReadableByteChannel}} for remote blocks, or a {{FileChannel}} for lock blocks.
I'm not sure if the AWS SDK plans to migrate to {{ByteBuffer}} anytime soon.
[This|https://aws.amazon.com/blogs/developer/aws-sdk-for-java-2-x-released/]
blog post about the 2.x AWS SDK suggests the 2.x SDK has {{ByteBuffer}}
support, but I haven't looked into it in detail / not sure if we plan to
upgrade to the new SDK anytime soon.
> S3A input stream to support ByteBufferReadable
> ----------------------------------------------
>
> Key: HADOOP-14603
> URL: https://issues.apache.org/jira/browse/HADOOP-14603
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 2.8.1
> Reporter: Steve Loughran
> Priority: Minor
>
> S3AInputStream could support {{ByteBufferReadable,
> HasEnhancedByteBufferAccess}} and the operations to read into byte buffers.
> This is only if we can see a clear performance benefit from doing this or the
> API is being more broadly used
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
