[
https://issues.apache.org/jira/browse/HADOOP-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874662#comment-15874662
]
Steve Loughran commented on HADOOP-14081:
-----------------------------------------
Patch LGTM except for where there's a new test which does a 6MB upload. We dont
need this as the huge block scale tests do large uploads. If we did need it,
then it MUST go into a scale {{ITestHuge*}} test, and so run serialized
explicitly when scale is enabled. And take configurable upload sizes for better
testing. But as it doesn't add anything to our huge tests, it's not needed here
Did you run the scale tests with this patch? I'm going to apply the patch,
revert the test patch and do that test run now.
> S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)
> --------------------------------------------------------------------------
>
> Key: HADOOP-14081
> URL: https://issues.apache.org/jira/browse/HADOOP-14081
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HADOOP-14081.001.patch
>
>
> In {{S3ADataBlocks::ByteArrayBlock}}, data is copied whenever {{startUpload}}
> is called. It might be possible to directly access the byte[] array from
> ByteArrayOutputStream.
> Might have to extend ByteArrayOutputStream and create a method like
> getInputStream() which can return ByteArrayInputStream. This would avoid
> expensive array copy during large upload.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
