[
https://issues.apache.org/jira/browse/HADOOP-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312227#comment-15312227
]
Rajesh Balamohan edited comment on HADOOP-13203 at 6/2/16 12:31 PM:
--------------------------------------------------------------------
Attaching .2 patch with test coverage and also fixed
TestS3AInputStreamPerformance.testReadAheadDefault/testReadBigBlocksBigReadAhead.
Ran aws s3a tests locally (everything passes with timeout exception in
TestS3ADeleteManyFiles, TestS3ADeleteFilesOneByOne, DistCp)
was (Author: rajesh.balamohan):
Attaching .2 patch with test coverage and also fixed
TestS3AInputStreamPerformance.testReadAheadDefault/testReadBigBlocksBigReadAhead.
> S3a: Consider reducing the number of connection aborts by setting correct
> length in s3 request
> ----------------------------------------------------------------------------------------------
>
> Key: HADOOP-13203
> URL: https://issues.apache.org/jira/browse/HADOOP-13203
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HADOOP-13203-branch-2-001.patch,
> HADOOP-13203-branch-2-002.patch
>
>
> Currently file's "contentLength" is set as the "requestedStreamLen", when
> invoking S3AInputStream::reopen(). As a part of lazySeek(), sometimes the
> stream had to be closed and reopened. But lots of times the stream was closed
> with abort() causing the internal http connection to be unusable. This incurs
> lots of connection establishment cost in some jobs. It would be good to set
> the correct value for the stream length to avoid connection aborts.
> I will post the patch once aws tests passes in my machine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
