Rajesh Balamohan created HADOOP-13203:
-----------------------------------------
Summary: S3a: Consider reducing the number of connection aborts by
setting correct length in s3 request
Key: HADOOP-13203
URL: https://issues.apache.org/jira/browse/HADOOP-13203
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Rajesh Balamohan
Priority: Minor
Currently file's "contentLength" is set as the "requestedStreamLen", when
invoking S3AInputStream::reopen(). As a part of lazySeek(), sometimes the
stream had to be closed and reopened. But lots of times the stream was closed
with abort() causing the internal http connection to be unusable. This incurs
lots of connection establishment cost in some jobs. It would be good to set
the correct value for the stream length to avoid connection aborts.
I will post the patch once aws tests passes in my machine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
