[
https://issues.apache.org/jira/browse/HADOOP-12353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mariusz Strzelecki updated HADOOP-12353:
----------------------------------------
Description:
Current implementation of NativeS3FileSystem.java uses RetryProxy that retries
exceptions that may occur on network communication with S3 API, but these
exceptions must be exact instances of IOException:
https://github.com/apache/hadoop/blob/master/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java#L349
Our tests show that HttpClient throws IOException subclasses which are not
handled by Proxy.
Additionally, not all methods that call S3 API are listed to be handled, i.e.
storeEmptyFile and retrieveMetadata are missing.
was:
Current implementation of NativeS3FileSystem.java uses RetryProxy that retries
exceptions that may occur on network communication with S3 API, but these
exceptions must be exact instances of IOException:
https://github.com/apache/hadoop/blob/master/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java#L349
Our tests show that HttpClient throws IOException subclasses which are not
handled by Proxy.
> S3 Native filesystem does not retry all connection failures
> -----------------------------------------------------------
>
> Key: HADOOP-12353
> URL: https://issues.apache.org/jira/browse/HADOOP-12353
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Mariusz Strzelecki
> Assignee: Mariusz Strzelecki
> Priority: Minor
> Attachments: HADOOP-12353.001.patch
>
>
> Current implementation of NativeS3FileSystem.java uses RetryProxy that
> retries exceptions that may occur on network communication with S3 API, but
> these exceptions must be exact instances of IOException:
> https://github.com/apache/hadoop/blob/master/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java#L349
> Our tests show that HttpClient throws IOException subclasses which are not
> handled by Proxy.
> Additionally, not all methods that call S3 API are listed to be handled, i.e.
> storeEmptyFile and retrieveMetadata are missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
