[
https://issues.apache.org/jira/browse/HADOOP-13674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543380#comment-15543380
]
Chris Nauroth commented on HADOOP-13674:
----------------------------------------
Here is the error I see when I attempt to access a bucket in us-west-2, but
with {{fs.s3a.endpoint}} pointing to us-west-1.
{code}
> hadoop fs -D fs.s3a.endpoint=s3-us-west-1.amazonaws.com -ls
> s3a://cnauroth-test-aws-s3a-logs/
ls: getFileStatus on : com.amazonaws.services.s3.model.AmazonS3Exception: The
bucket you are attempting to access must be addressed using the specified
endpoint. Please send all future requests to this endpoint. (Service: Amazon
S3; Status Code: 301; Error Code: PermanentRedirect; Request ID:
EC6C7FCF8B40B27C), S3 Extended Request ID:
EQ1h4MW2CRLV4ZJBGs2xz2CVXwsfGS5X+ByWfyl1tdzbXbf7bFn5DI5pejcWWCmu1/P/uDEOjaU=:
The bucket you are attempting to access must be addressed using the specified
endpoint. Please send all future requests to this endpoint. (Service: Amazon
S3; Status Code: 301; Error Code: PermanentRedirect; Request ID:
EC6C7FCF8B40B27C)
{code}
It says that the endpoint is wrong, but it doesn't say which endpoint is
correct. Turning on debug logging shows that the information does come back in
the HTTP 301 response:
{code}
> hadoop --loglevel DEBUG fs -D fs.s3a.endpoint=s3-us-west-1.amazonaws.com -ls
> s3a://cnauroth-test-aws-s3a-logs/
...
16/10/03 13:35:28 DEBUG http.wire: <<
"<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to
access must be addressed using the specified endpoint. Please send all future
requests to this
endpoint.</Message><Bucket>cnauroth-test-aws-s3a-logs</Bucket><Endpoint>cnauroth-test-aws-s3a-logs.s3-us-west-2.amazonaws.com</Endpoint><RequestId>995927D9C5DD8F90</RequestId><HostId>LK/kvbR/gdnxyr5JXj1L41TOfcO4VBF6MtT8FkwOXXyRdjhasccrHc2bux+b4uHSqJmiBEgHJcI=</HostId></Error>"
...
{code}
It appears that the AWS SDK maps the {{<Endpoint>}} element into the map
returned by
[{{AmazonS3Exception#getAdditionalDetails()}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/AmazonS3Exception.html#getAdditionalDetails--].
We can use that to get the information and put it into the exception thrown
from S3A.
> S3A can provide a more detailed error message when accessing a bucket through
> an incorrect S3 endpoint.
> -------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-13674
> URL: https://issues.apache.org/jira/browse/HADOOP-13674
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Priority: Minor
>
> When accessing the S3 service through a region-specific endpoint, the bucket
> must be located in that region. If the client attempts to access a bucket
> that is not located in that region, then the service replies with a 301
> redirect and the correct region endpoint. However, the exception thrown by
> S3A does not include the correct endpoint. If we included that information
> in the exception, it would make it easier for users to diagnose and fix
> incorrect configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
