[
https://issues.apache.org/jira/browse/HADOOP-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16711:
------------------------------------
Description:
1. in S3AFileSystem.initialize(), we check for the bucket existing with
verifyBucketExists(), which calls s3.doesBucketExist(). But that doesn't check
for auth issues.
s3. doesBucketExistV2() does at validate credentials, and should be switched
to. This will help things fail faster
2. For performance it would be good to skip verifyBuckets call during S3A
filesystem init(). This would save a call to S3 during init method.
was:When authoritative mode is enabled with s3guard, it would be good to skip
verifyBuckets call during S3A filesystem init(). This would save call to S3
during init method.
> S3A bucket existence checks to support v2 API and "no checks at all"
> --------------------------------------------------------------------
>
> Key: HADOOP-16711
> URL: https://issues.apache.org/jira/browse/HADOOP-16711
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.1
> Reporter: Rajesh Balamohan
> Assignee: Mukund Thakur
> Priority: Minor
> Labels: performance
> Fix For: 3.3.0
>
> Attachments: HADOOP-16711.prelim.1.patch
>
>
> 1. in S3AFileSystem.initialize(), we check for the bucket existing with
> verifyBucketExists(), which calls s3.doesBucketExist(). But that doesn't
> check for auth issues.
> s3. doesBucketExistV2() does at validate credentials, and should be switched
> to. This will help things fail faster
> 2. For performance it would be good to skip verifyBuckets call during S3A
> filesystem init(). This would save a call to S3 during init method.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
