[
https://issues.apache.org/jira/browse/HADOOP-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16560366#comment-16560366
]
Steve Loughran commented on HADOOP-15409:
-----------------------------------------
Looking into the AWS SDK here, the v2 call does getBucketAcl(); if the user
doesn't have the perms to query the permissions it is still treated as proof of
existence. The big diff here is that permissions are checked early, which is
good.
One thing I worry about: do all S3 re-implementations implement this call? Does
anyone know here?
> S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2
> ----------------------------------------------------------------
>
> Key: HADOOP-15409
> URL: https://issues.apache.org/jira/browse/HADOOP-15409
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Priority: Blocker
>
> in S3AFileSystem.initialize(), we check for the bucket existing with
> verifyBucketExists(), which calls s3.doesBucketExist(). But that doesn't
> check for auth issues.
> s3. doesBucketExistV2() does at least validate credentials, and should be
> switched to. This will help things fail faster
> See SPARK-24000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
