[
https://issues.apache.org/jira/browse/ARROW-13237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17462693#comment-17462693
]
Antoine Pitrou commented on ARROW-13237:
----------------------------------------
Note that there is automatic region resolution when you use a URI:
{code:python}
>>> from pyarrow.fs import _resolve_filesystem_and_path
>>> fs, path = _resolve_filesystem_and_path("s3://ursa-labs-taxi-data")
>>> fs
<pyarrow._s3fs.S3FileSystem at 0x7f348cd0b7b0>
>>> fs.region
'us-east-2'
>>> path
'ursa-labs-taxi-data'
{code}
> [C++] S3 FileSystem doesn't seem to handle redirects
> ----------------------------------------------------
>
> Key: ARROW-13237
> URL: https://issues.apache.org/jira/browse/ARROW-13237
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 4.0.1
> Reporter: Alessandro Molina
> Priority: Major
> Fix For: 8.0.0
>
>
> In some conditions AWS S3 seems to respond with a redirect, but Arrow seems
> to consider it an error instead of following the redirect.
> For example see
> {code}
> s3, bucket =
> fs.FileSystem.from_uri("s3://ursa-labs-taxi-data/?region=us-east-1")
> print(s3.get_file_info(fs.FileSelector(bucket+"/2011", recursive=True)))
> {code}
> The error that you get is
> {code}
> OSError: When listing objects under key '2011' in bucket
> 'ursa-labs-taxi-data': AWS Error [code 100]: Unable to parse ExceptionName:
> PermanentRedirect Message: The bucket you are attempting to access must be
> addressed using the specified endpoint. Please send all future requests to
> this endpoint.
> {code}
> It should probably follow the `PermanentRedirect` instead of choking over it
> IT is also possible to reproduce it using
> {code}
> from pyarrow import fs
> s3 = fs.SubTreeFileSystem("ursa-labs-taxi-data", fs.S3FileSystem())
> print(s3.get_file_info(fs.FileSelector("2011", recursive=True)))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
