[
https://issues.apache.org/jira/browse/ARROW-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631653#comment-17631653
]
Joris Van den Bossche commented on ARROW-9538:
----------------------------------------------
This is working for {{pyarrow.fs}}, and so it should also work for all methods
like {{pq.read_table(..)}} or {{ds.dataset(..)}} that use this under the hood,
and so those should support accepting a S3 URI.
{code}
In [1]: import pyarrow.fs
In [2]: pyarrow.fs._resolve_filesystem_and_path("s3://bucket/folder/file.ext")
Out[2]: (<pyarrow._s3fs.S3FileSystem at 0x7f638c707db0>,
'bucket/folder/file.ext')
{code}
Since pyarrow.filesystem module is deprecated and we are not developing that
anymore, going to close this issue.
> [Python] Allow pyarrow.filesystem.resolve_filesystem_and_path to parse S3 URL
> -----------------------------------------------------------------------------
>
> Key: ARROW-9538
> URL: https://issues.apache.org/jira/browse/ARROW-9538
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Affects Versions: 0.17.1
> Reporter: Adam Avilla
> Priority: Minor
> Labels: filesystem
>
> {{pyarrow.filesystem.resolve_filesystem_and_path}} should support a {{where}}
> that is a S3 URL like:
> {code:java}
> s3://bucket/folder/file.ext{code}
> It seems like all the pieces are there but was never developed. If given some
> light guidance I may be able to add the code in a PR.
> Thanks and LMK if this is a crazy request!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
