[
https://issues.apache.org/jira/browse/ARROW-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche updated ARROW-12299:
------------------------------------------
Summary: [Python] pq.write_to_dataset does not recognize S3FileSystem
(was: S3FileSystem is an Unrecognized filesystem)
> [Python] pq.write_to_dataset does not recognize S3FileSystem
> ------------------------------------------------------------
>
> Key: ARROW-12299
> URL: https://issues.apache.org/jira/browse/ARROW-12299
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 3.0.0
> Reporter: Samuel Sanders
> Priority: Major
>
> {code:java}
> pq.write_to_dataset(pa.concat_tables(pa_tables),
> File "C:\venv\*\lib\site-packages\pyarrow\parquet.py", line 1914, in
> write_to_dataset
> fs, root_path = legacyfs.resolve_filesystem_and_path(root_path,
> filesystem)
> File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 474, in
> resolve_filesystem_and_path
> filesystem = _ensure_filesystem(filesystem)
> File "C:\venv\*\lib\site-packages\pyarrow\filesystem.py", line 457, in
> _ensure_filesystem
> raise OSError('Unrecognized filesystem: {}'.format(fs_type))
> OSError: Unrecognized filesystem: <class 'pyarrow._s3fs.S3FileSystem'>
> {code}
> Creating the S3FileSystem these two ways produced the above error when
> invoking parquet.write_to_dataset with filesystem=s3_filesystem:
> {code:java}
> s3_filesystem = file_system.S3FileSystem(region='us-east-1')
> s3_filesystem, path =
> file_system.FileSystem.from_uri("s3://{0}".format(PARQUET_BUCKET))
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
