[
https://issues.apache.org/jira/browse/ARROW-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631708#comment-17631708
]
Miles Granger commented on ARROW-18272:
---------------------------------------
Just noting that this works for {{read_metadata}} because, while it takes an
optional {{filesystem}}, it calls [_resolve_filesystem_and_path |
https://github.com/apache/arrow/blob/18326f96834169b4573f3d4f1c0d46f1e8499295/python/pyarrow/parquet/core.py#L3476]
while {{ParquetFile}} does not.
If you want to make it work right now, one could do something like:
{code:python}
path = 'mybucket/abc/d.parquet'
uri = f'gs://{path}'
filesystem = pyarrow.fs.FileSystem.from_uri(uri)
with filesystem.open_input_stream(path) as f:
parquet_file = pq.ParquetFile(f)
{code}
cc [~apitrou] as part of this, I could add a similar
{{_resolve_filesystem_and_path}} inside of {{ParquetFile}}?
> [pyarrow] ParquetFile does not recognize GCS cloud path as a string
> -------------------------------------------------------------------
>
> Key: ARROW-18272
> URL: https://issues.apache.org/jira/browse/ARROW-18272
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 10.0.0
> Reporter: Zepu Zhang
> Priority: Minor
>
> I have a Parquet file at
>
> path = 'gs://mybucket/abc/d.parquet'
>
> `pyarrow.parquet.read_metadata(path)` works fine.
>
> `pyarrow.parquet.ParquetFile(path)` raises "Failed to open local file
> 'gs://mybucket/abc/d.parquet'.
>
> Looks like ParquetFile misses the path resolution logic found in
> `read_metadata`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
