natbprice opened a new issue, #37816:
URL: https://github.com/apache/arrow/issues/37816
### Describe the bug, including details regarding any error messages,
version, and platform.
I have a multi-file parquet dataset in cloud storage mounted as a local
directory. This directory appears as a local directory when calling arrow.
In R, when I call `open_dataset("nyc-taxi", unify_schemas=FALSE)` it lists
all the files, but then also opens each file which triggers all files to be
downloaded locally. The desired behavior would be to list all the files and
only open 1 file in order to determine the schema.
In Python, when I call `ds.dataset("nyc-taxi", partitioning="hive")` it only
downloads (i.e., opens) 1 file to learn the schema as expected.
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
