jorisvandenbossche commented on pull request #8704: URL: https://github.com/apache/arrow/pull/8704#issuecomment-729793788
- Slight downside is that we now do a `get_file_info` twice, but only in the case of a local file path with no filesystem specified (so I think for this case this is less of a concern, but I could still go through hoops to pass it through) - I added the ability to specify a `filesystem` as a string (the functionality @kszucs added to `pyarrow.dataset`) to the general utilities in `pyarrow.fs`, which means that other functions (eg `parquet.read_table`) also gain this functionality. I should still add a test for that (or if this is not desired, I could also hide this behind an optional keyword, but I think it is good to be consistent in what we accept across submodules). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
