kszucs commented on code in PR #45089:
URL: https://github.com/apache/arrow/pull/45089#discussion_r2177740838
##########
docs/source/python/filesystems.rst:
##########
@@ -388,6 +388,32 @@ Then all the functionalities of :class:`FileSystem` are
accessible::
ds.dataset("data/", filesystem=pa_fs)
+Using fsspec-compatible filesystem URIs
+---------------------------------------
+
+PyArrow can automatically instantiate fsspec filesystems by prefixing the URI
+scheme with ``fsspec+``. This allows you to use the fsspec-compatible
+filesystems directly with PyArrow's IO functions without needing to manually
+create a filesystem object. Example writing and reading a Parquet file
+using an in-memory filesystem provided by `fsspec`_::
+
+ import pyarrow as pa
+ import pyarrow.parquet as pq
+
+ table = pa.table({'a': [1, 2, 3]})
+ pq.write_table(table, "fsspec+memory://path/to/my_table.parquet")
+ pq.read_table("fsspec+memory://path/to/my_table.parquet")
+
+Example reading parquet file from GitHub directly::
+
+
pq.read_table("fsspec+github://apache:arrow-testing@/data/parquet/alltypes-java.parquet")
+
+Hugging Face's sceheme explicitly allowed as a shortcut without needing to
prefix
+with ``fsspec+``. This is useful for reading datasets hosted on Hugging Face::
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
