jorisvandenbossche commented on a change in pull request #8539:
URL: https://github.com/apache/arrow/pull/8539#discussion_r516047581
##########
File path: python/pyarrow/filesystem.py
##########
@@ -446,10 +446,6 @@ def _ensure_filesystem(fs):
for mro in inspect.getmro(fs_type):
if mro.__name__ == 'S3FileSystem':
return S3FSWrapper(fs)
- # In case its a simple LocalFileSystem (e.g. dask) use native arrow
- # FS
- elif mro.__name__ == 'LocalFileSystem':
- return LocalFileSystem._get_instance()
Review comment:
The reason this now gives troubles (while this code was already there
for more than 2 years), is because with pyarrow >= 2.0, `fsspec` will no longer
inherit from pyarrow.filesystem.FileSystem, and so this path was only taken if
the passed filesystem was *not* a pyarrow subclass (so before, in pyarrow.fs,
we simply didn't take this code path for fsspec filesystems)
##########
File path: python/pyarrow/filesystem.py
##########
@@ -446,10 +446,6 @@ def _ensure_filesystem(fs):
for mro in inspect.getmro(fs_type):
if mro.__name__ == 'S3FileSystem':
return S3FSWrapper(fs)
- # In case its a simple LocalFileSystem (e.g. dask) use native arrow
- # FS
- elif mro.__name__ == 'LocalFileSystem':
- return LocalFileSystem._get_instance()
Review comment:
I *think* it is fine to simply remove this code, as in practice this was
also never run anymore with previous pyarrow releases. I suppose this was only
ever reached with old fsspec versions that did not yet subclass pyarrow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
