thisisnic commented on issue #30481:
URL: https://github.com/apache/arrow/issues/30481#issuecomment-6013846075
I think we may still need a fix on the Arrow side. Excuse the AI-generated
response below, I'll dig properly into the details when I get around to looking
into this properly and familiarising myself with that bit of the codebase, but
gave the below a quick skim and will need to investigate more before closing.
:robot: :
> Thanks, that explains the retry-works report, but it's a different bug.
fsspec/gcsfs#312 was
about info()/isdir() giving inconsistent results across calls, and the
proposed fix
(fsspec/gcsfs#313) was never merged anyway. The problem here is in find(),
which pyarrow
uses for dataset discovery. In the current gcsfs release (2026.10.0),
_find() returns
listing entries with type: "file" and never applies the
_is_directory_marker check that
info() and ls() use, so .../partition_var=x/ still comes back as a file and
the reprex
above still fails. The Arrow-side fix is still needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
