amoeba opened a new pull request, #44176: URL: https://github.com/apache/arrow/pull/44176
### Rationale for this change WIP changes. No need for review at the moment. This is an experiment which builds on top of and takes a slightly different approach from https://github.com/apache/arrow/pull/34170 which the initial PR intended to resolve https://github.com/apache/arrow/issues/33618. This PR should dramatically speed up Arrow's LocalFS implementation and makes the internals more flexible. ### What changes are included in this PR? - Adds new option `needs_extended_file_info` to FileSelector for tracking whether operations need extended information such as size and mtime or not - Rewrites internals of LocalFS' GetFileInfo/ListDir to use a callback-based implementation ### Are these changes tested? Yes, but there are more tests to write. ### Are there any user-facing changes? TODO -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
