westonpace commented on issue #33618: URL: https://github.com/apache/arrow/issues/33618#issuecomment-1379550834
I don't think we can implement this without changes to the filesystem API. We have: ``` virtual Result<FileInfoVector> GetFileInfo(const std::vector<std::string>& paths); ``` So, we don't know at call time if the user is just after the file type or if they are after the type and the size. `FileInfo` is not virtual so we can't lazily call `stat` in the same way that `std::filesystem` does. However, I believe this would speed up dataset discovery (assuming the user is not asking to inspect all fragments) and so I am interested :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
