bkietz commented on PR #39067: URL: https://github.com/apache/arrow/pull/39067#issuecomment-1922090699
> I think we should provide the building blocks for lazy loading of filesystem libraries I don't think this presents much improvement over using direct factories. The main advantage I see with a FileSystemLoader approach is that initialization code can be run as part of loading the filesystem, including calls to `EnsureS3Finalized()` and dynamic loading of the library. However `EnsureS3Finalized()` could equally be called by the factory, or triggered by dynamic loading at the same time as the factory is registered. As for dynamic loading itself: either the library contains one of the built-in filesystems or it is a custom implementation. For the builtin S3 filesystem, it's reasonable that `libarrow.so` could include the string "libarrow_fs_s3.so" in order to automatically load when an `s3://` uri is encountered. For a custom filesystem `libarrow.so` is unaware of the name of a library containing support for `custom://` uris, and can only automatically load if - some naming convention is adopted so that we can load `libarrow_fs_custom.so` - the name is passed explicitly to `libarrow.so` with `SetLibraryNameForScheme(uri, libname)` which is slightly *more* complicated than passing the name to `LoadLibrary(libname)` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
