bkietz commented on issue #38309:
URL: https://github.com/apache/arrow/issues/38309#issuecomment-1830865427
The basic sketch of this reorganization is:
- filesystem.h remains part of the public API for libarrow.so, containing
- class FileInfo, class FileSystem
- FileSystem FileSystemFromUri(string)
- void AddFactory(function<FileSystem(string)>)
Some things don't quite fit into this picture, for example I'm not sure how
to handle functions like `S3FileSystem.region()` and `resolve_s3_region()`.
These use the S3 sdk (so we want to build it in the dedicated libarrow_s3fs.so
module). However it provides functionality only relevant to S3 since local file
systems don't have regions (so it cannot be retrieved from a registry of
FromUri functions).
The public API of libarrow.so will need to include some extra interfaces
which can be downcast to or lazily initialized in order to access s3's extra
functionality. (For completeness: we could instead include a soft code escape
hatch in the registry, like a function mapping strings to strings so that s3's
could handle "s3: region_for_bucket my-bucket"->"us-east-2")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
