roeap commented on issue #2304:
URL: https://github.com/apache/arrow-rs/issues/2304#issuecomment-1206074476
Not 100% sure if I understood the problem with the buckets. Wouldn't the
Store uri just be "s3://<bucket>"? and as such we could register multiple
buckets? I.e. scheme and host are used to store the store.
Maybe the equivalent to `ObjectStoreFromUri` is more the
`ObjectStoreProvider`:
```rs
/// Object store provider can detector an object store based on the url
pub trait ObjectStoreProvider: Send + Sync + 'static {
/// Detector a suitable object store based on its url if possible
/// Return the key and object store
fn get_by_url(&self, url: &Url) -> Option<Arc<dyn ObjectStore>>;
}
```
My main intent was to have something that sanitises user input and handles
all the variation one might refer to a local location on all platforms. This to
me, `ListingTableUrl` is doing that very well. For me a further useful piece
would to interpret the uri and map it to a specific service / implementation -
i.e s3 / azure / ... - based on well-known uris for these technologies, but
more or less just extracting all information that can be gathered from that.
Consumer could then decide what to do with that information.
The mentioned bonus API to unify creation of object stores, is likely best
served using something like the provider (which could have the
`ObjectStoreFromUri` behaviour. I guess that could be part of the object store
trait or a separate `FromUri` trait that does just that.
While a bit worried to overload the `object_store_rs` crate when moving /
implementing these structures, I am slightly leaning towards also moving
registry / provider, since managing multiple separate locations is something
that at least for me comes up regularly.
In short - @wjones127, yes it does match 😄.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
