The proposal [1] to switch DataFusion to object_store_rs is looking like it will be merged in the coming few days, so any concerns/reservations please speak up!
Regarding release management, my personal preference would be for it to managed and released as part of the arrow-rs release process. This will keep the maintenance burden low by building on a preexisting and mature release process, whilst permitting closer integration between arrow-rs and object storage going forward. However, should we decide on something different, both myself and Andrew are happy to continue to support the crates evolution regardless, and the hope is that through donating this project to arrow others may come to share this burden in time [1] https://github.com/apache/arrow-datafusion/pull/2677#issuecomment-1170271295 On 2022/06/28 22:52:50 Will Jones wrote: > Thanks for bringing up this discussion, Andrew! > > I started contributing to the crate in the hopes of using it in DataFusion > and delta-rs. I've thus far found it to be a high-quality codebase. I'd > like to see the crate adopted throughout the Rust Arrow ecosystem, and if > going under Apache governance helps that end, I support that. > > My one question is: Do we have maintainers (not sure if it has to be a > committer or a PMC member) in the Apache project who are ready to take on > release tasks for the crate? > > > > On Tue, Jun 28, 2022 at 11:44 AM Andrew Lamb <al...@influxdata.com> wrote: > > > Hello Rust / Arrow community, > > > > There is a proposal [1] in DataFusion to switch the abstraction used to > > read from remote object storage. Due to various reasons, this code is > > currently in its own crate/library [2] with unclear governance. We would > > like to contribute this code to the Apache Arrow project and I am hoping to > > gather community feedback on this idea. > > > > The full details can be found on [3]. Here is the rationale > > > > 1. A common, high quality object store abstraction for communicating with > > various remote object stores is useful for a range of projects and > > usecases. > > 2. Such a library is directly aligned with the Arrow mission of providing > > building blocks for modern high performance analytics systems > > 3. The clear governance of Apache Arrow offers the best chance to build a > > unified and strong community around this crate, hopefully both increasing > > its adoption and attracting community contributions for its long term > > evolution and maintenance > > > > Please let us know your thoughts, > > Andrew > > > > > > [1] https://github.com/apache/arrow-datafusion/issues/2489 > > [2] https://github.com/influxdata/object_store_rs/issues/41 > > [3] https://github.com/influxdata/object_store_rs/issues/41 > > >
