I have updated the proposal, especially around how we use CI to avoid breaking compatibility between DataFusion and Ballista, based on the earlier discussions in the document.
I think that it would be ideal to put this plan into action as soon as possible after the DataFusion 8.0.0 release so would like to see if there is support for the latest proposal. Thanks, Andy. On Wed, May 11, 2022 at 8:17 AM Andy Grove <andygrov...@gmail.com> wrote: > I would like to propose that we move the Ballista project to a new > top-level *arrow-ballista* repository. > > The rationale for this (copied from the GitHub issue [1]) is: > > - Decouple release process for DataFusion and Ballista > - Allow each project to have top-level documentation and user guides > that are targeting the appropriate audience > - Reduce issue tracking and PR review burden for DataFusion > maintainers who are not as interested in Ballista > - Help avoid accidental circular dependencies being introduced between > the projects (such as > - datafusion-cli crate has circular dependency #2433 > <https://github.com/apache/arrow-datafusion/issues/2433>) > - Helps formalize the public API for DataFusion that other query > engines should be using > > There is also a design document [2] where we will be discussing the finer > details of this and coordinating on the plan to implement. > > I do not recall if a change of this nature requires a formal vote or not > but I will plan on holding one before we create the new repo unless anyone > tells me this is not required. > > Thanks, > > Andy. > > [1] https://github.com/apache/arrow-datafusion/issues/2502 > [2] > https://docs.google.com/document/d/1jNRbadyStSrV5kifwn0khufAwq6OnzGczG4z8oTQJP4/edit?usp=sharing >
