I have incorporated feedback left on github. The current draft is below. I plan to submit this on Wednesday so please add any suggestions via email / github or on the google doc
Andrew ## Description: The mission of Apache DataFusion is the creation and maintenance of software related to an extensible query engine ## Project Status: Current project status: New + Ongoing (high activity) Issues for the board: None ## Membership Data: Apache DataFusion was founded 2024-04-16 (10 months ago) There are currently 43 committers and 15 PMC members in this project. The Committer-to-PMC ratio is roughly 3:1. Community changes, past quarter: - Jonah Gao was added to the PMC on 2024-12-16 - Piotr Findeisen was added as committer on 2024-12-03 - Ruiqiu Cao was added as committer on 2024-12-10 - Yongting You was added as committer on 2025-01-18 Note that almost all communication for DataFusion and its subprojects happens on github and so our dev mailing list traffic is fairly light. ## Project Activity: ### Overall DataFusion is participating in Google Summer of Code with a number of ideas for projects with mentors already selected[1][2][3]. Additionally, some ideas on how to make DataFusion an ideal selection for university database projects such as the CMU database classes have been put forward. [1]: https://github.com/apache/datafusion/issues/14577 [2]: https://summerofcode.withgoogle.com/programs/2025/organizations/apache-datafusion [3]: https://datafusion.apache.org/contributor-guide/gsoc_application_guidelines.html ### DataFusion core https://github.com/apache/datafusion - 46.0.0 was released on 2025-03-07. - 45.0.0 was released on 2025-02-07. - 44.0.0 was released on 2024-12-31. Releases continue monthly and the project has been very active with many commits a day. It seems more new projects have been using DataFusion for query processing, which brings more contributors but also means we are spending more time fielding questions and figuring out how many more features to accept. Bruce Ritchie recently authored a [blog] about some of the features and the outlook for the next 6 months. A relevant quote: > In the core DataFusion repo alone we reviewed and accepted almost 1600 PRs from 206 different committers, created over 1100 issues and closed 751 of them 🚀. We have been focusing more recently on pre-release testing and making it easier for downstream consumers to use DataFusion, which is still a challenge given how fast the project is moving. [blog]: https://datafusion.apache.org/blog/2025/02/20/datafusion-45.0.0/ ### Sub project: DataFusion Python https://github.com/apache/datafusion-python - PYTHON-45.2.0 was released on 2025-02-23. - PYTHON-44.0.0 was released on 2025-02-07. - PYTHON-43.1.0 was released on 2024-12-12. We have been working on making it easier to interoperate with other systems, including support for FFI TableProvider ([#12920]) and new user documentation on FFI [#1031] [#12920]: https://github.com/apache/datafusion/pull/12920 [#1031]: https://github.com/apache/datafusion-python/pull/1031 ### Sub project: DataFusion Comet https://github.com/apache/datafusion-comet - COMET-0.6.0 was released on 2025-02-17. - COMET-0.5.0 was released on 2025-01-17. You can read about the recent happenings in Comet in the [0.6.0 blog] [0.6.0 blog]: https://datafusion.apache.org/blog/2025/02/17/datafusion-comet-0.6.0/ ### Sub project: DataFusion Ballista https://github.com/apache/datafusion-ballista - BALLISTA-44.0.0 was released on 2025-03-05. There has been some renewed interest in this project as the foundation for distributed query engines, and we made a new release recently. ### (New!) Sub project: DataFusion Ray https://github.com/apache/datafusion-ray This is a new project aims to make it easier to run DataFusion in a distributed environment using the https://www.ray.io/ compute engine Contributors are working hard at the moment to get DataFusionRay 0.1.0 out! Hopefully we can do that before the announcement and then there should be plenty to add. ### Sub project: sqlparser-rs https://github.com/apache/datafusion-sqlparser-rs We have made two releases since sqlparser became part of DataFusion. - SQLPARSER-0.55.0 was released on 2025-03-05. - SQLPARSER-0.54.0 was released on 2025-01-23. - SQLPARSER-0.53.0 was released on 2024-12-18. Ifeanyi Ubah (iffyio) is doing a great job reviewing PRs to keep the code consistent and flowing. ## Community Health: While we as always struggle with code review capacity, we have many active committers, and the community in general helps each other out with reviews. We continue to actively grow our committer and PMC ranks. We had several in person meetups in Chicago, Boston, and Amsterdam, and are working on organizing one in London in April 2025[1]. [1]: https://github.com/apache/datafusion/discussions/14647 On Sat, Mar 1, 2025 at 7:06 AM Andrew Lamb <al...@influxdata.com> wrote: > Hello Fearless DataFusion(iers)! > > We have an ASF board report due in 2 weeks and I have started a draft. > > Please feel free to post comments to the doc[1] or the ticket[2] or this > thread and I will incorporate them > > [1]: > https://docs.google.com/document/d/11b2GEmPh5gblWWegeZi3G38e97vRqHSRElkLTwZHrjY > [2]: https://github.com/apache/datafusion/issues/13713 > > Current draft is below. > > > ``` > ## Description: > The mission of Apache DataFusion is the creation and maintenance of > software > related to an extensible query engine > > ## Project Status: > Current project status: New + Ongoing (high activity) > Issues for the board: None > > ## Membership Data: > Apache DataFusion was founded 2024-04-16 (10 months ago) > There are currently 43 committers and 15 PMC members in this project. > The Committer-to-PMC ratio is roughly 3:1. > > Community changes, past quarter: > - Jonah Gao was added to the PMC on 2024-12-16 > - Piotr Findeisen was added as committer on 2024-12-03 > - Ruiqiu Cao was added as committer on 2024-12-10 > - Yongting You was added as committer on 2025-01-18 > > Note that almost all communication for DataFusion and its subprojects > happens on github and so our dev mailing list traffic is fairly light. > > ## Project Activity: > > ### Overall > > ### DataFusion core > > 45.0.0 was released on 2025-02-07. > 44.0.0 was released on 2024-12-31. > > https://github.com/apache/datafusion > > Releases continue monthly and the project has been very active with many > commits a day. It seems many new projects have been using DataFusion for > query processing which brings more contributors but also means we are > spending more time fielding questions and figuring out how many more > features to accept. > > Bruce Ritchie recently authored a [blog] about some of the features and > the outlook for the next 6 months. > > We have been focusing more recently on pre-release testing and making it > easier for downstream consumers to use DataFusion, which is still a > challenge given how fast the project is moving. > > [blog]: https://datafusion.apache.org/blog/2025/02/20/datafusion-45.0.0/ > > ### Sub project: DataFusion Python > > https://github.com/apache/datafusion-python > > PYTHON-45.2.0 was released on 2025-02-23. > PYTHON-44.0.0 was released on 2025-02-07. > PYTHON-43.1.0 was released on 2024-12-12. > > > ### Sub project: DataFusion Comet > > https://github.com/apache/datafusion-comet > > > COMET-0.6.0 was released on 2025-02-17. > COMET-0.5.0 was released on 2025-01-17. > > > ### Sub project: DataFusion Ballista > > https://github.com/apache/datafusion-ballista > > ### (New!) Sub project: DataFusion Ray > > https://github.com/apache/datafusion-ray > > This is a new project aims to make it easier to run DataFusion in a > distributed environment using the https://www.ray.io/ compute engine > > ### Sub project: Sqlparser > > We have made two releases since sqlparser became part of DataFusion. > > - SQLPARSER-0.54.0 was released on 2025-01-23. > - SQLPARSER-0.53.0 was released on 2024-12-18. > > Ifeanyi / iffyio is doing a great job reviewing PRs to keep the code > consistent and flowing. > > ## Community Health: > > While we as always struggle to get enough code review capacity, we have > many > active committers, and the community in general helps each other out with > reviews. We continue to actively grow our committer and PMC ranks. > > We had several in person meetups in Chicago, Boston, and Amsterdam, though > we don’t have any more > > ``` >
