Thanks for getting this process going, Andy. I can’t wait to see where Comet could go as a TLP, and would love to be a part of it.
-Matt On Fri, Oct 9, 2026 at 7:44 AM Andy Grove <[email protected]> wrote: > To: [email protected] > Subject: [DISCUSS] Proposal to establish Apache Comet as a top-level > project > > Hi all, > > Following the discussion in [1], I would like to propose promoting the > DataFusion Comet sub-project to a new top-level Apache project, Apache > Comet. > > The draft proposal and board resolution are in a pull request [2] so > that anyone can comment on specific lines. You can read the rendered > proposal at [3], and the resolution is also included at the end of > this email. > > Why > > Comet is different from the other DataFusion sub-projects, which > extend DataFusion. Comet is an accelerator for Apache Spark that uses > DataFusion as its execution engine, and the DataFusion and Comet > communities are now largely distinct, though there is overlap. The > Comet community is large and active enough to stand on its own. We > have made 24 releases since July 2024, including 1.0.0 in August and > 1.1.0 on October 6, and 146 people have authored merged pull requests, > 81 of them in the last 12 months. Focused governance will make it > easier to grow the community and to recognize contributors whose work > is mainly on Comet. > > Key points > > - Name: "Apache Comet" was approved in PODLINGNAMESEARCH-255 [4]. > - Initial PMC: existing DataFusion PMC members. Nobody is promoted as > part of this process. The proposed PMC is Andy Grove, Andrew Lamb, > Liang-Chi Hsieh, Matt Butrovich, and Oleks V. > - Chair: I would be happy to serve as the initial chair. > - Initial committers: the existing DataFusion committers who have > contributed to Comet. They are Bhargava Vadlamani, Chao Sun, Dmitrii > Blaginin, Huaxin Gao, Kazuyuki Tanimura, Manu Zhang, Martin Grigorov, > Parth Chandra, Raz Luvaton, and Zhen Wang. > - Once Apache Comet is established, its PMC will vote on new > committers and PMC members from the Comet community. > - Joining Apache Comet does not change anyone's role in DataFusion. > - The repository would be renamed from apache/datafusion-comet to > apache/comet, and the documentation would move to comet.apache.org. > - The scope statement in the resolution is still open for discussion [5]. > > Feedback > > Please reply here or comment on the pull request. In particular: > > - If you are listed above, please reply to confirm that you would like > to join, or let me know privately if you would rather not. > - If you are an existing DataFusion PMC member or committer who > contributes to Comet and I have missed you, please let me know. > > Next steps > > I plan to leave this open for about a week and then start a formal > [VOTE] on this list, where DataFusion PMC votes are binding. If the > vote passes, the resolution would go to the ASF board, ideally at the > November 18 board meeting. The tracking issue for the whole process is > [6]. > > Thanks, > > Andy > > [1] https://github.com/apache/datafusion-comet/issues/5184 > [2] https://github.com/apache/datafusion-comet/pull/6817 > [3] > https://github.com/andygrove/datafusion-comet/blob/andygrove/comet-promotion-edits/docs/source/about/top_level_project_proposal.md > [4] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-255 > [5] https://github.com/apache/datafusion-comet/issues/6789 > [6] https://github.com/apache/datafusion-comet/issues/6793 > > ---------- Proposed Resolution ---------- > > X. Establish the Apache Comet Project > > WHEREAS, the Board of Directors deems it to be in the best interests of the > Foundation and consistent with the Foundation's purpose to establish a > Project > Management Committee charged with the creation and maintenance of > open-source > software related to accelerating Apache Spark using native, columnar > execution, for distribution at no charge to the public. > > NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), > to > be known as the "Apache Comet Project", be and hereby is established > pursuant > to Bylaws of the Foundation; and be it further > > RESOLVED, that the Apache Comet Project be and hereby is responsible for > the > creation and maintenance of software related to accelerating Apache Spark > using native, columnar execution; and be it further > > RESOLVED, that the office of "Vice President, Apache Comet" be and hereby > is > created, the person holding such office to serve at the direction of the > Board > of Directors as the chair of the Apache Comet Project, and to have primary > responsibility for management of the projects within the scope of > responsibility of the Apache Comet Project; and be it further > > RESOLVED, that the persons listed immediately below be and hereby are > appointed to serve as the initial members of the Apache Comet Project: > > * Andy Grove <[email protected]> > * Andrew Lamb <[email protected]> > * Liang-Chi Hsieh <[email protected]> > * Matt Butrovich <[email protected]> > * Oleks V. <[email protected]> > > NOW, THEREFORE, BE IT FURTHER RESOLVED, that Andy Grove be appointed to the > office of Vice President, Apache Comet, to serve in accordance with and > subject to the direction of the Board of Directors and the Bylaws of the > Foundation until death, resignation, retirement, removal or > disqualification, > or until a successor is appointed; and be it further > > RESOLVED, that the Apache Comet Project be and hereby is tasked with the > migration and rationalization of the Apache DataFusion Comet sub-project; > and > be it further > > RESOLVED, that all responsibilities pertaining to the Apache DataFusion > Comet > sub-project encumbered upon the Apache DataFusion Project are hereafter > discharged. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
