To: [email protected] Subject: [DISCUSS] Proposal to establish Apache Comet as a top-level project
Hi all, Following the discussion in [1], I would like to propose promoting the DataFusion Comet sub-project to a new top-level Apache project, Apache Comet. The draft proposal and board resolution are in a pull request [2] so that anyone can comment on specific lines. You can read the rendered proposal at [3], and the resolution is also included at the end of this email. Why Comet is different from the other DataFusion sub-projects, which extend DataFusion. Comet is an accelerator for Apache Spark that uses DataFusion as its execution engine, and the DataFusion and Comet communities are now largely distinct, though there is overlap. The Comet community is large and active enough to stand on its own. We have made 24 releases since July 2024, including 1.0.0 in August and 1.1.0 on October 6, and 146 people have authored merged pull requests, 81 of them in the last 12 months. Focused governance will make it easier to grow the community and to recognize contributors whose work is mainly on Comet. Key points - Name: "Apache Comet" was approved in PODLINGNAMESEARCH-255 [4]. - Initial PMC: existing DataFusion PMC members. Nobody is promoted as part of this process. The proposed PMC is Andy Grove, Andrew Lamb, Liang-Chi Hsieh, Matt Butrovich, and Oleks V. - Chair: I would be happy to serve as the initial chair. - Initial committers: the existing DataFusion committers who have contributed to Comet. They are Bhargava Vadlamani, Chao Sun, Dmitrii Blaginin, Huaxin Gao, Kazuyuki Tanimura, Manu Zhang, Martin Grigorov, Parth Chandra, Raz Luvaton, and Zhen Wang. - Once Apache Comet is established, its PMC will vote on new committers and PMC members from the Comet community. - Joining Apache Comet does not change anyone's role in DataFusion. - The repository would be renamed from apache/datafusion-comet to apache/comet, and the documentation would move to comet.apache.org. - The scope statement in the resolution is still open for discussion [5]. Feedback Please reply here or comment on the pull request. In particular: - If you are listed above, please reply to confirm that you would like to join, or let me know privately if you would rather not. - If you are an existing DataFusion PMC member or committer who contributes to Comet and I have missed you, please let me know. Next steps I plan to leave this open for about a week and then start a formal [VOTE] on this list, where DataFusion PMC votes are binding. If the vote passes, the resolution would go to the ASF board, ideally at the November 18 board meeting. The tracking issue for the whole process is [6]. Thanks, Andy [1] https://github.com/apache/datafusion-comet/issues/5184 [2] https://github.com/apache/datafusion-comet/pull/6817 [3] https://github.com/andygrove/datafusion-comet/blob/andygrove/comet-promotion-edits/docs/source/about/top_level_project_proposal.md [4] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-255 [5] https://github.com/apache/datafusion-comet/issues/6789 [6] https://github.com/apache/datafusion-comet/issues/6793 ---------- Proposed Resolution ---------- X. Establish the Apache Comet Project WHEREAS, the Board of Directors deems it to be in the best interests of the Foundation and consistent with the Foundation's purpose to establish a Project Management Committee charged with the creation and maintenance of open-source software related to accelerating Apache Spark using native, columnar execution, for distribution at no charge to the public. NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee (PMC), to be known as the "Apache Comet Project", be and hereby is established pursuant to Bylaws of the Foundation; and be it further RESOLVED, that the Apache Comet Project be and hereby is responsible for the creation and maintenance of software related to accelerating Apache Spark using native, columnar execution; and be it further RESOLVED, that the office of "Vice President, Apache Comet" be and hereby is created, the person holding such office to serve at the direction of the Board of Directors as the chair of the Apache Comet Project, and to have primary responsibility for management of the projects within the scope of responsibility of the Apache Comet Project; and be it further RESOLVED, that the persons listed immediately below be and hereby are appointed to serve as the initial members of the Apache Comet Project: * Andy Grove <[email protected]> * Andrew Lamb <[email protected]> * Liang-Chi Hsieh <[email protected]> * Matt Butrovich <[email protected]> * Oleks V. <[email protected]> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Andy Grove be appointed to the office of Vice President, Apache Comet, to serve in accordance with and subject to the direction of the Board of Directors and the Bylaws of the Foundation until death, resignation, retirement, removal or disqualification, or until a successor is appointed; and be it further RESOLVED, that the Apache Comet Project be and hereby is tasked with the migration and rationalization of the Apache DataFusion Comet sub-project; and be it further RESOLVED, that all responsibilities pertaining to the Apache DataFusion Comet sub-project encumbered upon the Apache DataFusion Project are hereafter discharged. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
