If Apache Spark supports an LTS version while its sister projects, managed by the Spark PMC do not, it would result in a poor end user experience. The larger ecosystem will ofcourse have its own support matrix.
Regards, Mridul On Wed, Feb 25, 2026 at 3:48 PM Holden Karau <[email protected]> wrote: > I've updated the doc with "The scope of the LTS is limited to Spark itself > and does not apply to related/sister projects (like Spark on K8s > operator)." Just checking in to make sure conversation is settled and it's > an ok time to bring this forward to a vote (if folks have more thoughts I > want to give them time to bring them up). > > On Mon, Feb 16, 2026 at 9:39 AM Holden Karau <[email protected]> > wrote: > >> Good points, responses in-line :) >> >> Twitter: https://twitter.com/holdenkarau >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >> <https://www.fighthealthinsurance.com/?q=hk_email> >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> Pronouns: she/her >> >> >> On Sat, Feb 14, 2026 at 3:32 PM Dongjoon Hyun <[email protected]> >> wrote: >> >>> Hi Holden, >>> >>> Thank you for your passion and for keeping this thread up to date. I >>> truly appreciate it. >>> >>> This situation is a bit tricky, as it exponentially increases the >>> maintenance burden for the following reasons: >>> >>> (1) Legacy Tech Stack Support: This would require us to support Java 8, >>> Scala 2.12, Python 3.8, and SparkR. For instance, I don’t believe we can >>> maintain PySpark 3.5 on Python 3.8 at our previously promised service >>> level. We would definitely need to downgrade the support level, and this >>> must be made crystal clear to the community. >>> >>> (2) Impact on Subprojects: It adds a significant burden to all Apache >>> Spark subprojects, such as the Apache K8s Operator and Spark Connect >>> clients (Swift/Go/Rust). For example, while Spark 3.5 must support K8s >>> 1.24+, Spark 4.0 targets K8s 1.30+. The extended period effectively locks >>> us into supporting older K8s versions that are no longer available from >>> major public cloud vendors. >>> >> This is an interesting consideration. I think we can scope this down to >> Spark itself, and Spark subprojects can make their own decision as they >> have their own release cadence broadley speaking -- but we should make that >> clear in the documentation. >> >>> >>> (3) Ecosystem-wide Pressure: This also places constraints on >>> Spark-related ASF sister projects like Iceberg, Parquet, ORC, Celeborn, >>> Gluten, and Comet. While the final decision rests with them, I believe they >>> need to be informed of our strategic direction. >>> >> This is a good point. I think we can reach out to their dev lists once >> we've reached an in-project consensus. >> >>> >>> To provide extended community support responsibly, we need at least >>> three clearly defined components: >>> >>> (a) A predefined extension period (Confirmed) >>> (b) A predefined pool of Release Managers (Holden has volunteered) >>> (c) A predefined release cadence (Currently unknown as discussed so far) >>> >>> Regarding (c), I believe we can keep it simple, similar to the Apache >>> Spark 4.x preview releases. Currently, Hyukjin has successfully managed all >>> previews by leveraging automation, with the only exception being >>> 4.1.0-preview4, which I assisted with. For 3.5.x, we should maintain the >>> same cadence as before. Perhaps Holden could manage a release every four >>> months if there are new patches on branch-3.5. Of course, other community >>> members can assist as well. >>> >> I think 4 months is a reasonable cadence, but we'd only bother with a >> release, of course, if there is a new security issue. >> >>> >>> Sincerely, >>> Dongjoon Hyun >>> >>> On 2026/02/13 19:27:48 Holden Karau wrote: >>> > I wasn’t super sure about that, I know the initial proposal was a >>> little >>> > sparse for some so I figured I’d make it a bit more formal and follow >>> the >>> > SPIP process like we did with the overall release process change. >>> > >>> > Twitter: https://twitter.com/holdenkarau >>> > Fight Health Insurance: https://www.fighthealthinsurance.com/ >>> > <https://www.fighthealthinsurance.com/?q=hk_email> >>> > Books (Learning Spark, High Performance Spark, etc.): >>> > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> > Pronouns: she/her >>> > >>> > >>> > On Thu, Feb 12, 2026 at 11:58 PM Wenchen Fan <[email protected]> >>> wrote: >>> > >>> > > This is a procedure change, not a code change, do we need SPIP? >>> Anyway, +1 >>> > > to the proposal! >>> > > >>> > > On Thu, Feb 12, 2026 at 6:28 AM Holden Karau <[email protected] >>> > >>> > > wrote: >>> > > >>> > >> This is a more formal (but still a discussion level thread) >>> proposing an >>> > >> extended 3.5 LTS window to allow for people to complete their >>> migrations >>> > >> given the less than one year between 4.0 release and 3.5 EOL. >>> > >> >>> > >> The draft SPIP is >>> > >> >>> https://docs.google.com/document/d/15fb-7DNztzpQMF1MQovK75UmpC6um7KgO8cg_X5oa1I/edit?usp=sharing >>> > >> and the JIRA SPARK-55489 >>> > >> <https://issues.apache.org/jira/browse/SPARK-55489> >>> > >> >>> > >> Previous discussion was: >>> > >> https://lists.apache.org/thread/4r6q8187b30p0ppclw0pyfjp8h8xs3rq >>> > >> >>> > >> Cheers, >>> > >> >>> > >> Holden :) >>> > >> >>> > >> -- >>> > >> Twitter: https://twitter.com/holdenkarau >>> > >> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>> > >> <https://www.fighthealthinsurance.com/?q=hk_email> >>> > >> Books (Learning Spark, High Performance Spark, etc.): >>> > >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> > >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> > >> Pronouns: she/her >>> > >> >>> > > >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: [email protected] >>> >>> > > -- > Twitter: https://twitter.com/holdenkarau > Fight Health Insurance: https://www.fighthealthinsurance.com/ > <https://www.fighthealthinsurance.com/?q=hk_email> > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > Pronouns: she/her >
