Hi Larry, Thank you so much, My initial thought was that we could continue supporting Spark 3 + Scala 2.12 and introducing Scala 2.13 as an additional module. Over time, we could fully drop the older combinations once the community feels comfortable. I’ve put together the table below summarizing the current state and the proposed effort required for modernization, happy to convert to a doc if needed.
*Requirement* Required Version Current Status Proposed Effort needed Scala Version Scala 2.13 Scala 2.12 Scala 2.13 upgrade needed Python Version Python 3.9 Python 3.9 No effort needed Java Version JDK 17 JDK 17 No effort needed Spark Version Spark 4 Spark 3 Major Spark 4 support needed Release Vehicle Post 1.0.0 or 0.10.0 0.9.0 Tentatively 1.0.0+ Would love to get suggestions or changes from others on whether the proposed effort and release vehicle make sense. Warm Regards, Arnav On Mon, Dec 8, 2025 at 10:40 AM larry mccay <[email protected]> wrote: > Thanks for bringing this up, Arnav! > This sounds like a good topic for 0.10.0 or maybe even a 1.0.0 post > graduation. > > Do we need to deprecate previous versions or drop them completely for this > change? > > In general I am a +1 on this but we need to tease out the details and > appropriate release vehicle. > > > On Sun, Dec 7, 2025 at 11:29 PM Arnav Balyan <[email protected]> > wrote: > > > Looks like I used the wrong dev@ address. cc > [email protected] > > > > On Mon, Dec 8, 2025 at 9:44 AM Arnav Balyan <[email protected]> > > wrote: > > > > > Hi team, > > > > > > Just starting a thread to discuss support for Scala 2.13 and > > > eventually Spark 4 (which requires scala 2.13) for Livy. With the > Spark 4 > > > release, it may be a good time to start evaluating compatibility on the > > > Livy side as well. > > > > > > Since this could introduce breaking changes, it may be targeted for a > > > future release. The goal is to gauge interest and determine whether > Scala > > > 2.13/Spark 4 support could be a post 0.9 release modernization effort. > > > > > > Would love to hear everyone’s thoughts, concerns and suggestions. > > > > > > Thanks, > > > Arnav > > > > > >
