Officially failing the 4.2.0 RC1. Also updating the list, https://github.com/apache/spark/pull/58929 is merged (the last relevant 4.3.0 blocker that I know) , so will cut 4.2.1 RC2 shortly, please take a look when it's out.
Thanks everyone for voting and commenting. Szehon On Thu, Oct 1, 2026 at 6:08 AM Jungtaek Lim <[email protected]> wrote: > https://github.com/apache/spark/pull/58929 > If I understand correctly about the issue, this is a blocker for 4.2 as it > can break checkpoint in stateful workloads (despite that v2 is not the > default format). The issue isn't framed well enough to emphasize the > severity; once we have a new JIRA ticket (or rewritten JIRA ticket), I'll > bump the severity to blocker. > > On Thu, Oct 1, 2026 at 12:29 PM Szehon Ho <[email protected]> wrote: > >> I took a look at the recent 4.3 fixes/blockers. We fixed two for 4.2, >> but we decided not to do the other two for 4.2, as listed below. >> >> - SPARK-59804 <https://issues.apache.org/jira/browse/SPARK-59804> — PR >> #59074 <https://github.com/apache/spark/pull/59074> — *merged into >> 4.2* >> - SPARK-51830 <https://issues.apache.org/jira/browse/SPARK-51830> — PR >> #57994 <https://github.com/apache/spark/pull/57994>; *merged into >> 4.2* >> - SPARK-44462 <https://issues.apache.org/jira/browse/SPARK-44462> — PR >> #55410 <https://github.com/apache/spark/pull/55410> — took a >> look. it is a complex fix with some risk, and there's some behavior >> change. We are leaning towards omitting it from 4.2.1. >> - SPARK-59122 <https://issues.apache.org/jira/browse/SPARK-59122> — PR >> #58419 <https://github.com/apache/spark/pull/58419> — took a look. >> It is also not so simple and also UNION codegen is disabled on 4.2, so >> also >> leaning towards omitting it from 4.2.1. >> >> I am not sure we need to wait until 4.3 RC passes, as the scope of 4.2.1 >> maintenance release should be a more focused fix for 4.2 users hitting >> obvious issues (example, on Iceberg side). As we see from some examples, >> some 4.3 issues may not be appropriate for 4.2. I would prefer to restart >> the process and halt only if there is an unresolved 4.2 blocker, unless >> someone feels strongly otherwise. >> >> Thanks, >> Szehon >> >> On Mon, Sep 28, 2026 at 7:01 PM Holden Karau <[email protected]> >> wrote: >> >>> I think it might be good to hold the point releases until the 4.3 RC >>> passes just incase we’ve got any blockers we discover and want to backport? >>> I don’t think it’s a “must” hold just it might make life easier. >>> >>> >>> Twitter: https://twitter.com/holdenkarau >>> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>> <https://www.fighthealthinsurance.com/?q=hk_email> >>> Books (Learning Spark, High Performance Spark, etc.): >>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> Pronouns: she/her >>> >>> On Mon, Sep 28, 2026 at 6:20 PM Uroš Bojanić <[email protected]> wrote: >>> >>>> Hi folks, thank you for working on this! Please also let me know if any >>>> of the issues above are blockers for other releases (e.g. 4.1.4, 4.0.5) or >>>> not; there are separate threads for those releases as well. >>>> >>>> On 2026/09/28 09:11:21 [email protected] wrote: >>>> > Please vote on releasing the following candidate as Apache Spark >>>> version 4.2.1. >>>> > >>>> > The vote is open until Thu, 01 Oct 2026 03:11:20 PDT and passes if a >>>> majority +1 PMC votes are cast, with >>>> > a minimum of 3 +1 votes. >>>> > >>>> > [ ] +1 Release this package as Apache Spark 4.2.1 >>>> > [ ] -1 Do not release this package because ... >>>> > >>>> > To learn more about Apache Spark, please see >>>> https://spark.apache.org/ >>>> > >>>> > The tag to be voted on is v4.2.1-rc1 (commit 72c18760854): >>>> > https://github.com/apache/spark/tree/v4.2.1-rc1 >>>> > >>>> > The release files, including signatures, digests, etc. can be found >>>> at: >>>> > https://dist.apache.org/repos/dist/dev/spark/v4.2.1-rc1-bin/ >>>> > >>>> > Signatures used for Spark RCs can be found in this file: >>>> > https://downloads.apache.org/spark/KEYS >>>> > >>>> > The staging repository for this release can be found at: >>>> > >>>> https://repository.apache.org/content/repositories/orgapachespark-1531/ >>>> > >>>> > The documentation corresponding to this release can be found at: >>>> > https://dist.apache.org/repos/dist/dev/spark/v4.2.1-rc1-docs/ >>>> > >>>> > The list of bug fixes going into 4.2.1 can be found at the following >>>> URL: >>>> > https://issues.apache.org/jira/projects/SPARK/versions/12357191 >>>> > >>>> > FAQ >>>> > >>>> > ========================= >>>> > How can I help test this release? >>>> > ========================= >>>> > >>>> > If you are a Spark user, you can help us test this release by taking >>>> > an existing Spark workload and running on this release candidate, then >>>> > reporting any regressions. >>>> > >>>> > If you're working in PySpark you can set up a virtual env and install >>>> > the current RC via "pip install >>>> https://dist.apache.org/repos/dist/dev/spark/v4.2.1-rc1-bin/pyspark-4.2.1.tar.gz >>>> " >>>> > and see if anything important breaks. >>>> > In the Java/Scala, you can add the staging repository to your >>>> project's resolvers and test >>>> > with the RC (make sure to clean up the artifact cache before/after so >>>> > you don't end up building with an out of date RC going forward). >>>> > >>>> > --------------------------------------------------------------------- >>>> > To unsubscribe e-mail: [email protected] >>>> > >>>> > >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: [email protected] >>>> >>>>
