Hi all, The WKB fix has been merged and backported to branch-4.2, so I'll cut RC5 soon.
As discussed, the log4j version upgrade is not a blocker and won't hold RC5. The DriverLogger workaround (SPARK-57774 https://github.com/apache/spark/pull/56890) is a small Spark-side fix, so I'm OK to include it in RC5 unless there are any objections. Thanks, Huaxin On Mon, Jun 29, 2026 at 11:14 PM Cheng Pan <[email protected]> wrote: > After taking a closer look, I think at least the Spark built-in > DriverLogger are affected - it sets ignoreExceptions=false, which means > that log4j internal exceptions will propagate to the caller and crash the > call site, that's why we have observed UT failed in GHA occasionally. For > other logging places, it depends on whether user to set > ignoreExceptions=false (default is true, safe, no exception propagation). > > I opened SPARK-57774 https://github.com/apache/spark/pull/56890 to fix > the DriverLogger ignoreExceptions issue. > > Thanks, > Cheng Pan > > > > On Jun 30, 2026, at 12:59, Yang Jie <[email protected]> wrote: > > Will this NPE make Spark 4.2 completely unusable? If not, dependency > upgrades theoretically should not be backported to the 4.2.x branch, as > third-party dependencies for the entire 4.x line have been frozen, correct? > > Jie Yang > > On 2026/06/30 02:35:43 huaxin gao wrote: > > Thanks Cheng for flagging this! > > I prefer not to delay RC5 (which is for the WKB security fix) just to wait > for an outside dependency. Also, 2.26.1 just started voting, so we can't > use it until it is on Maven Central (about 4-5 days). So I will cut RC5 > with the current log4j. If RC5 fails, we can include 2.26.1, otherwise > let's include it in 4.2.1. > > Thanks, > > Huaxin > > On Mon, Jun 29, 2026 at 6:57 PM Cheng Pan <[email protected]> wrote: > > Can we include the log4j 2.26.1 upgrading in next RC? > > TL:DR, the current log4j2 2.25.x used by branch-4.2 may throw NPE on > logging, downgrading to 2.24.x can fix the issue but bring some CVEs back, > log4j2 2.26.1 is in the voting phase and likely available soon. See full > context at [1] > > [1] https://github.com/apache/spark/pull/51719#issuecomment-3341344974 > > Thanks, > Cheng Pan > > > > On Jun 30, 2026, at 08:52, huaxin gao <[email protected]> wrote: > > Thanks Szehon for reporting the problem. I'm failing RC4, and will roll > RC5 once the fix <https://github.com/apache/spark/pull/56875> is merged > and backported to branch-4.2. > > Thanks to everyone who tested RC4! > > Huaxin > > On Mon, Jun 29, 2026 at 5:14 PM Szehon Ho <[email protected]> wrote: > > I made a fix for this, it would be great to get it into the release. > > https://github.com/apache/spark/pull/56875 > > Thanks Huaxin! > Szehon > > > On Mon, Jun 29, 2026 at 4:54 PM Szehon Ho <[email protected]> > wrote: > > Sorry I was informed of a potential vulnerability/problem in new fromWKB > in Spark 4.2, looking now at it. > > Thanks > Szehon > > On Mon, Jun 29, 2026 at 1:05 PM Allison Wang <[email protected]> > wrote: > > +1 > > On Mon, Jun 29, 2026 at 10:34 AM Max Gekk <[email protected]> wrote: > > +1 > > On Mon, Jun 29, 2026 at 6:32 PM Xiao Li <[email protected]> wrote: > > > +1 (binding) > > Verified on Linux (Ubuntu, JDK 17, Python 3.11): > > - Signatures: every artifact (source, the three binary > > distributions, the pyspark/pyspark_client/pyspark_connect tarballs, and > SparkR) has a good GPG signature from the RM key > 709226B910E0F10917123B6259B586ADA5A538D1, which is in KEYS. > > - Checksums: SHA512 matches for all artifacts.- Tag/commit: > > v4.2.0-rc4 resolves to f92a807c06b, and the RELEASE metadata embedded in > the binary distributions records the same git revision. > > - Source build: compiled and packaged spark-core and its module > > dependencies from the source tarball with ./build/mvn on JDK 17 (BUILD > SUCCESS). > > - Binary distribution: ran SparkPi and a spark-shell Scala job > > (range aggregation + Spark SQL) — results correct. > > - PySpark: pip-installed pyspark-4.2.0.tar.gz and ran DataFrame, > > Spark SQL, and a Python UDF in local mode — results correct. > > > Thanks Huaxin for driving the release! > > Xiao > > Uroš Bojanić <[email protected]> 于2026年6月29日周一 09:12写道： > > > +1 (non-binding) > > verified RC4 (tag v4.2.0-rc4, commit f92a807c06b) on macOS/arm64. > > - Good GPG signatures and SHA512 sums on all artifacts (source, > > three binaries, the PySpark/SparkR tarballs) against KEYS. > > - Built the full source tree from the tag with -Phive > > -Phive-thriftserver, got a clean BUILD SUCCESS across all 39 modules. > > - Ran quick smoke tests across every API surface: Java, Scala, SQL, > > and PySpark job from the bundled distro; all look good. > > - Sanity-checked the binary dist (RELEASE, LICENSE/NOTICE/licenses > > all present) and ran dev/check-license; RAT passes. > > - Diffed the docs against 4.1.0 and analyzed the changes (new > > pages, migration guides and version refs); all look good. > > > Thank you Huaxin Gao! > > On 2026/06/27 00:21:32 [email protected] wrote: > > Please vote on releasing the following candidate as Apache Spark > > version 4.2.0. > > > The vote is open until Mon, 29 Jun 2026 18:21:31 PDT and passes > > if a majority +1 PMC votes are cast, with > > a minimum of 3 +1 votes. > > [ ] +1 Release this package as Apache Spark 4.2.0 > [ ] -1 Do not release this package because ... > > To learn more about Apache Spark, please see > > https://spark.apache.org/ > > > The tag to be voted on is v4.2.0-rc4 (commit f92a807c06b): > https://github.com/apache/spark/tree/v4.2.0-rc4 > > The release files, including signatures, digests, etc. can be > > found at: > > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-bin/ > > Signatures used for Spark RCs can be found in this file: > https://downloads.apache.org/spark/KEYS > > The staging repository for this release can be found at: > > https://repository.apache.org/content/repositories/orgapachespark-1524/ > > > The documentation corresponding to this release can be found at: > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-docs/ > > The list of bug fixes going into 4.2.0 can be found at the > > following URL: > > https://issues.apache.org/jira/projects/SPARK/versions/12356380 > > FAQ > > ========================= > How can I help test this release? > ========================= > > If you are a Spark user, you can help us test this release by > > taking > > an existing Spark workload and running on this release candidate, > > then > > reporting any regressions. > > If you're working in PySpark you can set up a virtual env and > > install > > the current RC via "pip install > > > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-bin/pyspark-4.2.0.tar.gz > " > > and see if anything important breaks. > In the Java/Scala, you can add the staging repository to your > > project's resolvers and test > > with the RC (make sure to clean up the artifact cache > > before/after so > > you don't end up building with an out of date RC going forward). > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > > >
