A follow-up on my earlier note: the ~5x I quoted is for line splitting alone. I measured an end-to-end replay of a 226 MB (uncompressed) event log into the app status store on branch-4.0: about 1.51 s with RC2 vs 0.98 s with SPARK-60158 (the same as 4.0.4), so roughly 1.5x, since JSON parsing dominates. With a disk-backed store and less compressible logs the relative impact should be smaller. Given the modest impact, I don’t think it needs to block the release; SPARK-60158 is already on branch-4.0 if there is another RC.
On Mon, Oct 12, 2026 at 10:35 PM L. C. Hsieh <[email protected]> wrote: > FYI, this RC includes SPARK-59407/SPARK-59804, which made line splitting > in the event log replay reader about 5x slower than in 4.0.4 (History > Server UI loading and event log compaction). SPARK-60158 fixes it and is > already on branch-4.0, but it landed after RC2 was cut. It is a performance > regression, not a correctness issue, so I don’t consider it a blocker, but > it would be good to pick it up if there is another RC. > > On Mon, Oct 12, 2026 at 11:56 AM Uroš Bojanić <[email protected]> wrote: > >> Please vote on releasing the following candidate as Apache Spark version >> 4.0.5. >> >> The vote is open until Thu, 15 Oct 2026 12:38:20 PDT and passes if a >> majority +1 PMC votes are cast, with >> a minimum of 3 +1 votes. >> >> [ ] +1 Release this package as Apache Spark 4.0.5 >> [ ] -1 Do not release this package because ... >> >> To learn more about Apache Spark, please see https://spark.apache.org/ >> >> The tag to be voted on is v4.0.5-rc2 (commit 51d701c09de): >> https://github.com/apache/spark/tree/v4.0.5-rc2 >> >> The release files, including signatures, digests, etc. can be found at: >> https://dist.apache.org/repos/dist/dev/spark/v4.0.5-rc2-bin/ >> >> Signatures used for Spark RCs can be found in this file: >> https://downloads.apache.org/spark/KEYS >> >> The staging repository for this release can be found at: >> https://repository.apache.org/content/repositories/orgapachespark-1535/ >> >> The documentation corresponding to this release can be found at: >> https://dist.apache.org/repos/dist/dev/spark/v4.0.5-rc2-docs/ >> >> The list of bug fixes going into 4.0.5 can be found at the following URL: >> https://issues.apache.org/jira/projects/SPARK/versions/12357251 >> >> FAQ >> >> ========================= >> How can I help test this release? >> ========================= >> >> If you are a Spark user, you can help us test this release by taking >> an existing Spark workload and running on this release candidate, then >> reporting any regressions. >> >> If you're working in PySpark you can set up a virtual env and install >> the current RC via "pip install >> https://dist.apache.org/repos/dist/dev/spark/v4.0.5-rc2-bin/pyspark-4.0.5.tar.gz >> " >> and see if anything important breaks. >> In the Java/Scala, you can add the staging repository to your project's >> resolvers and test >> with the RC (make sure to clean up the artifact cache before/after so >> you don't end up building with an out of date RC going forward). >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: [email protected] >> >>
