Hi, All.
Apache Spark 3.4.1 tag was created on Jun 19th and `branch-3.4` has 103
commits including important security and correctness patches like
SPARK-44251, SPARK-44805, and SPARK-44940.
https://github.com/apache/spark/releases/tag/v3.4.1
$ git log --oneline v3.4.1..HEAD | wc -l
103
SPARK-44251 Potential for incorrect results or NPE when full outer
USING join has null key value
SPARK-44805 Data lost after union using
spark.sql.parquet.enableNestedColumnVectorizedReader=true
SPARK-44940 Improve performance of JSON parsing when
"spark.sql.json.enablePartialResults" is enabled
Currently, I'm checking the following open correctness issues. I'd like to
propose to release Apache Spark 3.4.2 after resolving them and volunteer as
the release manager for Apache Spark 3.4.2. If there are no additional
blockers, the first tentative RC1 vote date is November 13rd (Monday). If
it takes some time to resolve the open correctness issues, we can start the
vote after Thanksgiving holiday.
SPARK-44512 dataset.sort.select.write.partitionBy sorts wrong column
SPARK-45282 Join loses records for cached datasets
WDTY?
Dongjoon.
