Hi, All. Apache Spark 3.5.0 is scheduled for August (1st Release Candidate) and currently a few notable things are under discussions in the mailing list.
I believe it's a good time to share a short summary list (containing both completed and in-progress items) to give a highlight in advance and to collect your targets too. Please share your expectations or working items if you want to prioritize them more in the community in Apache Spark 3.5.0 timeframe. (Sorted by ID) SPARK-40497 Upgrade Scala 2.13.11 SPARK-42452 Remove hadoop-2 profile from Apache Spark 3.5.0 SPARK-42913 Upgrade to Hadoop 3.3.5 (aws-java-sdk-bundle: 1.12.262 -> 1.12.316) SPARK-43024 Upgrade Pandas to 2.0.0 SPARK-43200 Remove Hadoop 2 reference in docs SPARK-43347 Remove Python 3.7 Support SPARK-43348 Support Python 3.8 in PyPy3 SPARK-43351 Add Spark Connect Go prototype code and example SPARK-43379 Deprecate old Java 8 versions prior to 8u371 SPARK-43394 Upgrade to Maven 3.8.8 SPARK-43436 Upgrade to RocksDbjni 8.1.1.1 SPARK-43446 Upgrade to Apache Arrow 12.0.0 SPARK-43447 Support R 4.3.0 SPARK-43489 Remove protobuf 2.5.0 SPARK-43519 Bump Parquet to 1.13.1 SPARK-43581 Upgrade kubernetes-client to 6.6.2 SPARK-43588 Upgrade to ASM 9.5 SPARK-43600 Update K8s doc to recommend K8s 1.24+ SPARK-43738 Upgrade to DropWizard Metrics 4.2.18 SPARK-43831 Build and Run Spark on Java 21 SPARK-43832 Upgrade to Scala 2.12.18 SPARK-43836 Make Scala 2.13 as default in Spark 3.5 SPARK-43842 Upgrade gcs-connector to 2.2.14 SPARK-43844 Update to ORC 1.9.0 UMBRELLA: Add SQL functions into Scala, Python and R API Thanks, Dongjoon. PS. The above is not a list of release blockers. Instead, it could be a nice-to-have from someone's perspective.
