- 1 (non-binding) A ran into number of installation and launching problems. May be it’s my enviornment, even though I removed any old binaries and packages.
1. Pip installing pyspark4.0.0 and pyspark-connect-4.0 from .tz file workedl, launching pyspark results into 25/03/04 14:00:26 ERROR SparkContext: Error initializing SparkContext. java.lang.ClassNotFoundException: org.apache.spark.sql.connect.SparkConnectPlugin 2. Similary installing the tar balls of either distribution and launch spark-shell goes into a loop and terminated by the shutdown hook. Thank you Wenchen for leading these release onerous manager efforts, but slowly we should be able to install and launch seamlessly. Keep up the good work & tireless effort for the Spark community! cheers Jules WARNING: Using incubator modules: jdk.incubator.vector 25/03/04 14:49:35 INFO BaseAllocator: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true. 25/03/04 14:49:35 INFO DefaultAllocationManagerOption: allocation manager type not specified, using netty as the default type 25/03/04 14:49:35 INFO CheckAllocator: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception, retrying (wait=50 ms, currentRetryNum=1, policy=DefaultPolicy). 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception, retrying (wait=200 ms, currentRetryNum=2, policy=DefaultPolicy). 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception, retrying (wait=800 ms, currentRetryNum=3, policy=DefaultPolicy). 25/03/04 14:49:36 WARN GrpcRetryHandler: Non-Fatal error during RPC execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception, retrying (wait=3275 ms, currentRetryNum=4, policy=DefaultPolicy). 25/03/04 14:49:39 WARN GrpcRetryHandler: Non-Fatal error during RPC execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception, retrying (wait=12995 ms, currentRetryNum=5, policy=DefaultPolicy). ^C25/03/04 14:49:40 INFO ShutdownHookManager: Shutdown hook called > On Mar 4, 2025, at 2:24 PM, Chris Nauroth <cnaur...@apache.org> wrote: > > -1 (non-binding) > > I think I found some missing license information in the binary distribution. > We may want to include this in the next RC: > > https://github.com/apache/spark/pull/50158 > > Thank you for putting together this RC, Wenchen. > > Chris Nauroth > > > On Mon, Mar 3, 2025 at 6:10 AM Wenchen Fan <cloud0...@gmail.com > <mailto:cloud0...@gmail.com>> wrote: >> Thanks for bringing up these blockers! I know RC2 isn’t fully ready yet, but >> with over 70 commits since RC1, it’s time to have a new RC so people can >> start testing the latest changes. Please continue testing and keep the >> feedback coming! >> >> On Mon, Mar 3, 2025 at 6:06 PM beliefer <belie...@163.com >> <mailto:belie...@163.com>> wrote: >>> -1 >>> https://github.com/apache/spark/pull/50112 should be merged before release. >>> >>> >>> At 2025-03-01 15:25:06, "Wenchen Fan" <cloud0...@gmail.com >>> <mailto:cloud0...@gmail.com>> wrote: >>> >>> Please vote on releasing the following candidate as Apache Spark version >>> 4.0.0. >>> >>> The vote is open until March 5 (PST) and passes if a majority +1 PMC votes >>> are cast, with a minimum of 3 +1 votes. >>> >>> [ ] +1 Release this package as Apache Spark 4.0.0 >>> [ ] -1 Do not release this package because ... >>> >>> To learn more about Apache Spark, please see https://spark.apache.org/ >>> >>> The tag to be voted on is v4.0.0-rc2 (commit >>> 85188c07519ea809012db24421714bb75b45ab1b) >>> https://github.com/apache/spark/tree/v4.0.0-rc2 >>> >>> The release files, including signatures, digests, etc. can be found at: >>> https://dist.apache.org/repos/dist/dev/spark/v4.0.0-rc2-bin/ >>> >>> Signatures used for Spark RCs can be found in this file: >>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>> >>> The staging repository for this release can be found at: >>> https://repository.apache.org/content/repositories/orgapachespark-1478/ >>> >>> The documentation corresponding to this release can be found at: >>> https://dist.apache.org/repos/dist/dev/spark/v4.0.0-rc2-docs/ >>> >>> The list of bug fixes going into 4.0.0 can be found at the following URL: >>> https://issues.apache.org/jira/projects/SPARK/versions/12353359 >>> >>> This release is using the release script of the tag v4.0.0-rc2. >>> >>> FAQ >>> >>> ========================= >>> How can I help test this release? >>> ========================= >>> >>> If you are a Spark user, you can help us test this release by taking >>> an existing Spark workload and running on this release candidate, then >>> reporting any regressions. >>> >>> If you're working in PySpark you can set up a virtual env and install >>> the current RC and see if anything important breaks, in the Java/Scala >>> you can add the staging repository to your projects resolvers and test >>> with the RC (make sure to clean up the artifact cache before/after so >>> you don't end up building with a out of date RC going forward).
