RC2 fails and I'll cut RC3 next week. Thanks for the feedback! On Thu, Mar 6, 2025 at 6:44 AM Chris Nauroth <cnaur...@apache.org> wrote:
> Here is one more problem I found during RC2 verification: > > https://github.com/apache/spark/pull/50173 > > This one is just a test issue. > > Chris Nauroth > > > On Tue, Mar 4, 2025 at 2:55 PM Jules Damji <jules.da...@gmail.com> wrote: > >> - 1 (non-binding) >> >> A ran into number of installation and launching problems. May be it’s my >> enviornment, even though I removed any old binaries and packages. >> >> 1. Pip installing pyspark4.0.0 and pyspark-connect-4.0 from .tz file >> workedl, launching pyspark results into >> >> 25/03/04 14:00:26 ERROR SparkContext: Error initializing SparkContext. >> >> java.lang.ClassNotFoundException: >> org.apache.spark.sql.connect.SparkConnectPlugin >> >> >> 2. Similary installing the tar balls of either distribution and launch >> spark-shell goes into a loop and terminated by the shutdown hook. >> >> >> Thank you Wenchen for leading these release onerous manager efforts, but >> slowly we should be able to install and launch seamlessly. >> >> >> Keep up the good work & tireless effort for the Spark community! >> >> >> cheers >> >> Jules >> >> >> WARNING: Using incubator modules: jdk.incubator.vector >> >> 25/03/04 14:49:35 INFO BaseAllocator: Debug mode disabled. Enable with >> the VM option -Darrow.memory.debug.allocator=true. >> >> 25/03/04 14:49:35 INFO DefaultAllocationManagerOption: allocation manager >> type not specified, using netty as the default type >> >> 25/03/04 14:49:35 INFO CheckAllocator: Using DefaultAllocationManager at >> memory/netty/DefaultAllocationManagerFactory.class >> >> Using Spark's default log4j profile: >> org/apache/spark/log4j2-defaults.properties >> >> 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC >> execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io >> exception, retrying (wait=50 ms, currentRetryNum=1, policy=DefaultPolicy). >> >> 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC >> execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io >> exception, retrying (wait=200 ms, currentRetryNum=2, policy=DefaultPolicy). >> >> 25/03/04 14:49:35 WARN GrpcRetryHandler: Non-Fatal error during RPC >> execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io >> exception, retrying (wait=800 ms, currentRetryNum=3, policy=DefaultPolicy). >> >> 25/03/04 14:49:36 WARN GrpcRetryHandler: Non-Fatal error during RPC >> execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io >> exception, retrying (wait=3275 ms, currentRetryNum=4, policy=DefaultPolicy). >> >> 25/03/04 14:49:39 WARN GrpcRetryHandler: Non-Fatal error during RPC >> execution: org.sparkproject.io.grpc.StatusRuntimeException: UNAVAILABLE: io >> exception, retrying (wait=12995 ms, currentRetryNum=5, >> policy=DefaultPolicy). >> >> ^C25/03/04 14:49:40 INFO ShutdownHookManager: Shutdown hook called >> >> >> >> On Mar 4, 2025, at 2:24 PM, Chris Nauroth <cnaur...@apache.org> wrote: >> >> -1 (non-binding) >> >> I think I found some missing license information in the binary >> distribution. We may want to include this in the next RC: >> >> https://github.com/apache/spark/pull/50158 >> >> Thank you for putting together this RC, Wenchen. >> >> Chris Nauroth >> >> >> On Mon, Mar 3, 2025 at 6:10 AM Wenchen Fan <cloud0...@gmail.com> wrote: >> >>> Thanks for bringing up these blockers! I know RC2 isn’t fully ready yet, >>> but with over 70 commits since RC1, it’s time to have a new RC so people >>> can start testing the latest changes. Please continue testing and keep the >>> feedback coming! >>> >>> On Mon, Mar 3, 2025 at 6:06 PM beliefer <belie...@163.com> wrote: >>> >>>> -1 >>>> https://github.com/apache/spark/pull/50112 should be merged before >>>> release. >>>> >>>> >>>> At 2025-03-01 15:25:06, "Wenchen Fan" <cloud0...@gmail.com> wrote: >>>> >>>> Please vote on releasing the following candidate as Apache Spark >>>> version 4.0.0. >>>> >>>> The vote is open until March 5 (PST) and passes if a majority +1 PMC >>>> votes are cast, with a minimum of 3 +1 votes. >>>> >>>> [ ] +1 Release this package as Apache Spark 4.0.0 >>>> [ ] -1 Do not release this package because ... >>>> >>>> To learn more about Apache Spark, please see https://spark.apache.org/ >>>> >>>> The tag to be voted on is v4.0.0-rc2 (commit >>>> 85188c07519ea809012db24421714bb75b45ab1b) >>>> https://github.com/apache/spark/tree/v4.0.0-rc2 >>>> >>>> The release files, including signatures, digests, etc. can be found at: >>>> https://dist.apache.org/repos/dist/dev/spark/v4.0.0-rc2-bin/ >>>> >>>> Signatures used for Spark RCs can be found in this file: >>>> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>> >>>> The staging repository for this release can be found at: >>>> https://repository.apache.org/content/repositories/orgapachespark-1478/ >>>> >>>> The documentation corresponding to this release can be found at: >>>> https://dist.apache.org/repos/dist/dev/spark/v4.0.0-rc2-docs/ >>>> >>>> The list of bug fixes going into 4.0.0 can be found at the following >>>> URL: >>>> https://issues.apache.org/jira/projects/SPARK/versions/12353359 >>>> >>>> This release is using the release script of the tag v4.0.0-rc2. >>>> >>>> FAQ >>>> >>>> ========================= >>>> How can I help test this release? >>>> ========================= >>>> >>>> If you are a Spark user, you can help us test this release by taking >>>> an existing Spark workload and running on this release candidate, then >>>> reporting any regressions. >>>> >>>> If you're working in PySpark you can set up a virtual env and install >>>> the current RC and see if anything important breaks, in the Java/Scala >>>> you can add the staging repository to your projects resolvers and test >>>> with the RC (make sure to clean up the artifact cache before/after so >>>> you don't end up building with a out of date RC going forward). >>>> >>>> >>
