Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Xinrong Meng]

Hi All,

Considering there are no notable release blockers now, I am going to cut
the RC2 for Apache Spark 3.4.0.

On Fri, Feb 24, 2023 at 10:44 AM Hyukjin Kwon  wrote:

> Yes we should fix. I will take a look
>
> On Thu, 23 Feb 2023 at 07:32, Jonathan Kelly 
> wrote:
>
>> Thanks! I was wondering about that ClientE2ETestSuite failure today, so
>> I'm glad to know that it's also being experienced by others.
>>
>> On a similar note, I am experiencing the following error when running the
>> Python tests with Python 3.7:
>>
>> + ./python/run-tests --python-executables=python3
>> Running PySpark tests. Output is in
>> /home/ec2-user/spark/python/unit-tests.log
>> Will test against the following Python executables: ['python3']
>> Will test the following Python modules: ['pyspark-connect',
>> 'pyspark-core', 'pyspark-errors', 'pyspark-ml', 'pyspark-mllib',
>> 'pyspark-pandas', 'pyspark-pandas-slow', 'pyspark-resource', 'pyspark-sql',
>> 'pyspark-streaming']
>> python3 python_implementation is CPython
>> python3 version is: Python 3.7.16
>> Starting test(python3): pyspark.ml.tests.test_feature (temp output:
>> /home/ec2-user/spark/python/target/8ca9ab1a-05cc-4845-bf89-30d9001510bc/python3__pyspark.ml.tests.test_feature__kg6sseie.log)
>> Starting test(python3): pyspark.ml.tests.test_base (temp output:
>> /home/ec2-user/spark/python/target/f2264f3b-6b26-4e61-9452-8d6ddd7eb002/python3__pyspark.ml.tests.test_base__0902zf9_.log)
>> Starting test(python3): pyspark.ml.tests.test_algorithms (temp output:
>> /home/ec2-user/spark/python/target/d1dc4e07-e58c-4c03-abe5-09d8fab22e6a/python3__pyspark.ml.tests.test_algorithms__lh3wb2u8.log)
>> Starting test(python3): pyspark.ml.tests.test_evaluation (temp output:
>> /home/ec2-user/spark/python/target/3f42dc79-c945-4cf2-a1eb-83e72b40a9ee/python3__pyspark.ml.tests.test_evaluation__89idc7fa.log)
>> Finished test(python3): pyspark.ml.tests.test_base (16s)
>> Starting test(python3): pyspark.ml.tests.test_functions (temp output:
>> /home/ec2-user/spark/python/target/5a3b90f0-216b-4edd-9d15-6619d3e03300/python3__pyspark.ml.tests.test_functions__g5u1290s.log)
>> Traceback (most recent call last):
>>   File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
>> "__main__", mod_spec)
>>   File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
>> exec(code, run_globals)
>>   File "/home/ec2-user/spark/python/pyspark/ml/tests/test_functions.py",
>> line 21, in 
>> from pyspark.ml.functions import predict_batch_udf
>>   File "/home/ec2-user/spark/python/pyspark/ml/functions.py", line 38, in
>> 
>> from typing import Any, Callable, Iterator, List, Mapping, Protocol,
>> TYPE_CHECKING, Tuple, Union
>> ImportError: cannot import name 'Protocol' from 'typing'
>> (/usr/lib64/python3.7/typing.py)
>> Had test failures in pyspark.ml.tests.test_functions with python3; see
>> logs.
>>
>> I know we should move on to a newer version of Python, but isn't Python
>> 3.7 still officially supported?
>>
>> Thank you,
>> Jonathan Kelly
>>
>> On Wed, Feb 22, 2023 at 1:47 PM Herman van Hovell
>>  wrote:
>>
>>> Hi All,
>>>
>>> Thanks for testing the 3.4.0 RC! I apologize for the maven testing
>>> failures for the Spark Connect Scala Client. We will try to get those
>>> sorted as soon as possible.
>>>
>>> This is an artifact of having multiple build systems, and only running
>>> CI for one (SBT). That, however, is a debate for another day :)...
>>>
>>> Cheers,
>>> Herman
>>>
>>> On Wed, Feb 22, 2023 at 5:32 PM Bjørn Jørgensen <
>>> bjornjorgen...@gmail.com> wrote:
>>>
 ./build/mvn clean package

 I'm using ubuntu rolling, python 3.11 openjdk 17

 CompatibilitySuite:
 - compatibility MiMa tests *** FAILED ***
   java.lang.AssertionError: assertion failed: Failed to find the jar
 inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
   at scala.Predef$.assert(Predef.scala:223)
   at
 org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
   at
 org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
   at
 org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
   at
 org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$1(CompatibilitySuite.scala:69)
   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
   at org.scalatest.Transformer.apply(Transformer.scala:22)
   at org.scalatest.Transformer.apply(Transformer.scala:20)
   ...
 - compatibility API tests: Dataset *** FAILED ***
   java.lang.AssertionError: assertion failed: Failed to find the jar
 inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
   at scala.Predef$.assert(Predef.scala:223)
   at
 

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Hyukjin Kwon]

Yes we should fix. I will take a look

On Thu, 23 Feb 2023 at 07:32, Jonathan Kelly  wrote:

> Thanks! I was wondering about that ClientE2ETestSuite failure today, so
> I'm glad to know that it's also being experienced by others.
>
> On a similar note, I am experiencing the following error when running the
> Python tests with Python 3.7:
>
> + ./python/run-tests --python-executables=python3
> Running PySpark tests. Output is in
> /home/ec2-user/spark/python/unit-tests.log
> Will test against the following Python executables: ['python3']
> Will test the following Python modules: ['pyspark-connect',
> 'pyspark-core', 'pyspark-errors', 'pyspark-ml', 'pyspark-mllib',
> 'pyspark-pandas', 'pyspark-pandas-slow', 'pyspark-resource', 'pyspark-sql',
> 'pyspark-streaming']
> python3 python_implementation is CPython
> python3 version is: Python 3.7.16
> Starting test(python3): pyspark.ml.tests.test_feature (temp output:
> /home/ec2-user/spark/python/target/8ca9ab1a-05cc-4845-bf89-30d9001510bc/python3__pyspark.ml.tests.test_feature__kg6sseie.log)
> Starting test(python3): pyspark.ml.tests.test_base (temp output:
> /home/ec2-user/spark/python/target/f2264f3b-6b26-4e61-9452-8d6ddd7eb002/python3__pyspark.ml.tests.test_base__0902zf9_.log)
> Starting test(python3): pyspark.ml.tests.test_algorithms (temp output:
> /home/ec2-user/spark/python/target/d1dc4e07-e58c-4c03-abe5-09d8fab22e6a/python3__pyspark.ml.tests.test_algorithms__lh3wb2u8.log)
> Starting test(python3): pyspark.ml.tests.test_evaluation (temp output:
> /home/ec2-user/spark/python/target/3f42dc79-c945-4cf2-a1eb-83e72b40a9ee/python3__pyspark.ml.tests.test_evaluation__89idc7fa.log)
> Finished test(python3): pyspark.ml.tests.test_base (16s)
> Starting test(python3): pyspark.ml.tests.test_functions (temp output:
> /home/ec2-user/spark/python/target/5a3b90f0-216b-4edd-9d15-6619d3e03300/python3__pyspark.ml.tests.test_functions__g5u1290s.log)
> Traceback (most recent call last):
>   File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
> "__main__", mod_spec)
>   File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
> exec(code, run_globals)
>   File "/home/ec2-user/spark/python/pyspark/ml/tests/test_functions.py",
> line 21, in 
> from pyspark.ml.functions import predict_batch_udf
>   File "/home/ec2-user/spark/python/pyspark/ml/functions.py", line 38, in
> 
> from typing import Any, Callable, Iterator, List, Mapping, Protocol,
> TYPE_CHECKING, Tuple, Union
> ImportError: cannot import name 'Protocol' from 'typing'
> (/usr/lib64/python3.7/typing.py)
> Had test failures in pyspark.ml.tests.test_functions with python3; see
> logs.
>
> I know we should move on to a newer version of Python, but isn't Python
> 3.7 still officially supported?
>
> Thank you,
> Jonathan Kelly
>
> On Wed, Feb 22, 2023 at 1:47 PM Herman van Hovell
>  wrote:
>
>> Hi All,
>>
>> Thanks for testing the 3.4.0 RC! I apologize for the maven testing
>> failures for the Spark Connect Scala Client. We will try to get those
>> sorted as soon as possible.
>>
>> This is an artifact of having multiple build systems, and only running CI
>> for one (SBT). That, however, is a debate for another day :)...
>>
>> Cheers,
>> Herman
>>
>> On Wed, Feb 22, 2023 at 5:32 PM Bjørn Jørgensen 
>> wrote:
>>
>>> ./build/mvn clean package
>>>
>>> I'm using ubuntu rolling, python 3.11 openjdk 17
>>>
>>> CompatibilitySuite:
>>> - compatibility MiMa tests *** FAILED ***
>>>   java.lang.AssertionError: assertion failed: Failed to find the jar
>>> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>>>   at scala.Predef$.assert(Predef.scala:223)
>>>   at
>>> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>>>   at
>>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>>>   at
>>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
>>>   at
>>> org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$1(CompatibilitySuite.scala:69)
>>>   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
>>>   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
>>>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>>>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>>>   at org.scalatest.Transformer.apply(Transformer.scala:20)
>>>   ...
>>> - compatibility API tests: Dataset *** FAILED ***
>>>   java.lang.AssertionError: assertion failed: Failed to find the jar
>>> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>>>   at scala.Predef$.assert(Predef.scala:223)
>>>   at
>>> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>>>   at
>>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>>>   at
>>> 

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Gengliang Wang]

Thanks for creating the RC1, Xinrong!

Besides the blockers mentioned by Tom, let's include the following bug fix
in Spark 3.4.0 as well:
[SPARK-42406][SQL] Fix check for missing required fields of to_protobuf


Gengliang

On Wed, Feb 22, 2023 at 3:09 PM Tom Graves 
wrote:

> It looks like there are still blockers open, we need to make sure they are
> addressed before doing a release:
>
> https://issues.apache.org/jira/browse/SPARK-41793
> https://issues.apache.org/jira/browse/SPARK-42444
>
> Tom
> On Tuesday, February 21, 2023 at 10:35:45 PM CST, Xinrong Meng <
> xinrong.apa...@gmail.com> wrote:
>
>
> Please vote on releasing the following candidate as Apache Spark version
> 3.4.0.
>
> The vote is open until 11:59pm Pacific time *February 27th* and passes if
> a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is *v3.4.0-rc1* (commit
> e2484f626bb338274665a49078b528365ea18c3b):
> https://github.com/apache/spark/tree/v3.4.0-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1435
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>
> This release is using the release script of the tag v3.4.0-rc1.
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Thanks,
> Xinrong Meng
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Tom Graves]

 It looks like there are still blockers open, we need to make sure they are 
addressed before doing a release:
https://issues.apache.org/jira/browse/SPARK-41793
https://issues.apache.org/jira/browse/SPARK-42444

TomOn Tuesday, February 21, 2023 at 10:35:45 PM CST, Xinrong Meng 
 wrote:  
 
 Please vote on releasing the following candidate as Apache Spark version 3.4.0.

The vote is open until 11:59pm Pacific time February 27th and passes if a 
majority +1 PMC votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 3.4.0
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see http://spark.apache.org/

The tag to be voted on is v3.4.0-rc1 (commit 
e2484f626bb338274665a49078b528365ea18c3b):
https://github.com/apache/spark/tree/v3.4.0-rc1

The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1435

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/

The list of bug fixes going into 3.4.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12351465

This release is using the release script of the tag v3.4.0-rc1.


FAQ

=
How can I help test this release?
=
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.

If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with a out of date RC going forward).

===
What should happen to JIRA tickets still targeting 3.4.0?
===
The current list of open tickets targeted at 3.4.0 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" 
= 3.4.0

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.

Thanks,
Xinrong Meng
  

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Jonathan Kelly]

Thanks! I was wondering about that ClientE2ETestSuite failure today, so I'm
glad to know that it's also being experienced by others.

On a similar note, I am experiencing the following error when running the
Python tests with Python 3.7:

+ ./python/run-tests --python-executables=python3
Running PySpark tests. Output is in
/home/ec2-user/spark/python/unit-tests.log
Will test against the following Python executables: ['python3']
Will test the following Python modules: ['pyspark-connect', 'pyspark-core',
'pyspark-errors', 'pyspark-ml', 'pyspark-mllib', 'pyspark-pandas',
'pyspark-pandas-slow', 'pyspark-resource', 'pyspark-sql',
'pyspark-streaming']
python3 python_implementation is CPython
python3 version is: Python 3.7.16
Starting test(python3): pyspark.ml.tests.test_feature (temp output:
/home/ec2-user/spark/python/target/8ca9ab1a-05cc-4845-bf89-30d9001510bc/python3__pyspark.ml.tests.test_feature__kg6sseie.log)
Starting test(python3): pyspark.ml.tests.test_base (temp output:
/home/ec2-user/spark/python/target/f2264f3b-6b26-4e61-9452-8d6ddd7eb002/python3__pyspark.ml.tests.test_base__0902zf9_.log)
Starting test(python3): pyspark.ml.tests.test_algorithms (temp output:
/home/ec2-user/spark/python/target/d1dc4e07-e58c-4c03-abe5-09d8fab22e6a/python3__pyspark.ml.tests.test_algorithms__lh3wb2u8.log)
Starting test(python3): pyspark.ml.tests.test_evaluation (temp output:
/home/ec2-user/spark/python/target/3f42dc79-c945-4cf2-a1eb-83e72b40a9ee/python3__pyspark.ml.tests.test_evaluation__89idc7fa.log)
Finished test(python3): pyspark.ml.tests.test_base (16s)
Starting test(python3): pyspark.ml.tests.test_functions (temp output:
/home/ec2-user/spark/python/target/5a3b90f0-216b-4edd-9d15-6619d3e03300/python3__pyspark.ml.tests.test_functions__g5u1290s.log)
Traceback (most recent call last):
  File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
  File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
  File "/home/ec2-user/spark/python/pyspark/ml/tests/test_functions.py",
line 21, in 
from pyspark.ml.functions import predict_batch_udf
  File "/home/ec2-user/spark/python/pyspark/ml/functions.py", line 38, in

from typing import Any, Callable, Iterator, List, Mapping, Protocol,
TYPE_CHECKING, Tuple, Union
ImportError: cannot import name 'Protocol' from 'typing'
(/usr/lib64/python3.7/typing.py)
Had test failures in pyspark.ml.tests.test_functions with python3; see logs.

I know we should move on to a newer version of Python, but isn't Python 3.7
still officially supported?

Thank you,
Jonathan Kelly

On Wed, Feb 22, 2023 at 1:47 PM Herman van Hovell
 wrote:

> Hi All,
>
> Thanks for testing the 3.4.0 RC! I apologize for the maven testing
> failures for the Spark Connect Scala Client. We will try to get those
> sorted as soon as possible.
>
> This is an artifact of having multiple build systems, and only running CI
> for one (SBT). That, however, is a debate for another day :)...
>
> Cheers,
> Herman
>
> On Wed, Feb 22, 2023 at 5:32 PM Bjørn Jørgensen 
> wrote:
>
>> ./build/mvn clean package
>>
>> I'm using ubuntu rolling, python 3.11 openjdk 17
>>
>> CompatibilitySuite:
>> - compatibility MiMa tests *** FAILED ***
>>   java.lang.AssertionError: assertion failed: Failed to find the jar
>> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>>   at scala.Predef$.assert(Predef.scala:223)
>>   at
>> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$1(CompatibilitySuite.scala:69)
>>   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
>>   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
>>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>>   at org.scalatest.Transformer.apply(Transformer.scala:20)
>>   ...
>> - compatibility API tests: Dataset *** FAILED ***
>>   java.lang.AssertionError: assertion failed: Failed to find the jar
>> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>>   at scala.Predef$.assert(Predef.scala:223)
>>   at
>> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
>>   at
>> org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$7(CompatibilitySuite.scala:110)
>>   at
>> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
>>   at 

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Herman van Hovell]

Hi All,

Thanks for testing the 3.4.0 RC! I apologize for the maven testing failures
for the Spark Connect Scala Client. We will try to get those sorted as soon
as possible.

This is an artifact of having multiple build systems, and only running CI
for one (SBT). That, however, is a debate for another day :)...

Cheers,
Herman

On Wed, Feb 22, 2023 at 5:32 PM Bjørn Jørgensen 
wrote:

> ./build/mvn clean package
>
> I'm using ubuntu rolling, python 3.11 openjdk 17
>
> CompatibilitySuite:
> - compatibility MiMa tests *** FAILED ***
>   java.lang.AssertionError: assertion failed: Failed to find the jar
> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>   at scala.Predef$.assert(Predef.scala:223)
>   at
> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$1(CompatibilitySuite.scala:69)
>   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   at org.scalatest.Transformer.apply(Transformer.scala:20)
>   ...
> - compatibility API tests: Dataset *** FAILED ***
>   java.lang.AssertionError: assertion failed: Failed to find the jar
> inside folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
>   at scala.Predef$.assert(Predef.scala:223)
>   at
> org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
>   at
> org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$7(CompatibilitySuite.scala:110)
>   at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
>   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   ...
> SparkConnectClientSuite:
> - Placeholder test: Create SparkConnectClient
> - Test connection
> - Test connection string
> - Check URI: sc://host, isCorrect: true
> - Check URI: sc://localhost/, isCorrect: true
> - Check URI: sc://localhost:1234/, isCorrect: true
> - Check URI: sc://localhost/;, isCorrect: true
> - Check URI: sc://host:123, isCorrect: true
> - Check URI: sc://host:123/;user_id=a94, isCorrect: true
> - Check URI: scc://host:12, isCorrect: false
> - Check URI: http://host, isCorrect: false
> - Check URI: sc:/host:1234/path, isCorrect: false
> - Check URI: sc://host/path, isCorrect: false
> - Check URI: sc://host/;parm1;param2, isCorrect: false
> - Check URI: sc://host:123;user_id=a94, isCorrect: false
> - Check URI: sc:///user_id=123, isCorrect: false
> - Check URI: sc://host:-4, isCorrect: false
> - Check URI: sc://:123/, isCorrect: false
> - Non user-id parameters throw unsupported errors
> DatasetSuite:
> - limit
> - select
> - filter
> - write
> UserDefinedFunctionSuite:
> - udf and encoder serialization
> Run completed in 21 seconds, 944 milliseconds.
> Total number of tests run: 389
> Suites: completed 10, aborted 0
> Tests: succeeded 386, failed 3, canceled 0, ignored 0, pending 0
> *** 3 TESTS FAILED ***
> [INFO]
> 
> [INFO] Reactor Summary for Spark Project Parent POM 3.4.0:
> [INFO]
> [INFO] Spark Project Parent POM ... SUCCESS [
> 47.096 s]
> [INFO] Spark Project Tags . SUCCESS [
> 14.759 s]
> [INFO] Spark Project Sketch ... SUCCESS [
> 21.628 s]
> [INFO] Spark Project Local DB . SUCCESS [
> 20.311 s]
> [INFO] Spark Project Networking ... SUCCESS [01:07
> min]
> [INFO] Spark Project Shuffle Streaming Service  SUCCESS [
> 15.921 s]
> [INFO] Spark Project Unsafe ... SUCCESS [
> 16.020 s]
> [INFO] Spark Project Launcher . SUCCESS [
> 10.873 s]
> [INFO] Spark Project Core . SUCCESS [37:10
> min]
> [INFO] Spark Project ML Local Library . SUCCESS [
> 40.841 s]
> [INFO] Spark Project GraphX ... SUCCESS [02:39
> min]
> [INFO] Spark Project Streaming  SUCCESS [05:53
> min]
> [INFO] Spark Project Catalyst . SUCCESS [11:22
> min]
> 

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Bjørn Jørgensen]

./build/mvn clean package

I'm using ubuntu rolling, python 3.11 openjdk 17

CompatibilitySuite:
- compatibility MiMa tests *** FAILED ***
  java.lang.AssertionError: assertion failed: Failed to find the jar inside
folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
  at scala.Predef$.assert(Predef.scala:223)
  at
org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$1(CompatibilitySuite.scala:69)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  ...
- compatibility API tests: Dataset *** FAILED ***
  java.lang.AssertionError: assertion failed: Failed to find the jar inside
folder: /home/bjorn/spark-3.4.0/connector/connect/client/jvm/target
  at scala.Predef$.assert(Predef.scala:223)
  at
org.apache.spark.sql.connect.client.util.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:67)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar$lzycompute(CompatibilitySuite.scala:57)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.clientJar(CompatibilitySuite.scala:53)
  at
org.apache.spark.sql.connect.client.CompatibilitySuite.$anonfun$new$7(CompatibilitySuite.scala:110)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  ...
SparkConnectClientSuite:
- Placeholder test: Create SparkConnectClient
- Test connection
- Test connection string
- Check URI: sc://host, isCorrect: true
- Check URI: sc://localhost/, isCorrect: true
- Check URI: sc://localhost:1234/, isCorrect: true
- Check URI: sc://localhost/;, isCorrect: true
- Check URI: sc://host:123, isCorrect: true
- Check URI: sc://host:123/;user_id=a94, isCorrect: true
- Check URI: scc://host:12, isCorrect: false
- Check URI: http://host, isCorrect: false
- Check URI: sc:/host:1234/path, isCorrect: false
- Check URI: sc://host/path, isCorrect: false
- Check URI: sc://host/;parm1;param2, isCorrect: false
- Check URI: sc://host:123;user_id=a94, isCorrect: false
- Check URI: sc:///user_id=123, isCorrect: false
- Check URI: sc://host:-4, isCorrect: false
- Check URI: sc://:123/, isCorrect: false
- Non user-id parameters throw unsupported errors
DatasetSuite:
- limit
- select
- filter
- write
UserDefinedFunctionSuite:
- udf and encoder serialization
Run completed in 21 seconds, 944 milliseconds.
Total number of tests run: 389
Suites: completed 10, aborted 0
Tests: succeeded 386, failed 3, canceled 0, ignored 0, pending 0
*** 3 TESTS FAILED ***
[INFO]

[INFO] Reactor Summary for Spark Project Parent POM 3.4.0:
[INFO]
[INFO] Spark Project Parent POM ... SUCCESS [
47.096 s]
[INFO] Spark Project Tags . SUCCESS [
14.759 s]
[INFO] Spark Project Sketch ... SUCCESS [
21.628 s]
[INFO] Spark Project Local DB . SUCCESS [
20.311 s]
[INFO] Spark Project Networking ... SUCCESS [01:07
min]
[INFO] Spark Project Shuffle Streaming Service  SUCCESS [
15.921 s]
[INFO] Spark Project Unsafe ... SUCCESS [
16.020 s]
[INFO] Spark Project Launcher . SUCCESS [
10.873 s]
[INFO] Spark Project Core . SUCCESS [37:10
min]
[INFO] Spark Project ML Local Library . SUCCESS [
40.841 s]
[INFO] Spark Project GraphX ... SUCCESS [02:39
min]
[INFO] Spark Project Streaming  SUCCESS [05:53
min]
[INFO] Spark Project Catalyst . SUCCESS [11:22
min]
[INFO] Spark Project SQL .. SUCCESS [
 02:27 h]
[INFO] Spark Project ML Library ... SUCCESS [22:45
min]
[INFO] Spark Project Tools  SUCCESS [
 7.263 s]
[INFO] Spark Project Hive . SUCCESS [
 01:21 h]
[INFO] Spark Project REPL . SUCCESS [02:07
min]
[INFO] Spark Project Assembly . SUCCESS [
11.704 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ... SUCCESS [
26.748 s]
[INFO] Spark Integration for Kafka 

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Mridul Muralidharan]

Signatures, digests, etc check out fine - thanks for updating them !
Checked out tag and build/tested with -Phive -Pyarn -Pmesos -Pkubernetes


The test ClientE2ETestSuite.simple udf failed [1] in "Connect Client "
module ... yet to test "Spark Protobuf" module due to the failure.


Regards,
Mridul

[1]

- simple udf *** FAILED ***

  io.grpc.StatusRuntimeException: INTERNAL:
org.apache.spark.sql.ClientE2ETestSuite

  at io.grpc.Status.asRuntimeException(Status.java:535)

  at
io.grpc.stub.ClientCalls$BlockingResponseStream.hasNext(ClientCalls.java:660)

  at org.apache.spark.sql.connect.client.SparkResult.org
$apache$spark$sql$connect$client$SparkResult$$processResponses(SparkResult.scala:50)

  at
org.apache.spark.sql.connect.client.SparkResult.length(SparkResult.scala:95)

  at
org.apache.spark.sql.connect.client.SparkResult.toArray(SparkResult.scala:112)

  at org.apache.spark.sql.Dataset.$anonfun$collect$1(Dataset.scala:2037)

  at org.apache.spark.sql.Dataset.withResult(Dataset.scala:2267)

  at org.apache.spark.sql.Dataset.collect(Dataset.scala:2036)

  at
org.apache.spark.sql.ClientE2ETestSuite.$anonfun$new$5(ClientE2ETestSuite.scala:65)

  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)

  ...





On Wed, Feb 22, 2023 at 2:07 AM Mridul Muralidharan 
wrote:

>
> Thanks Xinrong !
> The signature verifications are fine now ... will continue with testing
> the release.
>
>
> Regards,
> Mridul
>
>
> On Wed, Feb 22, 2023 at 1:27 AM Xinrong Meng 
> wrote:
>
>> Hi Mridul,
>>
>> Would you please try that again? It should work now.
>>
>> On Wed, Feb 22, 2023 at 2:04 PM Mridul Muralidharan 
>> wrote:
>>
>>>
>>> Hi Xinrong,
>>>
>>>   Was it signed with the same key as present in KEYS [1] ?
>>> I am seeing errors with gpg when validating. For example:
>>>
>>>
>>> $ gpg --verify pyspark-3.4.0.tar.gz.asc
>>>
>>> gpg: assuming signed data in 'pyspark-3.4.0.tar.gz'
>>>
>>> gpg: Signature made Tue 21 Feb 2023 05:56:05 AM CST
>>>
>>> gpg:using RSA key
>>> CC68B3D16FE33A766705160BA7E57908C7A4E1B1
>>>
>>> gpg:issuer "xinr...@apache.org"
>>>
>>> gpg: Can't check signature: No public key
>>>
>>>
>>>
>>> Regards,
>>> Mridul
>>>
>>> [1] https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>>
>>> On Tue, Feb 21, 2023 at 10:36 PM Xinrong Meng 
>>> wrote:
>>>
 Please vote on releasing the following candidate as Apache Spark
 version 3.4.0.

 The vote is open until 11:59pm Pacific time *February 27th* and passes
 if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.

 [ ] +1 Release this package as Apache Spark 3.4.0
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see http://spark.apache.org/

 The tag to be voted on is *v3.4.0-rc1* (commit
 e2484f626bb338274665a49078b528365ea18c3b):
 https://github.com/apache/spark/tree/v3.4.0-rc1

 The release files, including signatures, digests, etc. can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/

 Signatures used for Spark RCs can be found in this file:
 https://dist.apache.org/repos/dist/dev/spark/KEYS

 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-1435

 The documentation corresponding to this release can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/

 The list of bug fixes going into 3.4.0 can be found at the following
 URL:
 https://issues.apache.org/jira/projects/SPARK/versions/12351465

 This release is using the release script of the tag v3.4.0-rc1.


 FAQ

 =
 How can I help test this release?
 =
 If you are a Spark user, you can help us test this release by taking
 an existing Spark workload and running on this release candidate, then
 reporting any regressions.

 If you're working in PySpark you can set up a virtual env and install
 the current RC and see if anything important breaks, in the Java/Scala
 you can add the staging repository to your projects resolvers and test
 with the RC (make sure to clean up the artifact cache before/after so
 you don't end up building with a out of date RC going forward).

 ===
 What should happen to JIRA tickets still targeting 3.4.0?
 ===
 The current list of open tickets targeted at 3.4.0 can be found at:
 https://issues.apache.org/jira/projects/SPARK and search for "Target
 Version/s" = 3.4.0

 Committers should look at those and triage. Extremely important bug
 fixes, documentation, and API tweaks that impact compatibility should
 be worked on immediately. Everything else please retarget to an
 appropriate release.

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Mridul Muralidharan]

Thanks Xinrong !
The signature verifications are fine now ... will continue with testing the
release.


Regards,
Mridul


On Wed, Feb 22, 2023 at 1:27 AM Xinrong Meng 
wrote:

> Hi Mridul,
>
> Would you please try that again? It should work now.
>
> On Wed, Feb 22, 2023 at 2:04 PM Mridul Muralidharan 
> wrote:
>
>>
>> Hi Xinrong,
>>
>>   Was it signed with the same key as present in KEYS [1] ?
>> I am seeing errors with gpg when validating. For example:
>>
>>
>> $ gpg --verify pyspark-3.4.0.tar.gz.asc
>>
>> gpg: assuming signed data in 'pyspark-3.4.0.tar.gz'
>>
>> gpg: Signature made Tue 21 Feb 2023 05:56:05 AM CST
>>
>> gpg:using RSA key
>> CC68B3D16FE33A766705160BA7E57908C7A4E1B1
>>
>> gpg:issuer "xinr...@apache.org"
>>
>> gpg: Can't check signature: No public key
>>
>>
>>
>> Regards,
>> Mridul
>>
>> [1] https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>>
>> On Tue, Feb 21, 2023 at 10:36 PM Xinrong Meng 
>> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 3.4.0.
>>>
>>> The vote is open until 11:59pm Pacific time *February 27th* and passes
>>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 3.4.0
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>
>>> The tag to be voted on is *v3.4.0-rc1* (commit
>>> e2484f626bb338274665a49078b528365ea18c3b):
>>> https://github.com/apache/spark/tree/v3.4.0-rc1
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1435
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/
>>>
>>> The list of bug fixes going into 3.4.0 can be found at the following URL:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>>>
>>> This release is using the release script of the tag v3.4.0-rc1.
>>>
>>>
>>> FAQ
>>>
>>> =
>>> How can I help test this release?
>>> =
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala
>>> you can add the staging repository to your projects resolvers and test
>>> with the RC (make sure to clean up the artifact cache before/after so
>>> you don't end up building with a out of date RC going forward).
>>>
>>> ===
>>> What should happen to JIRA tickets still targeting 3.4.0?
>>> ===
>>> The current list of open tickets targeted at 3.4.0 can be found at:
>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> Version/s" = 3.4.0
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>>
>>> ==
>>> But my bug isn't fixed?
>>> ==
>>> In order to make timely releases, we will typically not hold the
>>> release unless the bug in question is a regression from the previous
>>> release. That being said, if there is something which is a regression
>>> that has not been correctly targeted please ping me or a committer to
>>> help target the issue.
>>>
>>> Thanks,
>>> Xinrong Meng
>>>
>>

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Xinrong Meng]

Hi Mridul,

Would you please try that again? It should work now.

On Wed, Feb 22, 2023 at 2:04 PM Mridul Muralidharan 
wrote:

>
> Hi Xinrong,
>
>   Was it signed with the same key as present in KEYS [1] ?
> I am seeing errors with gpg when validating. For example:
>
>
> $ gpg --verify pyspark-3.4.0.tar.gz.asc
>
> gpg: assuming signed data in 'pyspark-3.4.0.tar.gz'
>
> gpg: Signature made Tue 21 Feb 2023 05:56:05 AM CST
>
> gpg:using RSA key CC68B3D16FE33A766705160BA7E57908C7A4E1B1
>
> gpg:issuer "xinr...@apache.org"
>
> gpg: Can't check signature: No public key
>
>
>
> Regards,
> Mridul
>
> [1] https://dist.apache.org/repos/dist/dev/spark/KEYS
>
>
> On Tue, Feb 21, 2023 at 10:36 PM Xinrong Meng 
> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 3.4.0.
>>
>> The vote is open until 11:59pm Pacific time *February 27th* and passes
>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.4.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> The tag to be voted on is *v3.4.0-rc1* (commit
>> e2484f626bb338274665a49078b528365ea18c3b):
>> https://github.com/apache/spark/tree/v3.4.0-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1435
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/
>>
>> The list of bug fixes going into 3.4.0 can be found at the following URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>>
>> This release is using the release script of the tag v3.4.0-rc1.
>>
>>
>> FAQ
>>
>> =
>> How can I help test this release?
>> =
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with a out of date RC going forward).
>>
>> ===
>> What should happen to JIRA tickets still targeting 3.4.0?
>> ===
>> The current list of open tickets targeted at 3.4.0 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 3.4.0
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==
>> But my bug isn't fixed?
>> ==
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>> Thanks,
>> Xinrong Meng
>>
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

[Mridul Muralidharan]

Hi Xinrong,

  Was it signed with the same key as present in KEYS [1] ?
I am seeing errors with gpg when validating. For example:


$ gpg --verify pyspark-3.4.0.tar.gz.asc

gpg: assuming signed data in 'pyspark-3.4.0.tar.gz'

gpg: Signature made Tue 21 Feb 2023 05:56:05 AM CST

gpg:using RSA key CC68B3D16FE33A766705160BA7E57908C7A4E1B1

gpg:issuer "xinr...@apache.org"

gpg: Can't check signature: No public key



Regards,
Mridul

[1] https://dist.apache.org/repos/dist/dev/spark/KEYS


On Tue, Feb 21, 2023 at 10:36 PM Xinrong Meng 
wrote:

> Please vote on releasing the following candidate as Apache Spark version
> 3.4.0.
>
> The vote is open until 11:59pm Pacific time *February 27th* and passes if
> a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is *v3.4.0-rc1* (commit
> e2484f626bb338274665a49078b528365ea18c3b):
> https://github.com/apache/spark/tree/v3.4.0-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1435
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc1-docs/
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>
> This release is using the release script of the tag v3.4.0-rc1.
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Thanks,
> Xinrong Meng
>