[VOTE] Release Apache Spark 3.4.0 (RC7)

[Xinrong Meng]

Please vote on releasing the following candidate(RC7) as Apache Spark
version 3.4.0.

The vote is open until 11:59pm Pacific time *April 12th* and passes if a
majority +1 PMC votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 3.4.0
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see http://spark.apache.org/

The tag to be voted on is v3.4.0-rc7 (commit
87a5442f7ed96b11051d8a9333476d080054e5a0):
https://github.com/apache/spark/tree/v3.4.0-rc7

The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1441

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-docs/

The list of bug fixes going into 3.4.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12351465

This release is using the release script of the tag v3.4.0-rc7.


FAQ

=
How can I help test this release?
=
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.

If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).

===
What should happen to JIRA tickets still targeting 3.4.0?
===
The current list of open tickets targeted at 3.4.0 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 3.4.0

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.

Thanks,
Xinrong Meng

Re: [VOTE] Release Apache Spark 3.4.0 (RC6)

[Dongjoon Hyun]

Thank you!

Dongjoon

On Fri, Apr 7, 2023 at 2:16 PM Xinrong Meng 
wrote:

> I am able to proceed with the release now. I'll send an announcement when
> the RC cut is completed.
>
> Xinrong
>
> On Fri, Apr 7, 2023 at 9:54 AM Dongjoon Hyun 
> wrote:
>
>> Got it. Thank you for sharing the current status.
>>
>> Dongjoon.
>>
>> On Fri, Apr 7, 2023 at 9:21 AM Xinrong Meng 
>> wrote:
>>
>>> Hi Dongjoon,
>>>
>>> Yes, it is. To be more specific, we failed to build documentation for
>>> RC7 because of the sbt build outage.
>>>
>>> Xinrong
>>>
>>> On Fri, Apr 7, 2023 at 9:12 AM Dongjoon Hyun 
>>> wrote:
>>>
 Hi, Xinrong.

 I saw the RC7 tag. Maybe, RC7 vote is blocked due to the on-going build
 outage?

 Dongjoon.


 On Thu, Apr 6, 2023 at 6:17 PM Xinrong Meng 
 wrote:

> Thank you! Let me recut the RC then.
>
> On Thu, Apr 6, 2023 at 6:14 PM Hyukjin Kwon 
> wrote:
>
>> Merged the fix.
>>
>> On Fri, 7 Apr 2023 at 10:07, Xinrong Meng 
>> wrote:
>>
>>> Thanks @yangjie01. I marked SPARK-39696 as a blocker.
>>>
>>> On Thu, Apr 6, 2023 at 4:35 PM yangjie01 
>>> wrote:
>>>
 -1 for me due to this RC not include the fix of SPARK-39696,
 SPARK-39696 will fix a data race issue in access to
 TaskMetrics.externalAccums when using Scala 2.13.8 and this issue will
 cause high-frequency Executor crash when use Scala 2.13 distribution
 according to the user's description(
 https://github.com/apache/spark/pull/37206#issuecomment-1486861885
 ).



 So I suggest wait for https://github.com/apache/spark/pull/40663
 to merge and solve this issue although SPARK-39696 was not set as a 
 blocker
 when reported.



 Yang Jie



 *发件人**: *Xinrong Meng 
 *日期**: *2023年4月7日 星期五 05:27
 *收件人**: *dev 
 *主题**: *[VOTE] Release Apache Spark 3.4.0 (RC6)



 Please vote on releasing the following candidate(RC6) as Apache
 Spark version 3.4.0.

 The vote is open until 11:59pm Pacific time *April 11th* and
 passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 
 votes.

 [ ] +1 Release this package as Apache Spark 3.4.0
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see
 http://spark.apache.org/
 

 The tag to be voted on is *v3.4.0-rc6* (commit
 28d0723beb3579c17df84bb22c98a487d7a72023):
 https://github.com/apache/spark/tree/v3.4.0-rc6
 

 The release files, including signatures, digests, etc. can be found
 at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-bin/
 

 Signatures used for Spark RCs can be found in this file:
 https://dist.apache.org/repos/dist/dev/spark/KEYS
 

 The staging repository for this release can be found at:

 https://repository.apache.org/content/repositories/orgapachespark-1440
 

 The documentation corresponding to this release can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-docs/
 

 The list of bug fixes going into 3.4.0 can be found at the
 following URL:
 https://issues.apache.org/jira/projects/SPARK/versions/12351465
 

 This release is using the release script of the tag v3.4.0-rc6.





 FAQ

 =
 How can I help test this release?
 =
 If you are a Spark user, you can help us test this release by taking
 an existing Spark workload and running on this release candidate,
 then
 reporting any regressions.

 If you're working in 

Re: [VOTE] Release Apache Spark 3.4.0 (RC6)

[Xinrong Meng]

I am able to proceed with the release now. I'll send an announcement when
the RC cut is completed.

Xinrong

On Fri, Apr 7, 2023 at 9:54 AM Dongjoon Hyun 
wrote:

> Got it. Thank you for sharing the current status.
>
> Dongjoon.
>
> On Fri, Apr 7, 2023 at 9:21 AM Xinrong Meng 
> wrote:
>
>> Hi Dongjoon,
>>
>> Yes, it is. To be more specific, we failed to build documentation for RC7
>> because of the sbt build outage.
>>
>> Xinrong
>>
>> On Fri, Apr 7, 2023 at 9:12 AM Dongjoon Hyun 
>> wrote:
>>
>>> Hi, Xinrong.
>>>
>>> I saw the RC7 tag. Maybe, RC7 vote is blocked due to the on-going build
>>> outage?
>>>
>>> Dongjoon.
>>>
>>>
>>> On Thu, Apr 6, 2023 at 6:17 PM Xinrong Meng 
>>> wrote:
>>>
 Thank you! Let me recut the RC then.

 On Thu, Apr 6, 2023 at 6:14 PM Hyukjin Kwon 
 wrote:

> Merged the fix.
>
> On Fri, 7 Apr 2023 at 10:07, Xinrong Meng 
> wrote:
>
>> Thanks @yangjie01. I marked SPARK-39696 as a blocker.
>>
>> On Thu, Apr 6, 2023 at 4:35 PM yangjie01  wrote:
>>
>>> -1 for me due to this RC not include the fix of SPARK-39696,
>>> SPARK-39696 will fix a data race issue in access to
>>> TaskMetrics.externalAccums when using Scala 2.13.8 and this issue will
>>> cause high-frequency Executor crash when use Scala 2.13 distribution
>>> according to the user's description(
>>> https://github.com/apache/spark/pull/37206#issuecomment-1486861885).
>>>
>>>
>>>
>>> So I suggest wait for https://github.com/apache/spark/pull/40663 to
>>> merge and solve this issue although SPARK-39696 was not set as a blocker
>>> when reported.
>>>
>>>
>>>
>>> Yang Jie
>>>
>>>
>>>
>>> *发件人**: *Xinrong Meng 
>>> *日期**: *2023年4月7日 星期五 05:27
>>> *收件人**: *dev 
>>> *主题**: *[VOTE] Release Apache Spark 3.4.0 (RC6)
>>>
>>>
>>>
>>> Please vote on releasing the following candidate(RC6) as Apache
>>> Spark version 3.4.0.
>>>
>>> The vote is open until 11:59pm Pacific time *April 11th* and passes
>>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 3.4.0
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see
>>> http://spark.apache.org/
>>> 
>>>
>>> The tag to be voted on is *v3.4.0-rc6* (commit
>>> 28d0723beb3579c17df84bb22c98a487d7a72023):
>>> https://github.com/apache/spark/tree/v3.4.0-rc6
>>> 
>>>
>>> The release files, including signatures, digests, etc. can be found
>>> at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-bin/
>>> 
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> 
>>>
>>> The staging repository for this release can be found at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachespark-1440
>>> 
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-docs/
>>> 
>>>
>>> The list of bug fixes going into 3.4.0 can be found at the following
>>> URL:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>>> 
>>>
>>> This release is using the release script of the tag v3.4.0-rc6.
>>>
>>>
>>>
>>>
>>>
>>> FAQ
>>>
>>> =
>>> How can I help test this release?
>>> =
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate,
>>> then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the
>>> Java/Scala
>>> you can add the staging repository to your projects resolvers and
>>> test
>>> with the RC (make sure to 

Re: Slack for Spark Community: Merging various threads

[Bjørn Jørgensen]

Yes, I have done some search for slack alternatives

I feel that we should do some search, to find if there can be a
better solution than slack.
For what I have found, there are two that can be an alternative for slack.

Rocket.Chat  

and

Zulip Chat 
Zulip Cloud Standard is free for open-source projects

Witch means we get

   - Unlimited search history
   - File storage up to 10 GB per user
   - Message retention policies
   
   - Brand Zulip with your logo
   - Priority commercial support
   - Funds the Zulip open source project


Rust is using zulip  

We can import chats from slack

We can use zulip for events   With multi-use
invite links , there’s no need to
create individual Zulip invitations.  This means that PMC doesn't have to
send a link to every user.
CODE BLOCKS

Discuss code with ease using Markdown code blocks, syntax
highlighting, and code
playgrounds .






fre. 7. apr. 2023 kl. 18:54 skrev Holden Karau :

> I think there was some concern around how to make any sync channel show up
> in logs / index / search results?
>
> On Fri, Apr 7, 2023 at 9:41 AM Dongjoon Hyun 
> wrote:
>
>> Thank you, All.
>>
>> I'm very satisfied with the focused and right questions for the real
>> issues by removing irrelevant claims. :)
>>
>> Let me collect your relevant comments simply.
>>
>>
>> # Category 1: Invitation Hurdle
>>
>> > The key question here is that do PMC members have the bandwidth of
>> inviting everyone in user@ and dev@?
>>
>> > Extending this to inviting everyone on @user (over >4k  subscribers
>> according to the previous thread) might be a stretch,
>>
>> > we should have an official project Slack with an easy invitation
>> process.
>>
>>
>> # Category 2: Controllability
>>
>> > Additionally. there is no indication that the-asf.slack.com is
>> intended for general support.
>>
>> > I would also lean towards a standalone workspace, where we have more
>> control over organizing the channels,
>>
>>
>> # Category 3: Policy Suggestion
>>
>> > *Developer* discussions should still happen on email, JIRA and GitHub
>> and be async-friendly (72-hour rule) to fit the ASF’s development model.
>>
>>
>> Are there any other questions?
>>
>>
>> Dongjoon.
>>
>>
>> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>


-- 
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297

Re: [VOTE] Release Apache Spark 3.4.0 (RC6)

[Dongjoon Hyun]

Got it. Thank you for sharing the current status.

Dongjoon.

On Fri, Apr 7, 2023 at 9:21 AM Xinrong Meng 
wrote:

> Hi Dongjoon,
>
> Yes, it is. To be more specific, we failed to build documentation for RC7
> because of the sbt build outage.
>
> Xinrong
>
> On Fri, Apr 7, 2023 at 9:12 AM Dongjoon Hyun 
> wrote:
>
>> Hi, Xinrong.
>>
>> I saw the RC7 tag. Maybe, RC7 vote is blocked due to the on-going build
>> outage?
>>
>> Dongjoon.
>>
>>
>> On Thu, Apr 6, 2023 at 6:17 PM Xinrong Meng 
>> wrote:
>>
>>> Thank you! Let me recut the RC then.
>>>
>>> On Thu, Apr 6, 2023 at 6:14 PM Hyukjin Kwon  wrote:
>>>
 Merged the fix.

 On Fri, 7 Apr 2023 at 10:07, Xinrong Meng 
 wrote:

> Thanks @yangjie01. I marked SPARK-39696 as a blocker.
>
> On Thu, Apr 6, 2023 at 4:35 PM yangjie01  wrote:
>
>> -1 for me due to this RC not include the fix of SPARK-39696,
>> SPARK-39696 will fix a data race issue in access to
>> TaskMetrics.externalAccums when using Scala 2.13.8 and this issue will
>> cause high-frequency Executor crash when use Scala 2.13 distribution
>> according to the user's description(
>> https://github.com/apache/spark/pull/37206#issuecomment-1486861885).
>>
>>
>>
>> So I suggest wait for https://github.com/apache/spark/pull/40663 to
>> merge and solve this issue although SPARK-39696 was not set as a blocker
>> when reported.
>>
>>
>>
>> Yang Jie
>>
>>
>>
>> *发件人**: *Xinrong Meng 
>> *日期**: *2023年4月7日 星期五 05:27
>> *收件人**: *dev 
>> *主题**: *[VOTE] Release Apache Spark 3.4.0 (RC6)
>>
>>
>>
>> Please vote on releasing the following candidate(RC6) as Apache Spark
>> version 3.4.0.
>>
>> The vote is open until 11:59pm Pacific time *April 11th* and passes
>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.4.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>> 
>>
>> The tag to be voted on is *v3.4.0-rc6* (commit
>> 28d0723beb3579c17df84bb22c98a487d7a72023):
>> https://github.com/apache/spark/tree/v3.4.0-rc6
>> 
>>
>> The release files, including signatures, digests, etc. can be found
>> at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-bin/
>> 
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>> 
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1440
>> 
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-docs/
>> 
>>
>> The list of bug fixes going into 3.4.0 can be found at the following
>> URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>> 
>>
>> This release is using the release script of the tag v3.4.0-rc6.
>>
>>
>>
>>
>>
>> FAQ
>>
>> =
>> How can I help test this release?
>> =
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with an out of date RC going forward).
>>
>> ===
>> What should happen to JIRA tickets still targeting 3.4.0?
>> ===
>> The current list of open tickets 

Re: Slack for Spark Community: Merging various threads

[Holden Karau]

I think there was some concern around how to make any sync channel show up
in logs / index / search results?

On Fri, Apr 7, 2023 at 9:41 AM Dongjoon Hyun 
wrote:

> Thank you, All.
>
> I'm very satisfied with the focused and right questions for the real
> issues by removing irrelevant claims. :)
>
> Let me collect your relevant comments simply.
>
>
> # Category 1: Invitation Hurdle
>
> > The key question here is that do PMC members have the bandwidth of
> inviting everyone in user@ and dev@?
>
> > Extending this to inviting everyone on @user (over >4k  subscribers
> according to the previous thread) might be a stretch,
>
> > we should have an official project Slack with an easy invitation process.
>
>
> # Category 2: Controllability
>
> > Additionally. there is no indication that the-asf.slack.com is intended
> for general support.
>
> > I would also lean towards a standalone workspace, where we have more
> control over organizing the channels,
>
>
> # Category 3: Policy Suggestion
>
> > *Developer* discussions should still happen on email, JIRA and GitHub
> and be async-friendly (72-hour rule) to fit the ASF’s development model.
>
>
> Are there any other questions?
>
>
> Dongjoon.
>
>
> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Slack for Spark Community: Merging various threads

[Dongjoon Hyun]

Thank you, All.

I'm very satisfied with the focused and right questions for the real issues
by removing irrelevant claims. :)

Let me collect your relevant comments simply.


# Category 1: Invitation Hurdle

> The key question here is that do PMC members have the bandwidth of
inviting everyone in user@ and dev@?

> Extending this to inviting everyone on @user (over >4k  subscribers
according to the previous thread) might be a stretch,

> we should have an official project Slack with an easy invitation process.


# Category 2: Controllability

> Additionally. there is no indication that the-asf.slack.com is intended
for general support.

> I would also lean towards a standalone workspace, where we have more
control over organizing the channels,


# Category 3: Policy Suggestion

> *Developer* discussions should still happen on email, JIRA and GitHub and
be async-friendly (72-hour rule) to fit the ASF’s development model.


Are there any other questions?


Dongjoon.

Re: [VOTE] Release Apache Spark 3.4.0 (RC6)

[Xinrong Meng]

Hi Dongjoon,

Yes, it is. To be more specific, we failed to build documentation for RC7
because of the sbt build outage.

Xinrong

On Fri, Apr 7, 2023 at 9:12 AM Dongjoon Hyun 
wrote:

> Hi, Xinrong.
>
> I saw the RC7 tag. Maybe, RC7 vote is blocked due to the on-going build
> outage?
>
> Dongjoon.
>
>
> On Thu, Apr 6, 2023 at 6:17 PM Xinrong Meng 
> wrote:
>
>> Thank you! Let me recut the RC then.
>>
>> On Thu, Apr 6, 2023 at 6:14 PM Hyukjin Kwon  wrote:
>>
>>> Merged the fix.
>>>
>>> On Fri, 7 Apr 2023 at 10:07, Xinrong Meng 
>>> wrote:
>>>
 Thanks @yangjie01. I marked SPARK-39696 as a blocker.

 On Thu, Apr 6, 2023 at 4:35 PM yangjie01  wrote:

> -1 for me due to this RC not include the fix of SPARK-39696,
> SPARK-39696 will fix a data race issue in access to
> TaskMetrics.externalAccums when using Scala 2.13.8 and this issue will
> cause high-frequency Executor crash when use Scala 2.13 distribution
> according to the user's description(
> https://github.com/apache/spark/pull/37206#issuecomment-1486861885).
>
>
>
> So I suggest wait for https://github.com/apache/spark/pull/40663 to
> merge and solve this issue although SPARK-39696 was not set as a blocker
> when reported.
>
>
>
> Yang Jie
>
>
>
> *发件人**: *Xinrong Meng 
> *日期**: *2023年4月7日 星期五 05:27
> *收件人**: *dev 
> *主题**: *[VOTE] Release Apache Spark 3.4.0 (RC6)
>
>
>
> Please vote on releasing the following candidate(RC6) as Apache Spark
> version 3.4.0.
>
> The vote is open until 11:59pm Pacific time *April 11th* and passes
> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
> 
>
> The tag to be voted on is *v3.4.0-rc6* (commit
> 28d0723beb3579c17df84bb22c98a487d7a72023):
> https://github.com/apache/spark/tree/v3.4.0-rc6
> 
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-bin/
> 
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
> 
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1440
> 
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-docs/
> 
>
> The list of bug fixes going into 3.4.0 can be found at the following
> URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
> 
>
> This release is using the release script of the tag v3.4.0-rc6.
>
>
>
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK
> 
>  and
> search for "Target 

Re: [VOTE] Release Apache Spark 3.4.0 (RC6)

[Dongjoon Hyun]

Hi, Xinrong.

I saw the RC7 tag. Maybe, RC7 vote is blocked due to the on-going build
outage?

Dongjoon.


On Thu, Apr 6, 2023 at 6:17 PM Xinrong Meng 
wrote:

> Thank you! Let me recut the RC then.
>
> On Thu, Apr 6, 2023 at 6:14 PM Hyukjin Kwon  wrote:
>
>> Merged the fix.
>>
>> On Fri, 7 Apr 2023 at 10:07, Xinrong Meng 
>> wrote:
>>
>>> Thanks @yangjie01. I marked SPARK-39696 as a blocker.
>>>
>>> On Thu, Apr 6, 2023 at 4:35 PM yangjie01  wrote:
>>>
 -1 for me due to this RC not include the fix of SPARK-39696,
 SPARK-39696 will fix a data race issue in access to
 TaskMetrics.externalAccums when using Scala 2.13.8 and this issue will
 cause high-frequency Executor crash when use Scala 2.13 distribution
 according to the user's description(
 https://github.com/apache/spark/pull/37206#issuecomment-1486861885).



 So I suggest wait for https://github.com/apache/spark/pull/40663 to
 merge and solve this issue although SPARK-39696 was not set as a blocker
 when reported.



 Yang Jie



 *发件人**: *Xinrong Meng 
 *日期**: *2023年4月7日 星期五 05:27
 *收件人**: *dev 
 *主题**: *[VOTE] Release Apache Spark 3.4.0 (RC6)



 Please vote on releasing the following candidate(RC6) as Apache Spark
 version 3.4.0.

 The vote is open until 11:59pm Pacific time *April 11th* and passes if
 a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.

 [ ] +1 Release this package as Apache Spark 3.4.0
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see http://spark.apache.org/
 

 The tag to be voted on is *v3.4.0-rc6* (commit
 28d0723beb3579c17df84bb22c98a487d7a72023):
 https://github.com/apache/spark/tree/v3.4.0-rc6
 

 The release files, including signatures, digests, etc. can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-bin/
 

 Signatures used for Spark RCs can be found in this file:
 https://dist.apache.org/repos/dist/dev/spark/KEYS
 

 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-1440
 

 The documentation corresponding to this release can be found at:
 https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc6-docs/
 

 The list of bug fixes going into 3.4.0 can be found at the following
 URL:
 https://issues.apache.org/jira/projects/SPARK/versions/12351465
 

 This release is using the release script of the tag v3.4.0-rc6.





 FAQ

 =
 How can I help test this release?
 =
 If you are a Spark user, you can help us test this release by taking
 an existing Spark workload and running on this release candidate, then
 reporting any regressions.

 If you're working in PySpark you can set up a virtual env and install
 the current RC and see if anything important breaks, in the Java/Scala
 you can add the staging repository to your projects resolvers and test
 with the RC (make sure to clean up the artifact cache before/after so
 you don't end up building with an out of date RC going forward).

 ===
 What should happen to JIRA tickets still targeting 3.4.0?
 ===
 The current list of open tickets targeted at 3.4.0 can be found at:
 https://issues.apache.org/jira/projects/SPARK
 
  and
 search for "Target Version/s" = 3.4.0

 Committers should look at those and triage. Extremely important bug
 fixes, documentation, and API tweaks that impact compatibility should
 be worked on immediately. Everything else please retarget to an
 appropriate release.

 ==
 But my bug isn't 

Re: sbt build is broken because repo is not available

[Dongjoon Hyun]

Thank you for the pointer, Yuming.

Dongjoon.

On Fri, Apr 7, 2023 at 12:18 AM Yuming Wang  wrote:

> Hi all,
>
> sbt build is broken because repo is not available. Please see:
> https://github.com/sbt/sbt/issues/7202.
>
>

[SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI

[Chenghao Lyu]

Hi,

The detailed stage page shows the involved WholeStageCodegen Ids in its DAG 
visualization from the Spark UI when running a SparkSQL. (e.g., under the link 
node:18088/history/application_1663600377480_62091/stages/stage/?id=1=0).

However, I have trouble extracting the WholeStageCodegen ids from the DAG 
visualization via the RESTAPIs. Is there any other way to get the 
WholeStageCodegen Ids information for each stage automatically?

Cheers,
Chenghao

sbt build is broken because repo is not available

[Yuming Wang]

Hi all,

sbt build is broken because repo is not available. Please see:
https://github.com/sbt/sbt/issues/7202.