Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

[Mridul Muralidharan]

+1

Signatures, digests, etc check out fine.
Checked out tag and build/tested with -Phive -Pyarn -Pmesos -Pkubernetes

Regards,
Mridul


On Sat, Apr 8, 2023 at 12:13 PM L. C. Hsieh  wrote:

> +1
>
> Thanks Xinrong.
>
> On Sat, Apr 8, 2023 at 8:23 AM yangjie01  wrote:
> >
> > +1
> >
> >
> >
> > 发件人: Sean Owen 
> > 日期: 2023年4月8日 星期六 20:27
> > 收件人: Xinrong Meng 
> > 抄送: dev 
> > 主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC7)
> >
> >
> >
> > +1 form me, same result as last time.
> >
> >
> >
> > On Fri, Apr 7, 2023 at 6:30 PM Xinrong Meng 
> wrote:
> >
> > Please vote on releasing the following candidate(RC7) as Apache Spark
> version 3.4.0.
> >
> > The vote is open until 11:59pm Pacific time April 12th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >
> > [ ] +1 Release this package as Apache Spark 3.4.0
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see http://spark.apache.org/
> >
> > The tag to be voted on is v3.4.0-rc7 (commit
> 87a5442f7ed96b11051d8a9333476d080054e5a0):
> > https://github.com/apache/spark/tree/v3.4.0-rc7
> >
> > The release files, including signatures, digests, etc. can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-bin/
> >
> > Signatures used for Spark RCs can be found in this file:
> > https://dist.apache.org/repos/dist/dev/spark/KEYS
> >
> > The staging repository for this release can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-1441
> >
> > The documentation corresponding to this release can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-docs/
> >
> > The list of bug fixes going into 3.4.0 can be found at the following URL:
> > https://issues.apache.org/jira/projects/SPARK/versions/12351465
> >
> > This release is using the release script of the tag v3.4.0-rc7.
> >
> >
> > FAQ
> >
> > =
> > How can I help test this release?
> > =
> > If you are a Spark user, you can help us test this release by taking
> > an existing Spark workload and running on this release candidate, then
> > reporting any regressions.
> >
> > If you're working in PySpark you can set up a virtual env and install
> > the current RC and see if anything important breaks, in the Java/Scala
> > you can add the staging repository to your projects resolvers and test
> > with the RC (make sure to clean up the artifact cache before/after so
> > you don't end up building with an out of date RC going forward).
> >
> > ===
> > What should happen to JIRA tickets still targeting 3.4.0?
> > ===
> > The current list of open tickets targeted at 3.4.0 can be found at:
> > https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
> >
> > Committers should look at those and triage. Extremely important bug
> > fixes, documentation, and API tweaks that impact compatibility should
> > be worked on immediately. Everything else please retarget to an
> > appropriate release.
> >
> > ==
> > But my bug isn't fixed?
> > ==
> > In order to make timely releases, we will typically not hold the
> > release unless the bug in question is a regression from the previous
> > release. That being said, if there is something which is a regression
> > that has not been correctly targeted please ping me or a committer to
> > help target the issue.
> >
> > Thanks,
> > Xinrong Meng
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

[L. C. Hsieh]

+1

Thanks Xinrong.

On Sat, Apr 8, 2023 at 8:23 AM yangjie01  wrote:
>
> +1
>
>
>
> 发件人: Sean Owen 
> 日期: 2023年4月8日 星期六 20:27
> 收件人: Xinrong Meng 
> 抄送: dev 
> 主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC7)
>
>
>
> +1 form me, same result as last time.
>
>
>
> On Fri, Apr 7, 2023 at 6:30 PM Xinrong Meng  wrote:
>
> Please vote on releasing the following candidate(RC7) as Apache Spark version 
> 3.4.0.
>
> The vote is open until 11:59pm Pacific time April 12th and passes if a 
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is v3.4.0-rc7 (commit 
> 87a5442f7ed96b11051d8a9333476d080054e5a0):
> https://github.com/apache/spark/tree/v3.4.0-rc7
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1441
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-docs/
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>
> This release is using the release script of the tag v3.4.0-rc7.
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target 
> Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Thanks,
> Xinrong Meng

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

[yangjie01]

+1

发件人: Sean Owen 
日期: 2023年4月8日 星期六 20:27
收件人: Xinrong Meng 
抄送: dev 
主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

+1 form me, same result as last time.

On Fri, Apr 7, 2023 at 6:30 PM Xinrong Meng 
mailto:xinrong.apa...@gmail.com>> wrote:
Please vote on releasing the following candidate(RC7) as Apache Spark version 
3.4.0.

The vote is open until 11:59pm Pacific time April 12th and passes if a majority 
+1 PMC votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 3.4.0
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see 
http://spark.apache.org/

The tag to be voted on is v3.4.0-rc7 (commit 
87a5442f7ed96b11051d8a9333476d080054e5a0):
https://github.com/apache/spark/tree/v3.4.0-rc7

The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1441

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-docs/

The list of bug fixes going into 3.4.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12351465

This release is using the release script of the tag v3.4.0-rc7.


FAQ

=
How can I help test this release?
=
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.

If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).

===
What should happen to JIRA tickets still targeting 3.4.0?
===
The current list of open tickets targeted at 3.4.0 can be found at:
https://issues.apache.org/jira/projects/SPARK
 and search for "Target Version/s" = 3.4.0

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.

Thanks,
Xinrong Meng

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

[Sean Owen]

+1 form me, same result as last time.

On Fri, Apr 7, 2023 at 6:30 PM Xinrong Meng 
wrote:

> Please vote on releasing the following candidate(RC7) as Apache Spark
> version 3.4.0.
>
> The vote is open until 11:59pm Pacific time *April 12th* and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is v3.4.0-rc7 (commit
> 87a5442f7ed96b11051d8a9333476d080054e5a0):
> https://github.com/apache/spark/tree/v3.4.0-rc7
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1441
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc7-docs/
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>
> This release is using the release script of the tag v3.4.0-rc7.
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Thanks,
> Xinrong Meng
>

Re: Slack for Spark Community: Merging various threads

[Maciej]

@Bjørn Matrix (represented by element in the linked summary. Also, since 
the last year, Rocket Chat uses Matrix under the hood) is already used 
for user support and related discussions for a number of large projects, 
since gitter migrated there. And it is not like we need Slack or its 
replacement in the first place. Some of the Slack features are useful 
for us, but its not exactly the best tool for user support IMHO.


@Dongjoon There are probably two more things we should discuss:

 * What are data privacy obligations while keeping a communication
   channel, advertised as official, outside the ASF?  Does it put it
   out of scope for the ASF legal and data privacy teams?

   If I recall correctly, Slack requires at least some of the
   formalities to be handled by the primary owner and as far as I am
   aware the project is not a legal person. Not sure how linen.dev or
   another retention tool fits into all of that, but it's unrealistic
   to expect it makes things easier.

   This might sound hypothetical, but we've already seen users leaking
   sensitive information on the mail list and requesting erasure
   (which, luckily for us, is not technically possible).

 * How are we going to handle moderation, if we assume number of users
   comparable to Delta Lake Slack and open registration? At minimum we
   have to ensure that the ASF Code of Conduct is respected. An
   official channel or not, failure to do that reflects badly on the
   project, the ASF and all of us.

--
Maciej



On 4/7/23 21:02, Bjørn Jørgensen wrote:
Yes, I have done some search for slack alternatives 

I feel that we should do some search, to find if there can be a 
better solution than slack.
For what I have found, there are two that can be an alternative for 
slack.


Rocket.Chat 

and

Zulip Chat 
Zulip Cloud Standard is free for open-source projects 


Witch means we get

  * Unlimited search history
  * File storage up to 10 GB per user
  * Message retention policies

  * Brand Zulip with your logo
  * Priority commercial support
  * Funds the Zulip open source project


Rust is using zulip 

We can import chats from slack 

We can use zulip for events  With 
multi-use invite links , 
there’s no need to create individual Zulip invitations.  This means 
that PMC doesn't have to send a link to every user.



  CODE BLOCKS

Discuss code with ease using Markdown code blocks, syntax 
highlighting, and code playgrounds 
.







fre. 7. apr. 2023 kl. 18:54 skrev Holden Karau :

I think there was some concern around how to make any sync channel
show up in logs / index / search results?

On Fri, Apr 7, 2023 at 9:41 AM Dongjoon Hyun
 wrote:

Thank you, All.

I'm very satisfied with the focused and right questions for
the real issues by removing irrelevant claims. :)

Let me collect your relevant comments simply.


# Category 1: Invitation Hurdle

> The key question here is that do PMC members have the
bandwidth of inviting everyone in user@ and dev@?

> Extending this to inviting everyone on @user (over >4k 
subscribers according to the previous thread) might be a stretch,

> we should have an official project Slack with an easy
invitation process.


# Category 2: Controllability

> Additionally. there is no indication thatthe-asf.slack.com
is intended for general support.

> I would also lean towards a standalone workspace, where we
have more control over organizing the channels,


# Category 3: Policy Suggestion

> *Developer* discussions should still happen on email, JIRA
and GitHub and be async-friendly (72-hour rule) to fit the
ASF’s development model.


Are there any other questions?


Dongjoon.


-- 
Twitter: https://twitter.com/holdenkarau

Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 
YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297





OpenPGP_signature
Description: OpenPGP digital signature