Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[Wenchen Fan]

Sorry for the last-minute change, but we found two wrong behaviors and want
to fix them before the release:

https://github.com/apache/spark/pull/40641
We missed a corner case when the input index for `array_insert` is 0. It
should fail as 0 is an invalid index.

https://github.com/apache/spark/pull/40623
We found some usability issues with a new API and need to change the API to
fix it. If people have concerns we can also remove the new API entirely.

Thus I'm -1 to this RC. I'll merge these 2 PRs today if no objections.

Thanks,
Wenchen

On Tue, Apr 4, 2023 at 3:47 AM L. C. Hsieh  wrote:

> +1
>
> Thanks Xinrong.
>
> On Mon, Apr 3, 2023 at 12:35 PM Dongjoon Hyun 
> wrote:
> >
> > +1
> >
> > I also verified that RC5 has SBOM artifacts.
> >
> >
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.12/3.4.0/spark-core_2.12-3.4.0-cyclonedx.json
> >
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.13/3.4.0/spark-core_2.13-3.4.0-cyclonedx.json
> >
> > Thanks,
> > Dongjoon.
> >
> >
> >
> > On Mon, Apr 3, 2023 at 1:57 AM yangjie01  wrote:
> >>
> >> +1, checked Java 17 + Scala 2.13 + Python 3.10.10.
> >>
> >>
> >>
> >> 发件人: Herman van Hovell 
> >> 日期: 2023年3月31日 星期五 12:12
> >> 收件人: Sean Owen 
> >> 抄送: Xinrong Meng , dev 
> >> 主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC5)
> >>
> >>
> >>
> >> +1
> >>
> >>
> >>
> >> On Thu, Mar 30, 2023 at 11:05 PM Sean Owen  wrote:
> >>
> >> +1 same result from me as last time.
> >>
> >>
> >>
> >> On Thu, Mar 30, 2023 at 3:21 AM Xinrong Meng 
> wrote:
> >>
> >> Please vote on releasing the following candidate(RC5) as Apache Spark
> version 3.4.0.
> >>
> >> The vote is open until 11:59pm Pacific time April 4th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >>
> >> [ ] +1 Release this package as Apache Spark 3.4.0
> >> [ ] -1 Do not release this package because ...
> >>
> >> To learn more about Apache Spark, please see http://spark.apache.org/
> >>
> >> The tag to be voted on is v3.4.0-rc5 (commit
> f39ad617d32a671e120464e4a75986241d72c487):
> >> https://github.com/apache/spark/tree/v3.4.0-rc5
> >>
> >> The release files, including signatures, digests, etc. can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/
> >>
> >> Signatures used for Spark RCs can be found in this file:
> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>
> >> The staging repository for this release can be found at:
> >> https://repository.apache.org/content/repositories/orgapachespark-1439
> >>
> >> The documentation corresponding to this release can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/
> >>
> >> The list of bug fixes going into 3.4.0 can be found at the following
> URL:
> >> https://issues.apache.org/jira/projects/SPARK/versions/12351465
> >>
> >> This release is using the release script of the tag v3.4.0-rc5.
> >>
> >>
> >>
> >>
> >>
> >> FAQ
> >>
> >> =
> >> How can I help test this release?
> >> =
> >> If you are a Spark user, you can help us test this release by taking
> >> an existing Spark workload and running on this release candidate, then
> >> reporting any regressions.
> >>
> >> If you're working in PySpark you can set up a virtual env and install
> >> the current RC and see if anything important breaks, in the Java/Scala
> >> you can add the staging repository to your projects resolvers and test
> >> with the RC (make sure to clean up the artifact cache before/after so
> >> you don't end up building with an out of date RC going forward).
> >>
> >> ===
> >> What should happen to JIRA tickets still targeting 3.4.0?
> >> ===
> >> The current list of open tickets targeted at 3.4.0 can be found at:
> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
> >>
> >> Committers should look at those and triage. Extremely important bug
> >> fixes, documentation, and API tweaks that impact compatibility should
> >> be worked on immediately. Everything else please retarget to an
> >> appropriate release.
> >>
> >> ==
> >> But my bug isn't fixed?
> >> ==
> >> In order to make timely releases, we will typically not hold the
> >> release unless the bug in question is a regression from the previous
> >> release. That being said, if there is something which is a regression
> >> that has not been correctly targeted please ping me or a committer to
> >> help target the issue.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Xinrong Meng
> >>
> >>
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[Yuming Wang]

+1

On Tue, Apr 4, 2023 at 3:46 AM L. C. Hsieh  wrote:

> +1
>
> Thanks Xinrong.
>
> On Mon, Apr 3, 2023 at 12:35 PM Dongjoon Hyun 
> wrote:
> >
> > +1
> >
> > I also verified that RC5 has SBOM artifacts.
> >
> >
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.12/3.4.0/spark-core_2.12-3.4.0-cyclonedx.json
> >
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.13/3.4.0/spark-core_2.13-3.4.0-cyclonedx.json
> >
> > Thanks,
> > Dongjoon.
> >
> >
> >
> > On Mon, Apr 3, 2023 at 1:57 AM yangjie01  wrote:
> >>
> >> +1, checked Java 17 + Scala 2.13 + Python 3.10.10.
> >>
> >>
> >>
> >> 发件人: Herman van Hovell 
> >> 日期: 2023年3月31日 星期五 12:12
> >> 收件人: Sean Owen 
> >> 抄送: Xinrong Meng , dev 
> >> 主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC5)
> >>
> >>
> >>
> >> +1
> >>
> >>
> >>
> >> On Thu, Mar 30, 2023 at 11:05 PM Sean Owen  wrote:
> >>
> >> +1 same result from me as last time.
> >>
> >>
> >>
> >> On Thu, Mar 30, 2023 at 3:21 AM Xinrong Meng 
> wrote:
> >>
> >> Please vote on releasing the following candidate(RC5) as Apache Spark
> version 3.4.0.
> >>
> >> The vote is open until 11:59pm Pacific time April 4th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >>
> >> [ ] +1 Release this package as Apache Spark 3.4.0
> >> [ ] -1 Do not release this package because ...
> >>
> >> To learn more about Apache Spark, please see http://spark.apache.org/
> >>
> >> The tag to be voted on is v3.4.0-rc5 (commit
> f39ad617d32a671e120464e4a75986241d72c487):
> >> https://github.com/apache/spark/tree/v3.4.0-rc5
> >>
> >> The release files, including signatures, digests, etc. can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/
> >>
> >> Signatures used for Spark RCs can be found in this file:
> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>
> >> The staging repository for this release can be found at:
> >> https://repository.apache.org/content/repositories/orgapachespark-1439
> >>
> >> The documentation corresponding to this release can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/
> >>
> >> The list of bug fixes going into 3.4.0 can be found at the following
> URL:
> >> https://issues.apache.org/jira/projects/SPARK/versions/12351465
> >>
> >> This release is using the release script of the tag v3.4.0-rc5.
> >>
> >>
> >>
> >>
> >>
> >> FAQ
> >>
> >> =
> >> How can I help test this release?
> >> =
> >> If you are a Spark user, you can help us test this release by taking
> >> an existing Spark workload and running on this release candidate, then
> >> reporting any regressions.
> >>
> >> If you're working in PySpark you can set up a virtual env and install
> >> the current RC and see if anything important breaks, in the Java/Scala
> >> you can add the staging repository to your projects resolvers and test
> >> with the RC (make sure to clean up the artifact cache before/after so
> >> you don't end up building with an out of date RC going forward).
> >>
> >> ===
> >> What should happen to JIRA tickets still targeting 3.4.0?
> >> ===
> >> The current list of open tickets targeted at 3.4.0 can be found at:
> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
> >>
> >> Committers should look at those and triage. Extremely important bug
> >> fixes, documentation, and API tweaks that impact compatibility should
> >> be worked on immediately. Everything else please retarget to an
> >> appropriate release.
> >>
> >> ==
> >> But my bug isn't fixed?
> >> ==
> >> In order to make timely releases, we will typically not hold the
> >> release unless the bug in question is a regression from the previous
> >> release. That being said, if there is something which is a regression
> >> that has not been correctly targeted please ping me or a committer to
> >> help target the issue.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Xinrong Meng
> >>
> >>
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Slack for PySpark users

[Dongjoon Hyun]

Thank you, Denny.

May I interpret your comment as a request to support multiple channels in
ASF too?

> because it would allow us to create multiple channels for different topics


Any other reasons?

Dongjoon.


On Mon, Apr 3, 2023 at 5:31 PM Denny Lee  wrote:

> I do think creating a new Slack channel would be helpful because it would
> allow us to create multiple channels for different topics - streaming,
> graph, ML, etc.
>
> We would need a volunteer core to maintain it so we can keep the spirit
> and letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
> active.
>
>
>
> On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun 
> wrote:
>
>> Shall we summarize the discussion so far?
>>
>> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
>> initiate this thread instead of "Slack" vs "Mailing list"?
>>
>> If ASF Slack provides what you need, is it better than creating a
>> new Slack channel?
>>
>> Or, is there another reason for us to create a new Slack channel?
>>
>> Dongjoon.
>>
>>
>> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh 
>> wrote:
>>
>>> I agree, whatever individual sentiments are.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim 
>>> wrote:
>>>
 Just to be clear, if there is no strong volunteer to make the new
 community channel stay active, I'd probably be OK to not fork the channel.
 You can see a strong counter example from #spark channel in ASF. It is the
 place where there are only questions and promos but zero answers. I see
 volunteers here demanding for another channel, so I want to see us go with
 the most preferred way for these volunteers.

 User mailing list does not go in a good shape. I hope we give another
 try with recent technology to see whether we can gain traction - if we
 fail, the user mailing list will still be there.

 On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
 kabhwan.opensou...@gmail.com> wrote:

> The number of subscribers doesn't give any meaningful value. Please
> look into the number of mails being sent to the list.
>
> https://lists.apache.org/list.html?u...@spark.apache.org
> The latest month there were more than 200 emails being sent was Feb
> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
> in 2015 and earlier.
> Let's face the fact. User mailing list is dying, even before we start
> discussion about alternative communication methods.
>
> Users never go with the way if it's just because PMC members (or ASF)
> have preference. They are going with the way they are convenient.
>
> Same applies here - if ASF Slack requires a restricted invitation
> mechanism then it won't work. Looks like there is a link for an 
> invitation,
> but we are also talking about the cost as well.
> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
> As long as we are being serious about the cost, I don't think we are
> going to land in the way "users" are convenient.
>
> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative
>> communication mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - u...@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside 
>>> ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative
>>> communication mechanism rather than the user mailing list. Before 

Re: Slack for PySpark users

[Denny Lee]

I do think creating a new Slack channel would be helpful because it would
allow us to create multiple channels for different topics - streaming,
graph, ML, etc.

We would need a volunteer core to maintain it so we can keep the spirit and
letter of ASF / code of conduct.  I’d be glad to volunteer to keep this
active.



On Mon, Apr 3, 2023 at 16:46 Dongjoon Hyun  wrote:

> Shall we summarize the discussion so far?
>
> To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
> initiate this thread instead of "Slack" vs "Mailing list"?
>
> If ASF Slack provides what you need, is it better than creating a
> new Slack channel?
>
> Or, is there another reason for us to create a new Slack channel?
>
> Dongjoon.
>
>
> On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh 
> wrote:
>
>> I agree, whatever individual sentiments are.
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim 
>> wrote:
>>
>>> Just to be clear, if there is no strong volunteer to make the new
>>> community channel stay active, I'd probably be OK to not fork the channel.
>>> You can see a strong counter example from #spark channel in ASF. It is the
>>> place where there are only questions and promos but zero answers. I see
>>> volunteers here demanding for another channel, so I want to see us go with
>>> the most preferred way for these volunteers.
>>>
>>> User mailing list does not go in a good shape. I hope we give another
>>> try with recent technology to see whether we can gain traction - if we
>>> fail, the user mailing list will still be there.
>>>
>>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim <
>>> kabhwan.opensou...@gmail.com> wrote:
>>>
 The number of subscribers doesn't give any meaningful value. Please
 look into the number of mails being sent to the list.

 https://lists.apache.org/list.html?u...@spark.apache.org
 The latest month there were more than 200 emails being sent was Feb
 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
 in 2015 and earlier.
 Let's face the fact. User mailing list is dying, even before we start
 discussion about alternative communication methods.

 Users never go with the way if it's just because PMC members (or ASF)
 have preference. They are going with the way they are convenient.

 Same applies here - if ASF Slack requires a restricted invitation
 mechanism then it won't work. Looks like there is a link for an invitation,
 but we are also talking about the cost as well.
 https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
 As long as we are being serious about the cost, I don't think we are
 going to land in the way "users" are convenient.

 On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
 wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
> channel.
>
> > It's unavoidable if "users" prefer to use an alternative
> communication mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - u...@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
> kabhwan.opensou...@gmail.com> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative
>> communication mechanism rather than the user mailing list. Before Stack
>> Overflow days, there had been a meaningful number of questions around 
>> user@.
>> It's just impossible to let them go back and post to the user mailing 
>> list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of 

Re: Slack for PySpark users

[Dongjoon Hyun]

Shall we summarize the discussion so far?

To sum up, "ASF Slack" vs "3rd-party Slack" was the real background to
initiate this thread instead of "Slack" vs "Mailing list"?

If ASF Slack provides what you need, is it better than creating a new Slack
channel?

Or, is there another reason for us to create a new Slack channel?

Dongjoon.


On Mon, Apr 3, 2023 at 3:27 PM Mich Talebzadeh 
wrote:

> I agree, whatever individual sentiments are.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim 
> wrote:
>
>> Just to be clear, if there is no strong volunteer to make the new
>> community channel stay active, I'd probably be OK to not fork the channel.
>> You can see a strong counter example from #spark channel in ASF. It is the
>> place where there are only questions and promos but zero answers. I see
>> volunteers here demanding for another channel, so I want to see us go with
>> the most preferred way for these volunteers.
>>
>> User mailing list does not go in a good shape. I hope we give another try
>> with recent technology to see whether we can gain traction - if we fail,
>> the user mailing list will still be there.
>>
>> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim 
>> wrote:
>>
>>> The number of subscribers doesn't give any meaningful value. Please look
>>> into the number of mails being sent to the list.
>>>
>>> https://lists.apache.org/list.html?u...@spark.apache.org
>>> The latest month there were more than 200 emails being sent was Feb
>>> 2022, more than a year ago. It was more than 1k in 2016, and more than 2k
>>> in 2015 and earlier.
>>> Let's face the fact. User mailing list is dying, even before we start
>>> discussion about alternative communication methods.
>>>
>>> Users never go with the way if it's just because PMC members (or ASF)
>>> have preference. They are going with the way they are convenient.
>>>
>>> Same applies here - if ASF Slack requires a restricted invitation
>>> mechanism then it won't work. Looks like there is a link for an invitation,
>>> but we are also talking about the cost as well.
>>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>>> As long as we are being serious about the cost, I don't think we are
>>> going to land in the way "users" are convenient.
>>>
>>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
>>> wrote:
>>>
 As Mich Talebzadeh pointed out, Apache Spark has an official Slack
 channel.

 > It's unavoidable if "users" prefer to use an alternative
 communication mechanism rather than the user mailing list.

 The following is the number of people in the official channels.

 - u...@spark.apache.org has 4519 subscribers.
 - dev@spark.apache.org has 3149 subscribers.
 - ASF Official Slack channel has 602 subscribers.

 May I ask if the users prefer to use the ASF Official Slack channel
 than the user mailing list?

 Dongjoon.



 On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
 kabhwan.opensou...@gmail.com> wrote:

> I'm reading through the page "Briefing: The Apache Way", and in the
> section of "Open Communications", restriction of communication inside ASF
> INFRA (mailing list) is more about code and decision-making.
>
> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>
> It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list. Before Stack Overflow days,
> there had been a meaningful number of questions around user@. It's
> just impossible to let them go back and post to the user mailing list.
>
> We just need to make sure it is not the purpose of employing Slack to
> move all discussions about developments, direction of the project, etc
> which must happen in dev@/private@. The purpose of Slack thread here
> does not seem to aim to serve the purpose.
>
>
> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
>> Good discussions and proposals.all around.
>>
>> I have used slack in anger on a customer site before. For small and
>> medium size groups it is good and affordable. Alternatives have been
>> suggested as well so those who like investigative search can agree and 
>> come
>> up with a freebie one.
>> I am inclined to agree with 

Re: Slack for PySpark users

[Mich Talebzadeh]

I agree, whatever individual sentiments are.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 23:21, Jungtaek Lim 
wrote:

> Just to be clear, if there is no strong volunteer to make the new
> community channel stay active, I'd probably be OK to not fork the channel.
> You can see a strong counter example from #spark channel in ASF. It is the
> place where there are only questions and promos but zero answers. I see
> volunteers here demanding for another channel, so I want to see us go with
> the most preferred way for these volunteers.
>
> User mailing list does not go in a good shape. I hope we give another try
> with recent technology to see whether we can gain traction - if we fail,
> the user mailing list will still be there.
>
> On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim 
> wrote:
>
>> The number of subscribers doesn't give any meaningful value. Please look
>> into the number of mails being sent to the list.
>>
>> https://lists.apache.org/list.html?u...@spark.apache.org
>> The latest month there were more than 200 emails being sent was Feb 2022,
>> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
>> and earlier.
>> Let's face the fact. User mailing list is dying, even before we start
>> discussion about alternative communication methods.
>>
>> Users never go with the way if it's just because PMC members (or ASF)
>> have preference. They are going with the way they are convenient.
>>
>> Same applies here - if ASF Slack requires a restricted invitation
>> mechanism then it won't work. Looks like there is a link for an invitation,
>> but we are also talking about the cost as well.
>> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
>> As long as we are being serious about the cost, I don't think we are
>> going to land in the way "users" are convenient.
>>
>> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
>> wrote:
>>
>>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>>> channel.
>>>
>>> > It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list.
>>>
>>> The following is the number of people in the official channels.
>>>
>>> - u...@spark.apache.org has 4519 subscribers.
>>> - dev@spark.apache.org has 3149 subscribers.
>>> - ASF Official Slack channel has 602 subscribers.
>>>
>>> May I ask if the users prefer to use the ASF Official Slack channel
>>> than the user mailing list?
>>>
>>> Dongjoon.
>>>
>>>
>>>
>>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>>> kabhwan.opensou...@gmail.com> wrote:
>>>
 I'm reading through the page "Briefing: The Apache Way", and in the
 section of "Open Communications", restriction of communication inside ASF
 INFRA (mailing list) is more about code and decision-making.

 https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define

 It's unavoidable if "users" prefer to use an alternative communication
 mechanism rather than the user mailing list. Before Stack Overflow days,
 there had been a meaningful number of questions around user@. It's
 just impossible to let them go back and post to the user mailing list.

 We just need to make sure it is not the purpose of employing Slack to
 move all discussions about developments, direction of the project, etc
 which must happen in dev@/private@. The purpose of Slack thread here
 does not seem to aim to serve the purpose.


 On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

> Good discussions and proposals.all around.
>
> I have used slack in anger on a customer site before. For small and
> medium size groups it is good and affordable. Alternatives have been
> suggested as well so those who like investigative search can agree and 
> come
> up with a freebie one.
> I am inclined to agree with Bjorn that this slack has more social
> dimensions than the mailing list. It is akin to a sports club using
> WhatsApp groups for communication. Remember we were originally looking for
> space for webinars, including Spark on Linkedin that Denney Lee suggested.
> I think Slack and mailing groups can coexist happily. On a more serious
> note, when I joined the user group back in 2015-2016, there was a lot of
> traffic. Currently we hardly get many mails daily <> less than 5. So 
> having

Re: Slack for PySpark users

[Jungtaek Lim]

Just to be clear, if there is no strong volunteer to make the new community
channel stay active, I'd probably be OK to not fork the channel. You can
see a strong counter example from #spark channel in ASF. It is the place
where there are only questions and promos but zero answers. I see
volunteers here demanding for another channel, so I want to see us go with
the most preferred way for these volunteers.

User mailing list does not go in a good shape. I hope we give another try
with recent technology to see whether we can gain traction - if we fail,
the user mailing list will still be there.

On Tue, Apr 4, 2023 at 7:04 AM Jungtaek Lim 
wrote:

> The number of subscribers doesn't give any meaningful value. Please look
> into the number of mails being sent to the list.
>
> https://lists.apache.org/list.html?u...@spark.apache.org
> The latest month there were more than 200 emails being sent was Feb 2022,
> more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
> and earlier.
> Let's face the fact. User mailing list is dying, even before we start
> discussion about alternative communication methods.
>
> Users never go with the way if it's just because PMC members (or ASF) have
> preference. They are going with the way they are convenient.
>
> Same applies here - if ASF Slack requires a restricted invitation
> mechanism then it won't work. Looks like there is a link for an invitation,
> but we are also talking about the cost as well.
> https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
> As long as we are being serious about the cost, I don't think we are going
> to land in the way "users" are convenient.
>
> On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - u...@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 Good discussions and proposals.all around.

 I have used slack in anger on a customer site before. For small and
 medium size groups it is good and affordable. Alternatives have been
 suggested as well so those who like investigative search can agree and come
 up with a freebie one.
 I am inclined to agree with Bjorn that this slack has more social
 dimensions than the mailing list. It is akin to a sports club using
 WhatsApp groups for communication. Remember we were originally looking for
 space for webinars, including Spark on Linkedin that Denney Lee suggested.
 I think Slack and mailing groups can coexist happily. On a more serious
 note, when I joined the user group back in 2015-2016, there was a lot of
 traffic. Currently we hardly get many mails daily <> less than 5. So having
 a slack type medium may improve members participation.

 so +1 for me as well.

 Mich Talebzadeh,
 Lead Solutions Architect/Engineering Lead
 Palantir Technologies Limited


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Thu, 30 Mar 

Re: Slack for PySpark users

[Jungtaek Lim]

The number of subscribers doesn't give any meaningful value. Please look
into the number of mails being sent to the list.

https://lists.apache.org/list.html?u...@spark.apache.org
The latest month there were more than 200 emails being sent was Feb 2022,
more than a year ago. It was more than 1k in 2016, and more than 2k in 2015
and earlier.
Let's face the fact. User mailing list is dying, even before we start
discussion about alternative communication methods.

Users never go with the way if it's just because PMC members (or ASF) have
preference. They are going with the way they are convenient.

Same applies here - if ASF Slack requires a restricted invitation mechanism
then it won't work. Looks like there is a link for an invitation, but we
are also talking about the cost as well.
https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites
As long as we are being serious about the cost, I don't think we are going
to land in the way "users" are convenient.

On Tue, Apr 4, 2023 at 4:59 AM Dongjoon Hyun 
wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - u...@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim 
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee  wrote:
>>>
 +1.

 To Shani’s point, there are multiple OSS projects that use the free
 Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
 MLflow, etc.

 On Thu, Mar 30, 2023 at 14:15  wrote:

> Hey everyone,
>
> I think we should remain on a free program in slack.
>
> In my option the free program is more then enough, the only down side
> is we could only see the last 90 days messages.
>
> From what I know the Airflow community (which has strong active
> community in slack) also use the free program (You can tell by the 90 days
> limit notice in their workspace).
>
> You can find the pricing and features comparison between the slack
> programs here 

Re: Slack for PySpark users

[Dongjoon Hyun]

Do you think there is a way to put it back to the official ASF-provided
Slack channel?

Dongjoon.

On Mon, Apr 3, 2023 at 2:18 PM Mich Talebzadeh 
wrote:

>
> I for myself prefer to use the newly formed slack.
>
> sparkcommunitytalk.slack.com
>
> In summary, it may be a good idea to take a tour of it and see for
> yourself. Topics are sectioned as per user requests.
>
> I trust this answers your question.
>
> Mich Talebzadeh,
> Lead Solutions Architect/Engineering Lead
> Palantir Technologies Limited
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun 
> wrote:
>
>> As Mich Talebzadeh pointed out, Apache Spark has an official Slack
>> channel.
>>
>> > It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list.
>>
>> The following is the number of people in the official channels.
>>
>> - u...@spark.apache.org has 4519 subscribers.
>> - dev@spark.apache.org has 3149 subscribers.
>> - ASF Official Slack channel has 602 subscribers.
>>
>> May I ask if the users prefer to use the ASF Official Slack channel
>> than the user mailing list?
>>
>> Dongjoon.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> I'm reading through the page "Briefing: The Apache Way", and in the
>>> section of "Open Communications", restriction of communication inside ASF
>>> INFRA (mailing list) is more about code and decision-making.
>>>
>>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>>
>>> It's unavoidable if "users" prefer to use an alternative communication
>>> mechanism rather than the user mailing list. Before Stack Overflow days,
>>> there had been a meaningful number of questions around user@. It's just
>>> impossible to let them go back and post to the user mailing list.
>>>
>>> We just need to make sure it is not the purpose of employing Slack to
>>> move all discussions about developments, direction of the project, etc
>>> which must happen in dev@/private@. The purpose of Slack thread here
>>> does not seem to aim to serve the purpose.
>>>
>>>
>>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 Good discussions and proposals.all around.

 I have used slack in anger on a customer site before. For small and
 medium size groups it is good and affordable. Alternatives have been
 suggested as well so those who like investigative search can agree and come
 up with a freebie one.
 I am inclined to agree with Bjorn that this slack has more social
 dimensions than the mailing list. It is akin to a sports club using
 WhatsApp groups for communication. Remember we were originally looking for
 space for webinars, including Spark on Linkedin that Denney Lee suggested.
 I think Slack and mailing groups can coexist happily. On a more serious
 note, when I joined the user group back in 2015-2016, there was a lot of
 traffic. Currently we hardly get many mails daily <> less than 5. So having
 a slack type medium may improve members participation.

 so +1 for me as well.

 Mich Talebzadeh,
 Lead Solutions Architect/Engineering Lead
 Palantir Technologies Limited


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Thu, 30 Mar 2023 at 22:19, Denny Lee  wrote:

> +1.
>
> To Shani’s point, there are multiple OSS projects that use the free
> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
> MLflow, etc.
>
> On Thu, Mar 30, 2023 at 14:15  wrote:
>
>> Hey everyone,
>>
>> I think we should remain on a free program in slack.
>>
>> In my option the free program is more then enough, the only down side
>> is we could only see the last 90 days messages.
>>
>> From what I know the Airflow community (which has strong active
>> community in slack) also use the free program (You can tell by the 90 

Re: Slack for PySpark users

[Mich Talebzadeh]

I for myself prefer to use the newly formed slack.

sparkcommunitytalk.slack.com

In summary, it may be a good idea to take a tour of it and see for
yourself. Topics are sectioned as per user requests.

I trust this answers your question.

Mich Talebzadeh,
Lead Solutions Architect/Engineering Lead
Palantir Technologies Limited


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Mon, 3 Apr 2023 at 20:59, Dongjoon Hyun  wrote:

> As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.
>
> > It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list.
>
> The following is the number of people in the official channels.
>
> - u...@spark.apache.org has 4519 subscribers.
> - dev@spark.apache.org has 3149 subscribers.
> - ASF Official Slack channel has 602 subscribers.
>
> May I ask if the users prefer to use the ASF Official Slack channel
> than the user mailing list?
>
> Dongjoon.
>
>
>
> On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim 
> wrote:
>
>> I'm reading through the page "Briefing: The Apache Way", and in the
>> section of "Open Communications", restriction of communication inside ASF
>> INFRA (mailing list) is more about code and decision-making.
>>
>> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>>
>> It's unavoidable if "users" prefer to use an alternative communication
>> mechanism rather than the user mailing list. Before Stack Overflow days,
>> there had been a meaningful number of questions around user@. It's just
>> impossible to let them go back and post to the user mailing list.
>>
>> We just need to make sure it is not the purpose of employing Slack to
>> move all discussions about developments, direction of the project, etc
>> which must happen in dev@/private@. The purpose of Slack thread here
>> does not seem to aim to serve the purpose.
>>
>>
>> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>> Good discussions and proposals.all around.
>>>
>>> I have used slack in anger on a customer site before. For small and
>>> medium size groups it is good and affordable. Alternatives have been
>>> suggested as well so those who like investigative search can agree and come
>>> up with a freebie one.
>>> I am inclined to agree with Bjorn that this slack has more social
>>> dimensions than the mailing list. It is akin to a sports club using
>>> WhatsApp groups for communication. Remember we were originally looking for
>>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>>> I think Slack and mailing groups can coexist happily. On a more serious
>>> note, when I joined the user group back in 2015-2016, there was a lot of
>>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>>> a slack type medium may improve members participation.
>>>
>>> so +1 for me as well.
>>>
>>> Mich Talebzadeh,
>>> Lead Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Thu, 30 Mar 2023 at 22:19, Denny Lee  wrote:
>>>
 +1.

 To Shani’s point, there are multiple OSS projects that use the free
 Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
 MLflow, etc.

 On Thu, Mar 30, 2023 at 14:15  wrote:

> Hey everyone,
>
> I think we should remain on a free program in slack.
>
> In my option the free program is more then enough, the only down side
> is we could only see the last 90 days messages.
>
> From what I know the Airflow community (which has strong active
> community in slack) also use the free program (You can tell by the 90 days
> limit notice in their workspace).
>
> You can find the pricing and features comparison between the slack
> programs here  .
>
> Have a great day,
> Shani
>
> On 30 Mar 2023, at 23:38, Mridul Muralidharan 
> wrote:
>
> 
>
>
> Thanks for 

Re: Slack for PySpark users

[Dongjoon Hyun]

As Mich Talebzadeh pointed out, Apache Spark has an official Slack channel.

> It's unavoidable if "users" prefer to use an alternative communication
mechanism rather than the user mailing list.

The following is the number of people in the official channels.

- u...@spark.apache.org has 4519 subscribers.
- dev@spark.apache.org has 3149 subscribers.
- ASF Official Slack channel has 602 subscribers.

May I ask if the users prefer to use the ASF Official Slack channel
than the user mailing list?

Dongjoon.



On Thu, Mar 30, 2023 at 9:10 PM Jungtaek Lim 
wrote:

> I'm reading through the page "Briefing: The Apache Way", and in the
> section of "Open Communications", restriction of communication inside ASF
> INFRA (mailing list) is more about code and decision-making.
>
> https://www.apache.org/theapacheway/#what-makes-the-apache-way-so-hard-to-define
>
> It's unavoidable if "users" prefer to use an alternative communication
> mechanism rather than the user mailing list. Before Stack Overflow days,
> there had been a meaningful number of questions around user@. It's just
> impossible to let them go back and post to the user mailing list.
>
> We just need to make sure it is not the purpose of employing Slack to move
> all discussions about developments, direction of the project, etc which
> must happen in dev@/private@. The purpose of Slack thread here does not
> seem to aim to serve the purpose.
>
>
> On Fri, Mar 31, 2023 at 7:00 AM Mich Talebzadeh 
> wrote:
>
>> Good discussions and proposals.all around.
>>
>> I have used slack in anger on a customer site before. For small and
>> medium size groups it is good and affordable. Alternatives have been
>> suggested as well so those who like investigative search can agree and come
>> up with a freebie one.
>> I am inclined to agree with Bjorn that this slack has more social
>> dimensions than the mailing list. It is akin to a sports club using
>> WhatsApp groups for communication. Remember we were originally looking for
>> space for webinars, including Spark on Linkedin that Denney Lee suggested.
>> I think Slack and mailing groups can coexist happily. On a more serious
>> note, when I joined the user group back in 2015-2016, there was a lot of
>> traffic. Currently we hardly get many mails daily <> less than 5. So having
>> a slack type medium may improve members participation.
>>
>> so +1 for me as well.
>>
>> Mich Talebzadeh,
>> Lead Solutions Architect/Engineering Lead
>> Palantir Technologies Limited
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Thu, 30 Mar 2023 at 22:19, Denny Lee  wrote:
>>
>>> +1.
>>>
>>> To Shani’s point, there are multiple OSS projects that use the free
>>> Slack version - top of mind include Delta, Presto, Flink, Trino, Datahub,
>>> MLflow, etc.
>>>
>>> On Thu, Mar 30, 2023 at 14:15  wrote:
>>>
 Hey everyone,

 I think we should remain on a free program in slack.

 In my option the free program is more then enough, the only down side
 is we could only see the last 90 days messages.

 From what I know the Airflow community (which has strong active
 community in slack) also use the free program (You can tell by the 90 days
 limit notice in their workspace).

 You can find the pricing and features comparison between the slack
 programs here  .

 Have a great day,
 Shani

 On 30 Mar 2023, at 23:38, Mridul Muralidharan  wrote:

 


 Thanks for flagging the concern Dongjoon, I was not aware of the
 discussion - but I can understand the concern.
 Would be great if you or Matei could update the thread on the result of
 deliberations, once it reaches a logical consensus: before we set up
 official policy around it.

 Regards,
 Mridul


 On Thu, Mar 30, 2023 at 4:23 PM Bjørn Jørgensen <
 bjornjorgen...@gmail.com> wrote:

> I like the idea of having a talk channel. It can make it easier for
> everyone to say hello. Or to dare to ask about small or big matters that
> you would not have dared to ask about before on mailing lists.
> But then there is the price and what is the best for an open source
> project.
>
> The price for using slack is expensive.
> Right now for those that have join spark slack
> $8.75 USD
> 72 members
> 1 month
> $630 USD
>
> https://app.slack.com/plans/T04URTRBZ1R/checkout/form?entry_point=hero_banner_upgrade_cta=2
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[L. C. Hsieh]

+1

Thanks Xinrong.

On Mon, Apr 3, 2023 at 12:35 PM Dongjoon Hyun  wrote:
>
> +1
>
> I also verified that RC5 has SBOM artifacts.
>
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.12/3.4.0/spark-core_2.12-3.4.0-cyclonedx.json
> https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.13/3.4.0/spark-core_2.13-3.4.0-cyclonedx.json
>
> Thanks,
> Dongjoon.
>
>
>
> On Mon, Apr 3, 2023 at 1:57 AM yangjie01  wrote:
>>
>> +1, checked Java 17 + Scala 2.13 + Python 3.10.10.
>>
>>
>>
>> 发件人: Herman van Hovell 
>> 日期: 2023年3月31日 星期五 12:12
>> 收件人: Sean Owen 
>> 抄送: Xinrong Meng , dev 
>> 主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC5)
>>
>>
>>
>> +1
>>
>>
>>
>> On Thu, Mar 30, 2023 at 11:05 PM Sean Owen  wrote:
>>
>> +1 same result from me as last time.
>>
>>
>>
>> On Thu, Mar 30, 2023 at 3:21 AM Xinrong Meng  
>> wrote:
>>
>> Please vote on releasing the following candidate(RC5) as Apache Spark 
>> version 3.4.0.
>>
>> The vote is open until 11:59pm Pacific time April 4th and passes if a 
>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.4.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> The tag to be voted on is v3.4.0-rc5 (commit 
>> f39ad617d32a671e120464e4a75986241d72c487):
>> https://github.com/apache/spark/tree/v3.4.0-rc5
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1439
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/
>>
>> The list of bug fixes going into 3.4.0 can be found at the following URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>>
>> This release is using the release script of the tag v3.4.0-rc5.
>>
>>
>>
>>
>>
>> FAQ
>>
>> =
>> How can I help test this release?
>> =
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with an out of date RC going forward).
>>
>> ===
>> What should happen to JIRA tickets still targeting 3.4.0?
>> ===
>> The current list of open tickets targeted at 3.4.0 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target 
>> Version/s" = 3.4.0
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==
>> But my bug isn't fixed?
>> ==
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>>
>>
>> Thanks,
>>
>> Xinrong Meng
>>
>>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[Jacek Laskowski]

+1

Compiled on Java 17 with Scala 2.13 on macos and ran some basic code.

Pozdrawiam,
Jacek Laskowski

"The Internals Of" Online Books 
Follow me on https://twitter.com/jaceklaskowski




On Thu, Mar 30, 2023 at 10:21 AM Xinrong Meng 
wrote:

> Please vote on releasing the following candidate(RC5) as Apache Spark
> version 3.4.0.
>
> The vote is open until 11:59pm Pacific time *April 4th* and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is *v3.4.0-rc5* (commit
> f39ad617d32a671e120464e4a75986241d72c487):
> https://github.com/apache/spark/tree/v3.4.0-rc5
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1439
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
>
> This release is using the release script of the tag v3.4.0-rc5.
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Thanks,
> Xinrong Meng
>
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[Dongjoon Hyun]

+1

I also verified that RC5 has SBOM artifacts.

https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.12/3.4.0/spark-core_2.12-3.4.0-cyclonedx.json
https://repository.apache.org/content/repositories/orgapachespark-1439/org/apache/spark/spark-core_2.13/3.4.0/spark-core_2.13-3.4.0-cyclonedx.json

Thanks,
Dongjoon.



On Mon, Apr 3, 2023 at 1:57 AM yangjie01  wrote:

> +1, checked Java 17 + Scala 2.13 + Python 3.10.10.
>
>
>
> *发件人**: *Herman van Hovell 
> *日期**: *2023年3月31日 星期五 12:12
> *收件人**: *Sean Owen 
> *抄送**: *Xinrong Meng , dev  >
> *主题**: *Re: [VOTE] Release Apache Spark 3.4.0 (RC5)
>
>
>
> +1
>
>
>
> On Thu, Mar 30, 2023 at 11:05 PM Sean Owen  wrote:
>
> +1 same result from me as last time.
>
>
>
> On Thu, Mar 30, 2023 at 3:21 AM Xinrong Meng 
> wrote:
>
> Please vote on releasing the following candidate(RC5) as Apache Spark
> version 3.4.0.
>
> The vote is open until 11:59pm Pacific time *April 4th* and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
> 
>
> The tag to be voted on is *v3.4.0-rc5* (commit
> f39ad617d32a671e120464e4a75986241d72c487):
> https://github.com/apache/spark/tree/v3.4.0-rc5
> 
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/
> 
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
> 
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1439
> 
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/
> 
>
> The list of bug fixes going into 3.4.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12351465
> 
>
> This release is using the release script of the tag v3.4.0-rc5.
>
>
>
>
>
> FAQ
>
> =
> How can I help test this release?
> =
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.0?
> ===
> The current list of open tickets targeted at 3.4.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK
> 
>  and
> search for "Target Version/s" = 3.4.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
>
>
> Thanks,
>
> Xinrong Meng
>
>
>
>

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

[yangjie01]

+1, checked Java 17 + Scala 2.13 + Python 3.10.10.

发件人: Herman van Hovell 
日期: 2023年3月31日 星期五 12:12
收件人: Sean Owen 
抄送: Xinrong Meng , dev 
主题: Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

+1

On Thu, Mar 30, 2023 at 11:05 PM Sean Owen 
mailto:sro...@apache.org>> wrote:
+1 same result from me as last time.

On Thu, Mar 30, 2023 at 3:21 AM Xinrong Meng 
mailto:xinrong.apa...@gmail.com>> wrote:
Please vote on releasing the following candidate(RC5) as Apache Spark version 
3.4.0.

The vote is open until 11:59pm Pacific time April 4th and passes if a majority 
+1 PMC votes are cast, with a minimum of 3 +1 votes.

[ ] +1 Release this package as Apache Spark 3.4.0
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see 
http://spark.apache.org/

The tag to be voted on is v3.4.0-rc5 (commit 
f39ad617d32a671e120464e4a75986241d72c487):
https://github.com/apache/spark/tree/v3.4.0-rc5

The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-bin/

Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1439

The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.4.0-rc5-docs/

The list of bug fixes going into 3.4.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12351465

This release is using the release script of the tag v3.4.0-rc5.


FAQ

=
How can I help test this release?
=
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.

If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).

===
What should happen to JIRA tickets still targeting 3.4.0?
===
The current list of open tickets targeted at 3.4.0 can be found at:
https://issues.apache.org/jira/projects/SPARK
 and search for "Target Version/s" = 3.4.0

Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.

==
But my bug isn't fixed?
==
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.

Thanks,
Xinrong Meng