Re: Welcoming some new Apache Spark committers

[Yuanjian Li]

Congratulations!!

huaxin gao  于2020年7月16日周四 上午6:24写道：

> Thanks everyone! I am looking forward to working with you all in the
> future.
>
> On Tue, Jul 14, 2020 at 5:02 PM Hyukjin Kwon  wrote:
>
>> Congrats!
>>
>> 2020년 7월 15일 (수) 오전 7:56, Takeshi Yamamuro 님이 작성:
>>
>>> Congrats, all!
>>>
>>> On Wed, Jul 15, 2020 at 5:15 AM Takuya UESHIN 
>>> wrote:
>>>
 Congrats and welcome!

 On Tue, Jul 14, 2020 at 1:07 PM Bryan Cutler  wrote:

> Congratulations and welcome!
>
> On Tue, Jul 14, 2020 at 12:36 PM Xingbo Jiang 
> wrote:
>
>> Welcome, Huaxin, Jungtaek, and Dilip!
>>
>> Congratulations!
>>
>> On Tue, Jul 14, 2020 at 10:37 AM Matei Zaharia <
>> matei.zaha...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> The Spark PMC recently voted to add several new committers. Please
>>> join me in welcoming them to their new roles! The new committers are:
>>>
>>> - Huaxin Gao
>>> - Jungtaek Lim
>>> - Dilip Biswal
>>>
>>> All three of them contributed to Spark 3.0 and we’re excited to have
>>> them join the project.
>>>
>>> Matei and the Spark PMC
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

 --
 Takuya UESHIN


>>>
>>> --
>>> ---
>>> Takeshi Yamamuro
>>>
>>

Re: Welcoming some new Apache Spark committers

[huaxin gao]

Thanks everyone! I am looking forward to working with you all in the
future.

On Tue, Jul 14, 2020 at 5:02 PM Hyukjin Kwon  wrote:

> Congrats!
>
> 2020년 7월 15일 (수) 오전 7:56, Takeshi Yamamuro 님이 작성:
>
>> Congrats, all!
>>
>> On Wed, Jul 15, 2020 at 5:15 AM Takuya UESHIN 
>> wrote:
>>
>>> Congrats and welcome!
>>>
>>> On Tue, Jul 14, 2020 at 1:07 PM Bryan Cutler  wrote:
>>>
 Congratulations and welcome!

 On Tue, Jul 14, 2020 at 12:36 PM Xingbo Jiang 
 wrote:

> Welcome, Huaxin, Jungtaek, and Dilip!
>
> Congratulations!
>
> On Tue, Jul 14, 2020 at 10:37 AM Matei Zaharia <
> matei.zaha...@gmail.com> wrote:
>
>> Hi all,
>>
>> The Spark PMC recently voted to add several new committers. Please
>> join me in welcoming them to their new roles! The new committers are:
>>
>> - Huaxin Gao
>> - Jungtaek Lim
>> - Dilip Biswal
>>
>> All three of them contributed to Spark 3.0 and we’re excited to have
>> them join the project.
>>
>> Matei and the Spark PMC
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>
>>>
>>> --
>>> Takuya UESHIN
>>>
>>>
>>
>> --
>> ---
>> Takeshi Yamamuro
>>
>

Re: [DISCUSS] -1s and commits

[Sean Owen]

I agree with all that, and would be surprised if anyone here objects
to any of that in principle. In practice, I'm sure it doesn't end up
that way sometimes, even in good faith. That is, I would not be
surprised if the parties involved don't even see the disconnect.

What are the specific examples? for private@ if necessary, but, I
think dev@ could be fine because we're discussing general patterns
from specific examples, for everyone to learn from. This isn't
necessarily about individuals. (Heck, maybe I've gotten it wrong
myself)

One general principle I'd add: we are probably getting more
conservative about big changes over time as Spark enters the long
plateau of maturity. See the discussion about breaking changes in 3.0,
or comments about waiting for review. That argues even more against
proceeding against raised issues.

On the flip side, we have to be constructive. I like the idea of
proposing alternatives. Can you achieve this goal by doing Y instead
of X?  I also think there's a burden on the objector to provide a
rationale, certainly, but also drive a resolution. That could also
mean standing firm on the objection but calling in other reviewers and
being willing to accede to a majority. Put another way: someone who
objects and never really follows up with a path to consensus about
compromise or rejection isn't really objecting correctly. We can VOTE
if needed, but, if someone objected and didn't follow up and I
couldn't find anyone else backing it up and thought I'd addressed the
objection, I'd consider it resolved and proceed.



On Wed, Jul 15, 2020 at 3:18 PM Holden Karau  wrote:
>
> Hi Spark Development Community,
>
>
> Since Spark 3 has shipped I've been going through some of the PRs and I've 
> noticed some PRs have been merged with pending -1s with technical reasons, 
> including those from committers. I'm bringing this up because I believe we, 
> as PMC, committers and contributors, do not currently have a consistent 
> understanding, and I believe we should develop one. The foundation level 
> guidance is at 
> https://www.apache.org/foundation/voting.html#votes-on-code-modification.
>
>
>
> It is my belief that we should not be merging code with -1s from active 
> committers, -1 is a very strong signal and generally a sign that we should 
> step back and try and build consensus around the change. Looking at how the 
> httpd project (e.g. the original Apache project) handles it 
> https://httpd.apache.org/dev/guidelines.html#voting it seems to be that once 
> a -1 from a committer is received we can no longer use our regular lazy 
> consensus mechanism for the change, and we then need to have a vote or get 
> the -1 resolved with the person who placed it.
>
>
>
> Now of course, if the -1s aren't following the guidelines that's a different 
> story (e.g. no technical reason provided), but regardless a -1 from a 
> committer to me would require a public vote on dev@ following the 
> foundation's voting guidelines.
>
>
>
> I believe, especially in the Spark project, committers have demonstrated 
> sufficient merit and they are qualified to vote on code changes so a -1 
> should block merging, however in talking with other ASF projects there are a 
> variety of ways of handling this. The unanimous opinion from the different 
> folks I talked with is that any technical disagreement should be considered 
> before moving on. Just waiting a few days and merging the code is not a valid 
> solution. In the context of how much work and history we've chosen to require 
> Spark committers to demonstrate, most folks seem to believe that committers 
> in the Spark project would be qualified voters for these purposes.
>
>
>
> In general I expect -1s to continue to be relatively rare in the Spark 
> community, and if this is no longer the case I believe we should take the 
> areas where we are seeing more -1s and have a broader discussion to give us 
> the opportunity to build consensus prior to making any further non-bugfix 
> changes in those components/areas. Most folks from other projects seemed to 
> share this concern as well.
>
>
> One of the things that was also brought up in this context is some projects 
> require -1s to provide an alternative suggestion as well as the technical 
> objection. That could be something we, as a project, could adopt if were 
> concerned with over use of -1s.
>
>
>
> Sincerely,
>
>
>
> Holden
>
>
>
> P.S.
>
>
>
> I know this may be a sensitive topic, and we're all under more stress than 
> usual right now. I appreciate everyone's patience while we discuss this. I 
> know I find this a sensitive topic as well given the seemingly inconsistent 
> standards. I'm tired of having people ask me to wait on PRs that have been 
> open for more than a month and merging their own code with pending 
> substantial issues raised by qualified members of the community.
>
>
>
> I'm not saying that the actions taken were necessarily wrong, just that I 
> believe reaching a common 

[DISCUSS] -1s and commits

[Holden Karau]

Hi Spark Development Community,


Since Spark 3 has shipped I've been going through some of the PRs and I've
noticed some PRs have been merged with pending -1s with technical reasons,
including those from committers. I'm bringing this up because I believe we,
as PMC, committers and contributors, do not currently have a consistent
understanding, and I believe we should develop one. The foundation level
guidance is at
https://www.apache.org/foundation/voting.html#votes-on-code-modification.



It is my belief that we should not be merging code with -1s from active
committers, -1 is a very strong signal and generally a sign that we should
step back and try and build consensus around the change. Looking at how the
httpd project (e.g. the original Apache project) handles it
https://httpd.apache.org/dev/guidelines.html#voting it seems to be that
once a -1 from a committer is received we can no longer use our regular
lazy consensus mechanism for the change, and we then need to have a vote or
get the -1 resolved with the person who placed it.



Now of course, if the -1s aren't following the guidelines that's a
different story (e.g. no technical reason provided), but regardless a -1
from a committer to me would require a public vote on dev@ following the
foundation's voting guidelines.



I believe, especially in the Spark project, committers have demonstrated
sufficient merit and they are qualified to vote on code changes so a -1
should block merging, however in talking with other ASF projects there are
a variety of ways of handling this. The unanimous opinion from the
different folks I talked with is that any technical disagreement should be
considered before moving on. Just waiting a few days and merging the code
is not a valid solution. In the context of how much work and history we've
chosen to require Spark committers to demonstrate, most folks seem to
believe that committers in the Spark project would be qualified voters for
these purposes.



In general I expect -1s to continue to be relatively rare in the Spark
community, and if this is no longer the case I believe we should take the
areas where we are seeing more -1s and have a broader discussion to give us
the opportunity to build consensus prior to making any further non-bugfix
changes in those components/areas. Most folks from other projects seemed to
share this concern as well.


One of the things that was also brought up in this context is some projects
require -1s to provide an alternative suggestion as well as the technical
objection. That could be something we, as a project, could adopt if were
concerned with over use of -1s.



Sincerely,



Holden



P.S.



I know this may be a sensitive topic, and we're all under more stress than
usual right now. I appreciate everyone's patience while we discuss this. I
know I find this a sensitive topic as well given the seemingly inconsistent
standards. I'm tired of having people ask me to wait on PRs that have been
open for more than a month and merging their own code with pending
substantial issues raised by qualified members of the community.



I'm not saying that the actions taken were necessarily wrong, just that I
believe reaching a common understanding here is crucial to the healthy
functioning of the project.



I have not included any specific examples of this since this is the public
list and I believe discussions involving individuals does not belong on dev@.
If we want to discuss the specific situations that I noticed from the Spark
3 release, we can fork that conversation to private@.



For any ASF members you can also find the discussion with folks from other
projects and their views on that mailing list and get at the details.


-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Welcoming some new Apache Spark committers

[Maxim Gekk]

Congratulations, Huaxin, Jungtaek, Dilip!!!

Maxim Gekk

Software Engineer

Databricks, Inc.


On Wed, Jul 15, 2020 at 8:34 PM Felix Cheung 
wrote:

> Welcome!
>
> --
> *From:* Nick Pentreath 
> *Sent:* Tuesday, July 14, 2020 10:21:17 PM
> *To:* dev 
> *Cc:* Dilip Biswal ; Jungtaek Lim <
> kabhwan.opensou...@gmail.com>; huaxin gao 
> *Subject:* Re: Welcoming some new Apache Spark committers
>
> Congratulations and welcome as Apache Spark committers!
>
> On Wed, 15 Jul 2020 at 06:59, Prashant Sharma 
> wrote:
>
> Congratulations all ! It's great to have such committed folks as
> committers. :)
>
> On Wed, Jul 15, 2020 at 9:24 AM Yi Wu  wrote:
>
> Congrats!!
>
> On Wed, Jul 15, 2020 at 8:02 AM Hyukjin Kwon  wrote:
>
> Congrats!
>
> 2020년 7월 15일 (수) 오전 7:56, Takeshi Yamamuro 님이 작성:
>
> Congrats, all!
>
> On Wed, Jul 15, 2020 at 5:15 AM Takuya UESHIN 
> wrote:
>
> Congrats and welcome!
>
> On Tue, Jul 14, 2020 at 1:07 PM Bryan Cutler  wrote:
>
> Congratulations and welcome!
>
> On Tue, Jul 14, 2020 at 12:36 PM Xingbo Jiang 
> wrote:
>
> Welcome, Huaxin, Jungtaek, and Dilip!
>
> Congratulations!
>
> On Tue, Jul 14, 2020 at 10:37 AM Matei Zaharia 
> wrote:
>
> Hi all,
>
> The Spark PMC recently voted to add several new committers. Please join me
> in welcoming them to their new roles! The new committers are:
>
> - Huaxin Gao
> - Jungtaek Lim
> - Dilip Biswal
>
> All three of them contributed to Spark 3.0 and we’re excited to have them
> join the project.
>
> Matei and the Spark PMC
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>
>
> --
> Takuya UESHIN
>
>
>
> --
> ---
> Takeshi Yamamuro
>
>

Re: R installation broken on ubuntu workers, impacts K8s PRB builds

[Holden Karau]

Oh cool, I filed a JIRA for this already and assigned it to you (noticed in
one of my PRs)- https://issues.apache.org/jira/browse/SPARK-32326

On Wed, Jul 15, 2020 at 12:09 PM shane knapp ☠  wrote:

> i'm not entirely sure when the dep for R got bumped to 3.5+, but it's
> breaking the k8s builds.
>
> i'll need to purge these workers of all previous versions of R + packages,
> then reinstall from scratch.  this isn't a horrible task as i have most of
> it automated but it will still require a ~few hours of downtime.
>
> i'll file a JIRA, and figure out when i will be able to get to this...
> possibly this afternoon.
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>


-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

R installation broken on ubuntu workers, impacts K8s PRB builds

[shane knapp ☠]

i'm not entirely sure when the dep for R got bumped to 3.5+, but it's
breaking the k8s builds.

i'll need to purge these workers of all previous versions of R + packages,
then reinstall from scratch.  this isn't a horrible task as i have most of
it automated but it will still require a ~few hours of downtime.

i'll file a JIRA, and figure out when i will be able to get to this...
possibly this afternoon.
-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Re: Welcoming some new Apache Spark committers

[Felix Cheung]

Welcome!


From: Nick Pentreath 
Sent: Tuesday, July 14, 2020 10:21:17 PM
To: dev 
Cc: Dilip Biswal ; Jungtaek Lim 
; huaxin gao 
Subject: Re: Welcoming some new Apache Spark committers

Congratulations and welcome as Apache Spark committers!

On Wed, 15 Jul 2020 at 06:59, Prashant Sharma 
mailto:scrapco...@gmail.com>> wrote:
Congratulations all ! It's great to have such committed folks as committers. :)

On Wed, Jul 15, 2020 at 9:24 AM Yi Wu 
mailto:yi...@databricks.com>> wrote:
Congrats!!

On Wed, Jul 15, 2020 at 8:02 AM Hyukjin Kwon 
mailto:gurwls...@gmail.com>> wrote:
Congrats!

2020년 7월 15일 (수) 오전 7:56, Takeshi Yamamuro 
mailto:linguin@gmail.com>>님이 작성:
Congrats, all!

On Wed, Jul 15, 2020 at 5:15 AM Takuya UESHIN 
mailto:ues...@happy-camper.st>> wrote:
Congrats and welcome!

On Tue, Jul 14, 2020 at 1:07 PM Bryan Cutler 
mailto:cutl...@gmail.com>> wrote:
Congratulations and welcome!

On Tue, Jul 14, 2020 at 12:36 PM Xingbo Jiang 
mailto:jiangxb1...@gmail.com>> wrote:
Welcome, Huaxin, Jungtaek, and Dilip!

Congratulations!

On Tue, Jul 14, 2020 at 10:37 AM Matei Zaharia 
mailto:matei.zaha...@gmail.com>> wrote:
Hi all,

The Spark PMC recently voted to add several new committers. Please join me in 
welcoming them to their new roles! The new committers are:

- Huaxin Gao
- Jungtaek Lim
- Dilip Biswal

All three of them contributed to Spark 3.0 and we’re excited to have them join 
the project.

Matei and the Spark PMC
-
To unsubscribe e-mail: 
dev-unsubscr...@spark.apache.org



--
Takuya UESHIN



--
---
Takeshi Yamamuro

Re: [DISCUSS] Apache Spark 3.0.1 Release

[Wenchen Fan]

Supporting Python 3.8.0 sounds like a new feature, and doesn't qualify a
backport. But I'm open to other opinions.

On Wed, Jul 15, 2020 at 11:24 PM Ismaël Mejía  wrote:

> Any chance that SPARK-29536 PySpark does not work with Python 3.8.0
> can be backported to 2.4.7 ?
> This was not done for Spark 2.4.6 because it was too late on the vote
> process but it makes perfect sense to have this in 2.4.7.
>
> On Wed, Jul 15, 2020 at 9:07 AM Wenchen Fan  wrote:
> >
> > Yea I think 2.4.7 is good to go. Let's start!
> >
> > On Wed, Jul 15, 2020 at 1:50 PM Prashant Sharma 
> wrote:
> >>
> >> Hi Folks,
> >>
> >> So, I am back, and searched the JIRAS with target version as "2.4.7"
> and Resolved, found only 2 jiras. So, are we good to go, with just a couple
> of jiras fixed ? Shall I proceed with making a RC?
> >>
> >> Thanks,
> >> Prashant
> >>
> >> On Thu, Jul 2, 2020 at 5:23 PM Prashant Sharma 
> wrote:
> >>>
> >>> Thank you, Holden.
> >>>
> >>> Folks, My health has gone down a bit. So, I will start working on this
> in a few days. If this needs to be published sooner, then maybe someone
> else has to help out.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Jul 2, 2020 at 10:11 AM Holden Karau 
> wrote:
> 
>  I’m happy to have Prashant do 2.4.7 :)
> 
>  On Wed, Jul 1, 2020 at 9:40 PM Xiao Li  wrote:
> >
> > +1 on releasing both 3.0.1 and 2.4.7
> >
> > Great! Three committers volunteer to be a release manager. Ruifeng,
> Prashant and Holden. Holden just helped release Spark 2.4.6. This time,
> maybe, Ruifeng and Prashant can be the release manager of 3.0.1 and 2.4.7
> respectively.
> >
> > Xiao
> >
> > On Wed, Jul 1, 2020 at 2:24 PM Jungtaek Lim <
> kabhwan.opensou...@gmail.com> wrote:
> >>
> >> https://issues.apache.org/jira/browse/SPARK-32148 was reported
> yesterday, and if the report is valid it looks to be a blocker. I'll try to
> take a look sooner.
> >>
> >> On Thu, Jul 2, 2020 at 12:48 AM Shivaram Venkataraman <
> shiva...@eecs.berkeley.edu> wrote:
> >>>
> >>> Thanks Holden -- it would be great to also get 2.4.7 started
> >>>
> >>> Thanks
> >>> Shivaram
> >>>
> >>> On Tue, Jun 30, 2020 at 10:31 PM Holden Karau <
> hol...@pigscanfly.ca> wrote:
> >>> >
> >>> > I can take care of 2.4.7 unless someone else wants to do it.
> >>> >
> >>> > On Tue, Jun 30, 2020 at 8:29 PM Jason Moore <
> jason.mo...@quantium.com.au> wrote:
> >>> >>
> >>> >> Hi all,
> >>> >>
> >>> >>
> >>> >>
> >>> >> Could I get some input on the severity of this one that I found
> yesterday?  If that’s a correctness issue, should it block this patch?  Let
> me know under the ticket if there’s more info that I can provide to help.
> >>> >>
> >>> >>
> >>> >>
> >>> >> https://issues.apache.org/jira/browse/SPARK-32136
> >>> >>
> >>> >>
> >>> >>
> >>> >> Thanks,
> >>> >>
> >>> >> Jason.
> >>> >>
> >>> >>
> >>> >>
> >>> >> From: Jungtaek Lim 
> >>> >> Date: Wednesday, 1 July 2020 at 10:20 am
> >>> >> To: Shivaram Venkataraman 
> >>> >> Cc: Prashant Sharma , 郑瑞峰 <
> ruife...@foxmail.com>, Gengliang Wang ,
> gurwls223 , Dongjoon Hyun ,
> Jules Damji , Holden Karau ,
> Reynold Xin , Yuanjian Li , "
> dev@spark.apache.org" , Takeshi Yamamuro <
> linguin@gmail.com>
> >>> >> Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release
> >>> >>
> >>> >>
> >>> >>
> >>> >> SPARK-32130 [1] looks to be a performance regression introduced
> in Spark 3.0.0, which is ideal to look into before releasing another bugfix
> version.
> >>> >>
> >>> >>
> >>> >>
> >>> >> 1. https://issues.apache.org/jira/browse/SPARK-32130
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Wed, Jul 1, 2020 at 7:05 AM Shivaram Venkataraman <
> shiva...@eecs.berkeley.edu> wrote:
> >>> >>
> >>> >> Hi all
> >>> >>
> >>> >>
> >>> >>
> >>> >> I just wanted to ping this thread to see if all the outstanding
> blockers for 3.0.1 have been fixed. If so, it would be great if we can get
> the release going. The CRAN team sent us a note that the version SparkR
> available on CRAN for the current R version (4.0.2) is broken and hence we
> need to update the package soon --  it will be great to do it with 3.0.1.
> >>> >>
> >>> >>
> >>> >>
> >>> >> Thanks
> >>> >>
> >>> >> Shivaram
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Wed, Jun 24, 2020 at 8:31 PM Prashant Sharma <
> scrapco...@gmail.com> wrote:
> >>> >>
> >>> >> +1 for 3.0.1 release.
> >>> >>
> >>> >> I too can help out as release manager.
> >>> >>
> >>> >>
> >>> >>
> >>> >> On Thu, Jun 25, 2020 at 4:58 AM 郑瑞峰 
> wrote:
> >>> >>
> >>> >> I volunteer to be a release manager of 3.0.1, if nobody is
> working on this.
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> -- 原始邮件 

Re: [DISCUSS] Apache Spark 3.0.1 Release

[Ismaël Mejía]

Any chance that SPARK-29536 PySpark does not work with Python 3.8.0
can be backported to 2.4.7 ?
This was not done for Spark 2.4.6 because it was too late on the vote
process but it makes perfect sense to have this in 2.4.7.

On Wed, Jul 15, 2020 at 9:07 AM Wenchen Fan  wrote:
>
> Yea I think 2.4.7 is good to go. Let's start!
>
> On Wed, Jul 15, 2020 at 1:50 PM Prashant Sharma  wrote:
>>
>> Hi Folks,
>>
>> So, I am back, and searched the JIRAS with target version as "2.4.7" and 
>> Resolved, found only 2 jiras. So, are we good to go, with just a couple of 
>> jiras fixed ? Shall I proceed with making a RC?
>>
>> Thanks,
>> Prashant
>>
>> On Thu, Jul 2, 2020 at 5:23 PM Prashant Sharma  wrote:
>>>
>>> Thank you, Holden.
>>>
>>> Folks, My health has gone down a bit. So, I will start working on this in a 
>>> few days. If this needs to be published sooner, then maybe someone else has 
>>> to help out.
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jul 2, 2020 at 10:11 AM Holden Karau  wrote:

 I’m happy to have Prashant do 2.4.7 :)

 On Wed, Jul 1, 2020 at 9:40 PM Xiao Li  wrote:
>
> +1 on releasing both 3.0.1 and 2.4.7
>
> Great! Three committers volunteer to be a release manager. Ruifeng, 
> Prashant and Holden. Holden just helped release Spark 2.4.6. This time, 
> maybe, Ruifeng and Prashant can be the release manager of 3.0.1 and 2.4.7 
> respectively.
>
> Xiao
>
> On Wed, Jul 1, 2020 at 2:24 PM Jungtaek Lim 
>  wrote:
>>
>> https://issues.apache.org/jira/browse/SPARK-32148 was reported 
>> yesterday, and if the report is valid it looks to be a blocker. I'll try 
>> to take a look sooner.
>>
>> On Thu, Jul 2, 2020 at 12:48 AM Shivaram Venkataraman 
>>  wrote:
>>>
>>> Thanks Holden -- it would be great to also get 2.4.7 started
>>>
>>> Thanks
>>> Shivaram
>>>
>>> On Tue, Jun 30, 2020 at 10:31 PM Holden Karau  
>>> wrote:
>>> >
>>> > I can take care of 2.4.7 unless someone else wants to do it.
>>> >
>>> > On Tue, Jun 30, 2020 at 8:29 PM Jason Moore 
>>> >  wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >>
>>> >>
>>> >> Could I get some input on the severity of this one that I found 
>>> >> yesterday?  If that’s a correctness issue, should it block this 
>>> >> patch?  Let me know under the ticket if there’s more info that I can 
>>> >> provide to help.
>>> >>
>>> >>
>>> >>
>>> >> https://issues.apache.org/jira/browse/SPARK-32136
>>> >>
>>> >>
>>> >>
>>> >> Thanks,
>>> >>
>>> >> Jason.
>>> >>
>>> >>
>>> >>
>>> >> From: Jungtaek Lim 
>>> >> Date: Wednesday, 1 July 2020 at 10:20 am
>>> >> To: Shivaram Venkataraman 
>>> >> Cc: Prashant Sharma , 郑瑞峰 
>>> >> , Gengliang Wang 
>>> >> , gurwls223 , 
>>> >> Dongjoon Hyun , Jules Damji 
>>> >> , Holden Karau , Reynold 
>>> >> Xin , Yuanjian Li , 
>>> >> "dev@spark.apache.org" , Takeshi Yamamuro 
>>> >> 
>>> >> Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release
>>> >>
>>> >>
>>> >>
>>> >> SPARK-32130 [1] looks to be a performance regression introduced in 
>>> >> Spark 3.0.0, which is ideal to look into before releasing another 
>>> >> bugfix version.
>>> >>
>>> >>
>>> >>
>>> >> 1. https://issues.apache.org/jira/browse/SPARK-32130
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Jul 1, 2020 at 7:05 AM Shivaram Venkataraman 
>>> >>  wrote:
>>> >>
>>> >> Hi all
>>> >>
>>> >>
>>> >>
>>> >> I just wanted to ping this thread to see if all the outstanding 
>>> >> blockers for 3.0.1 have been fixed. If so, it would be great if we 
>>> >> can get the release going. The CRAN team sent us a note that the 
>>> >> version SparkR available on CRAN for the current R version (4.0.2) 
>>> >> is broken and hence we need to update the package soon --  it will 
>>> >> be great to do it with 3.0.1.
>>> >>
>>> >>
>>> >>
>>> >> Thanks
>>> >>
>>> >> Shivaram
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Jun 24, 2020 at 8:31 PM Prashant Sharma 
>>> >>  wrote:
>>> >>
>>> >> +1 for 3.0.1 release.
>>> >>
>>> >> I too can help out as release manager.
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jun 25, 2020 at 4:58 AM 郑瑞峰  wrote:
>>> >>
>>> >> I volunteer to be a release manager of 3.0.1, if nobody is working 
>>> >> on this.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> -- 原始邮件 --
>>> >>
>>> >> 发件人: "Gengliang Wang";
>>> >>
>>> >> 发送时间: 2020年6月24日(星期三) 下午4:15
>>> >>
>>> >> 收件人: "Hyukjin Kwon";
>>> >>
>>> >> 抄送: "Dongjoon Hyun";"Jungtaek 
>>> >> Lim";"Jules 
>>> >> Damji";"Holden 
>>> >> Karau";"Reynold 
>>> >> Xin";"Shivaram 
>>> >> Venkataraman";"Yuanjian 
>>> >> Li";"Spark dev 

Re: Welcoming some new Apache Spark committers

[Jungtaek Lim]

Thanks everyone for the warm welcome and all the kind words! Looking
forward to collaborate with you all.

On Wed, Jul 15, 2020 at 4:17 PM Herman van Hovell 
wrote:

> Congratulations!
>
> On Wed, Jul 15, 2020 at 9:00 AM angers.zhu  wrote:
>
>> Congratulations !
>>
>> angers.zhu
>> angers@gmail.com
>>
>> 
>> 签名由 网易邮箱大师  定制
>>
>> On 07/15/2020 14:53，Wenchen Fan
>>  wrote：
>>
>> Congrats and welcome!
>>
>> On Wed, Jul 15, 2020 at 2:18 PM Mridul Muralidharan 
>> wrote:
>>
>>>
>>> Congratulations !
>>>
>>> Regards,
>>> Mridul
>>>
>>> On Tue, Jul 14, 2020 at 12:37 PM Matei Zaharia 
>>> wrote:
>>>
 Hi all,

 The Spark PMC recently voted to add several new committers. Please join
 me in welcoming them to their new roles! The new committers are:

 - Huaxin Gao
 - Jungtaek Lim
 - Dilip Biswal

 All three of them contributed to Spark 3.0 and we’re excited to have
 them join the project.

 Matei and the Spark PMC
 -
 To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Welcoming some new Apache Spark committers

[Herman van Hovell]

Congratulations!

On Wed, Jul 15, 2020 at 9:00 AM angers.zhu  wrote:

> Congratulations !
>
> angers.zhu
> angers@gmail.com
>
> 
> 签名由 网易邮箱大师  定制
>
> On 07/15/2020 14:53，Wenchen Fan 
> wrote：
>
> Congrats and welcome!
>
> On Wed, Jul 15, 2020 at 2:18 PM Mridul Muralidharan 
> wrote:
>
>>
>> Congratulations !
>>
>> Regards,
>> Mridul
>>
>> On Tue, Jul 14, 2020 at 12:37 PM Matei Zaharia 
>> wrote:
>>
>>> Hi all,
>>>
>>> The Spark PMC recently voted to add several new committers. Please join
>>> me in welcoming them to their new roles! The new committers are:
>>>
>>> - Huaxin Gao
>>> - Jungtaek Lim
>>> - Dilip Biswal
>>>
>>> All three of them contributed to Spark 3.0 and we’re excited to have
>>> them join the project.
>>>
>>> Matei and the Spark PMC
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>

Re: [DISCUSS] Apache Spark 3.0.1 Release

[Wenchen Fan]

Yea I think 2.4.7 is good to go. Let's start!

On Wed, Jul 15, 2020 at 1:50 PM Prashant Sharma 
wrote:

> Hi Folks,
>
> So, I am back, and searched the JIRAS with target version as "2.4.7" and
> Resolved, found only 2 jiras. So, are we good to go, with just a couple of
> jiras fixed ? Shall I proceed with making a RC?
>
> Thanks,
> Prashant
>
> On Thu, Jul 2, 2020 at 5:23 PM Prashant Sharma 
> wrote:
>
>> Thank you, Holden.
>>
>> Folks, My health has gone down a bit. So, I will start working on this in
>> a few days. If this needs to be published sooner, then maybe someone else
>> has to help out.
>>
>>
>>
>>
>>
>> On Thu, Jul 2, 2020 at 10:11 AM Holden Karau 
>> wrote:
>>
>>> I’m happy to have Prashant do 2.4.7 :)
>>>
>>> On Wed, Jul 1, 2020 at 9:40 PM Xiao Li  wrote:
>>>
 +1 on releasing both 3.0.1 and 2.4.7

 Great! Three committers volunteer to be a release manager. Ruifeng,
 Prashant and Holden. Holden just helped release Spark 2.4.6. This time,
 maybe, Ruifeng and Prashant can be the release manager of 3.0.1 and 2.4.7
 respectively.

 Xiao

 On Wed, Jul 1, 2020 at 2:24 PM Jungtaek Lim <
 kabhwan.opensou...@gmail.com> wrote:

> https://issues.apache.org/jira/browse/SPARK-32148 was reported
> yesterday, and if the report is valid it looks to be a blocker. I'll try 
> to
> take a look sooner.
>
> On Thu, Jul 2, 2020 at 12:48 AM Shivaram Venkataraman <
> shiva...@eecs.berkeley.edu> wrote:
>
>> Thanks Holden -- it would be great to also get 2.4.7 started
>>
>> Thanks
>> Shivaram
>>
>> On Tue, Jun 30, 2020 at 10:31 PM Holden Karau 
>> wrote:
>> >
>> > I can take care of 2.4.7 unless someone else wants to do it.
>> >
>> > On Tue, Jun 30, 2020 at 8:29 PM Jason Moore <
>> jason.mo...@quantium.com.au> wrote:
>> >>
>> >> Hi all,
>> >>
>> >>
>> >>
>> >> Could I get some input on the severity of this one that I found
>> yesterday?  If that’s a correctness issue, should it block this patch?  
>> Let
>> me know under the ticket if there’s more info that I can provide to help.
>> >>
>> >>
>> >>
>> >> https://issues.apache.org/jira/browse/SPARK-32136
>> >>
>> >>
>> >>
>> >> Thanks,
>> >>
>> >> Jason.
>> >>
>> >>
>> >>
>> >> From: Jungtaek Lim 
>> >> Date: Wednesday, 1 July 2020 at 10:20 am
>> >> To: Shivaram Venkataraman 
>> >> Cc: Prashant Sharma , 郑瑞峰 <
>> ruife...@foxmail.com>, Gengliang Wang ,
>> gurwls223 , Dongjoon Hyun <
>> dongjoon.h...@gmail.com>, Jules Damji , Holden
>> Karau , Reynold Xin ,
>> Yuanjian Li , "dev@spark.apache.org" <
>> dev@spark.apache.org>, Takeshi Yamamuro 
>> >> Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release
>> >>
>> >>
>> >>
>> >> SPARK-32130 [1] looks to be a performance regression introduced in
>> Spark 3.0.0, which is ideal to look into before releasing another bugfix
>> version.
>> >>
>> >>
>> >>
>> >> 1. https://issues.apache.org/jira/browse/SPARK-32130
>> >>
>> >>
>> >>
>> >> On Wed, Jul 1, 2020 at 7:05 AM Shivaram Venkataraman <
>> shiva...@eecs.berkeley.edu> wrote:
>> >>
>> >> Hi all
>> >>
>> >>
>> >>
>> >> I just wanted to ping this thread to see if all the outstanding
>> blockers for 3.0.1 have been fixed. If so, it would be great if we can 
>> get
>> the release going. The CRAN team sent us a note that the version SparkR
>> available on CRAN for the current R version (4.0.2) is broken and hence 
>> we
>> need to update the package soon --  it will be great to do it with 3.0.1.
>> >>
>> >>
>> >>
>> >> Thanks
>> >>
>> >> Shivaram
>> >>
>> >>
>> >>
>> >> On Wed, Jun 24, 2020 at 8:31 PM Prashant Sharma <
>> scrapco...@gmail.com> wrote:
>> >>
>> >> +1 for 3.0.1 release.
>> >>
>> >> I too can help out as release manager.
>> >>
>> >>
>> >>
>> >> On Thu, Jun 25, 2020 at 4:58 AM 郑瑞峰  wrote:
>> >>
>> >> I volunteer to be a release manager of 3.0.1, if nobody is working
>> on this.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -- 原始邮件 --
>> >>
>> >> 发件人: "Gengliang Wang";
>> >>
>> >> 发送时间: 2020年6月24日(星期三) 下午4:15
>> >>
>> >> 收件人: "Hyukjin Kwon";
>> >>
>> >> 抄送: "Dongjoon Hyun";"Jungtaek Lim"<
>> kabhwan.opensou...@gmail.com>;"Jules Damji";"Holden
>> Karau";"Reynold Xin";"Shivaram
>> Venkataraman";"Yuanjian Li"<
>> xyliyuanj...@gmail.com>;"Spark dev list";"Takeshi
>> Yamamuro";
>> >>
>> >> 主题: Re: [DISCUSS] Apache Spark 3.0.1 Release
>> >>
>> >>
>> >>
>> >> +1, the issues mentioned are really serious.
>> >>
>> >>
>> >>
>> >> On Tue, Jun 23, 2020 at 7:56 PM Hyukjin Kwon 
>> wrote:
>> >>
>> 

Re: Welcoming some new Apache Spark committers

[angers . zhu]

Congratulations !






  










angers.zhu




angers@gmail.com








签名由
网易邮箱大师
定制

 


On 07/15/2020 14:53，Wenchen Fan wrote： 


Congrats and welcome!On Wed, Jul 15, 2020 at 2:18 PM Mridul Muralidharan  wrote:Congratulations !Regards,MridulOn Tue, Jul 14, 2020 at 12:37 PM Matei Zaharia  wrote:Hi all,

The Spark PMC recently voted to add several new committers. Please join me in welcoming them to their new roles! The new committers are:

- Huaxin Gao
- Jungtaek Lim
- Dilip Biswal

All three of them contributed to Spark 3.0 and we’re excited to have them join the project.

Matei and the Spark PMC
-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Welcoming some new Apache Spark committers

[Wenchen Fan]

Congrats and welcome!

On Wed, Jul 15, 2020 at 2:18 PM Mridul Muralidharan 
wrote:

>
> Congratulations !
>
> Regards,
> Mridul
>
> On Tue, Jul 14, 2020 at 12:37 PM Matei Zaharia 
> wrote:
>
>> Hi all,
>>
>> The Spark PMC recently voted to add several new committers. Please join
>> me in welcoming them to their new roles! The new committers are:
>>
>> - Huaxin Gao
>> - Jungtaek Lim
>> - Dilip Biswal
>>
>> All three of them contributed to Spark 3.0 and we’re excited to have them
>> join the project.
>>
>> Matei and the Spark PMC
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: Welcoming some new Apache Spark committers

[Mridul Muralidharan]

Congratulations !

Regards,
Mridul

On Tue, Jul 14, 2020 at 12:37 PM Matei Zaharia 
wrote:

> Hi all,
>
> The Spark PMC recently voted to add several new committers. Please join me
> in welcoming them to their new roles! The new committers are:
>
> - Huaxin Gao
> - Jungtaek Lim
> - Dilip Biswal
>
> All three of them contributed to Spark 3.0 and we’re excited to have them
> join the project.
>
> Matei and the Spark PMC
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>