答复: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Pan,Bingkun
Okay, Let me double-check it carefully. Thank you very much for your help! 发件人: Jungtaek Lim 发送时间: 2024年3月5日 21:56:41 收件人: Pan,Bingkun 抄送: Dongjoon Hyun; dev; user 主题: Re: [ANNOUNCE] Apache Spark 3.5.1 released Yeah the approach seems OK to me - please double

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Jungtaek Lim
024年3月5日 17:09:07 > *收件人:* Pan,Bingkun > *抄送:* Dongjoon Hyun; dev; user > *主题:* Re: [ANNOUNCE] Apache Spark 3.5.1 released > > Let me be more specific. > > We have two active release version lines, 3.4.x and 3.5.x. We just > released Spark 3.5.1, having a dropdown as 3.5.1 and

答复: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Pan,Bingkun
:07 收件人: Pan,Bingkun 抄送: Dongjoon Hyun; dev; user 主题: Re: [ANNOUNCE] Apache Spark 3.5.1 released Let me be more specific. We have two active release version lines, 3.4.x and 3.5.x. We just released Spark 3.5.1, having a dropdown as 3.5.1 and 3.4.2 given the fact the last version of 3.4.x is 3.4.2

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Jungtaek Lim
Let me be more specific. We have two active release version lines, 3.4.x and 3.5.x. We just released Spark 3.5.1, having a dropdown as 3.5.1 and 3.4.2 given the fact the last version of 3.4.x is 3.4.2. After a month we released Spark 3.4.3. In the dropdown of Spark 3.4.3, there will be 3.5.1

答复: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Pan,Bingkun
; dev; user 主题: Re: [ANNOUNCE] Apache Spark 3.5.1 released But this does not answer my question about updating the dropdown for the doc of "already released versions", right? Let's say we just released version D, and the dropdown has version A, B, C. We have another release tomorrow as

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Jungtaek Lim
https://github.com/apache/spark/pull/42881 > > So, we need to manually update this file. I can manually submit an update > first to get this feature working. > -- > *发件人:* Jungtaek Lim > *发送时间:* 2024年3月4日 6:34:42 > *收件人:* Dongjoon Hyun > *抄送:* dev;

答复: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-05 Thread Pan,Bingkun
收件人: Dongjoon Hyun 抄送: dev; user 主题: Re: [ANNOUNCE] Apache Spark 3.5.1 released Shall we revisit this functionality? The API doc is built with individual versions, and for each individual version we depend on other released versions. This does not seem to be right to me. Also, the functionality

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-04 Thread Yang Jie
Date: Feb 24, 2024 Version: master > >>> > >>> [image: Screenshot 2024-02-29 at 21.12.24.png] > >>> > >>> > >>> Could you do the follow-up, please? > >>> > >>> Thank you in advance. > >>> > >>> Dongjoon

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-04 Thread yangjie01
That sounds like a great suggestion. 发件人: Jungtaek Lim 日期: 2024年3月5日 星期二 10:46 收件人: Hyukjin Kwon 抄送: yangjie01 , Dongjoon Hyun , dev , user 主题: Re: [ANNOUNCE] Apache Spark 3.5.1 released Yes, it's relevant to that PR. I wonder, if we want to expose version switcher, it should

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-04 Thread Jungtaek Lim
.html#pyspark-overview> >>>> >>>>Date: Feb 24, 2024 Version: master >>>> >>>> [image: Screenshot 2024-02-29 at 21.12.24.png] >>>> >>>> >>>> Could you do the follow-up, please? >>>> >>>> Than

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-04 Thread Hyukjin Kwon
Could you do the follow-up, please? >>> >>> Thank you in advance. >>> >>> Dongjoon. >>> >>> >>> On Thu, Feb 29, 2024 at 2:48 PM John Zhuge wrote: >>> >>>> Excellent work, congratulations! >>>>

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-03-03 Thread Jungtaek Lim
; Dongjoon. >> >> >> On Thu, Feb 29, 2024 at 2:48 PM John Zhuge wrote: >> >>> Excellent work, congratulations! >>> >>> On Wed, Feb 28, 2024 at 10:12 PM Dongjoon Hyun >>> wrote: >>> >>>> Congratulations! >>>&g

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread Peter Toth
;> Dongjoon. >> >> On Wed, Feb 28, 2024 at 11:43 AM beliefer wrote: >> >>> Congratulations! >>> >>> >>> >>> At 2024-02-28 17:43:25, "Jungtaek Lim" >>> wrote: >>> >>> Hi everyone, >>> >

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread Jungtaek Lim
, Feb 28, 2024 at 10:12 PM Dongjoon Hyun >> wrote: >> >>> Congratulations! >>> >>> Bests, >>> Dongjoon. >>> >>> On Wed, Feb 28, 2024 at 11:43 AM beliefer wrote: >>> >>>> Congratulations! >>>> >>>&

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread Dongjoon Hyun
ongratulations! >> >> Bests, >> Dongjoon. >> >> On Wed, Feb 28, 2024 at 11:43 AM beliefer wrote: >> >>> Congratulations! >>> >>> >>> >>> At 2024-02-28 17:43:25, "Jungtaek Lim" >>> wrote: >

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread John Zhuge
uot;Jungtaek Lim" >> wrote: >> >> Hi everyone, >> >> We are happy to announce the availability of Spark 3.5.1! >> >> Spark 3.5.1 is a maintenance release containing stability fixes. This >> release is based on the branch-3.5 maintenance branch o

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread Prem Sahoo
.@163.com> wrote:Congratulations!At 2024-02-28 17:43:25, "Jungtaek Lim" <kabhwan.opensou...@gmail.com> wrote:Hi everyone,We are happy to announce the availability of Spark 3.5.1!Spark 3.5.1 is a maintenance release containing stability fixes. Thisrelease is based on the branch-3.5 mai

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-29 Thread Xinrong Meng
uot;Jungtaek Lim" >> wrote: >> >> Hi everyone, >> >> We are happy to announce the availability of Spark 3.5.1! >> >> Spark 3.5.1 is a maintenance release containing stability fixes. This >> release is based on the branch-3.5 maintenance branch o

Re: [ANNOUNCE] Apache Spark 3.5.1 released

2024-02-28 Thread Dongjoon Hyun
Congratulations! Bests, Dongjoon. On Wed, Feb 28, 2024 at 11:43 AM beliefer wrote: > Congratulations! > > > > At 2024-02-28 17:43:25, "Jungtaek Lim" > wrote: > > Hi everyone, > > We are happy to announce the availability of Spark 3.5.1! > > S

Re:[ANNOUNCE] Apache Spark 3.5.1 released

2024-02-28 Thread beliefer
Congratulations! At 2024-02-28 17:43:25, "Jungtaek Lim" wrote: Hi everyone, We are happy to announce the availability of Spark 3.5.1! Spark 3.5.1 is a maintenance release containing stability fixes. This release is based on the branch-3.5 maintenance branch of Spark. W

[ANNOUNCE] Apache Spark 3.5.1 released

2024-02-28 Thread Jungtaek Lim
Hi everyone, We are happy to announce the availability of Spark 3.5.1! Spark 3.5.1 is a maintenance release containing stability fixes. This release is based on the branch-3.5 maintenance branch of Spark. We strongly recommend all 3.5 users to upgrade to this stable release. To download Spark

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-23 Thread Jungtaek Lim
) and I accidentally rebuilt the doc with the branch which I used for debugging issue in RC. I'll rebuild the doc from tag and submit a PR again. On Sat, Feb 24, 2024 at 7:16 AM Dongjoon Hyun wrote: > Hi, All. > > Unfortunately, the Apache Spark `3.5.1 RC2` document artifact seems to be &g

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-23 Thread Dongjoon Hyun
Hi, All. Unfortunately, the Apache Spark `3.5.1 RC2` document artifact seems to be generated from unknown source code instead of the correct source code of the tag, `3.5.1`. https://spark.apache.org/docs/3.5.1/ [image: Screenshot 2024-02-23 at 14.13.07.png] Dongjoon. On Wed, Feb 21, 2024

[VOTE][RESULT] Release Apache Spark 3.5.1 (RC2)

2024-02-21 Thread Jungtaek Lim
The vote passes with 6 +1s (4 binding +1s). Thanks to all who helped with the release! (* = binding) +1: Jungtaek Lim Wenchen Fan (*) Cheng Pan Xiao Li (*) Hyukjin Kwon (*) Maxim Gekk (*) +0: None -1: None

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-21 Thread Jungtaek Lim
o I didn't >>> advocate for SPARK-45357 to be backported to branch-3.5. >>> > As far as I know, the condition to trigger this test failure is: when >>> using Maven to test the `connect` module, if `sparkTestRelation` in >>> `SparkConnectProtoSuite` is not the first `DataFr

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-20 Thread Maxim Gekk
longer be 0. So, I think this >> is indeed related to the order in which Maven executes the test cases in >> the `connect` module. >> > I have submitted a backport PR to branch-3.5, and if necessary, we can >> merge it to fix this test issue. >> > Jie Yang >&

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-20 Thread Hyukjin Kwon
zed, > then the `id` of `sparkTestRelation` will no longer be 0. So, I think this > is indeed related to the order in which Maven executes the test cases in > the `connect` module. > > I have submitted a backport PR to branch-3.5, and if necessary, we can > merge it to fix this

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-20 Thread Xiao Li
hen the `id` of `sparkTestRelation` will no longer be 0. So, I think this > is indeed related to the order in which Maven executes the test cases in > the `connect` module. > > I have submitted a backport PR to branch-3.5, and if necessary, we can > merge it to fix this test iss

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-20 Thread Cheng Pan
necessary, we can merge > it to fix this test issue. > Jie Yang > 发件人: Jungtaek Lim > 日期: 2024年2月16日 星期五 22:15 > 收件人: Sean Owen , Rui Wang > 抄送: dev > 主题: Re: [VOTE] Release Apache Spark 3.5.1 (RC2) > I traced back relevant changes and got a sense of what happen

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-19 Thread Wenchen Fan
backport PR >> <https://github.com/apache/spark/pull/45141> to branch-3.5, and if >> necessary, we can merge it to fix this test issue. >> >> >> >> Jie Yang >> >> >> >> *发件人**: *Jungtaek Lim >> *日期**: *2024年2月16日 星期五 22:15 >&g

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-19 Thread Jungtaek Lim
he `connect` module. >> >> >> >> I have submitted a backport PR >> <https://github.com/apache/spark/pull/45141> to branch-3.5, and if >> necessary, we can merge it to fix this test issue. >> >> >> >> Jie Yang >> >> &

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-16 Thread Sean Owen
; > *发件人**: *Jungtaek Lim > *日期**: *2024年2月16日 星期五 22:15 > *收件人**: *Sean Owen , Rui Wang > *抄送**: *dev > *主题**: *Re: [VOTE] Release Apache Spark 3.5.1 (RC2) > > > > I traced back relevant changes and got a sense

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-16 Thread yangjie01
年2月16日 星期五 22:15 收件人: Sean Owen , Rui Wang 抄送: dev 主题: Re: [VOTE] Release Apache Spark 3.5.1 (RC2) I traced back relevant changes and got a sense of what happened. Yangjie figured out the issue via link<https://mailshield.baidu.com/check?q=8dOSfwXDF

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-16 Thread Jungtaek Lim
vote is open until February 18th 9AM (PST) and passes if a majority >> +1 PMC votes are cast, with >> a minimum of 3 +1 votes. >> >> [ ] +1 Release this package as Apache Spark 3.5.1 >> [ ] -1 Do not release this package because ... >> >> To learn mor

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-15 Thread Sean Owen
Jungtaek Lim wrote: > DISCLAIMER: RC for Apache Spark 3.5.1 starts with RC2 as I lately figured > out doc generation issue after tagging RC1. > > Please vote on releasing the following candidate as Apache Spark version > 3.5.1. > > The vote is open until February 18th

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-15 Thread Jungtaek Lim
UPDATE: The vote thread is up now. https://lists.apache.org/thread/f28h0brncmkoyv5mtsqtxx38hx309c2j On Tue, Feb 6, 2024 at 11:30 AM Jungtaek Lim wrote: > Thanks all for the positive feedback! Will figure out time to go through > the RC process. Stay tuned! > > On Mon, Feb 5, 2024 at 7:46 AM

Re: Heads-up: Update on Spark 3.5.1 RC

2024-02-15 Thread Jungtaek Lim
UPDATE: Now the vote thread is up for RC2. https://lists.apache.org/thread/f28h0brncmkoyv5mtsqtxx38hx309c2j On Wed, Feb 14, 2024 at 2:59 AM Dongjoon Hyun wrote: > Thank you for the update, Jungtaek. > > Dongjoon. > > On Tue, Feb 13, 2024 at 7:29 AM Jungtaek Lim > wrote: > >> Hi, >> >> Just a

[VOTE] Release Apache Spark 3.5.1 (RC2)

2024-02-15 Thread Jungtaek Lim
DISCLAIMER: RC for Apache Spark 3.5.1 starts with RC2 as I lately figured out doc generation issue after tagging RC1. Please vote on releasing the following candidate as Apache Spark version 3.5.1. The vote is open until February 18th 9AM (PST) and passes if a majority +1 PMC votes are cast

Re: Heads-up: Update on Spark 3.5.1 RC

2024-02-13 Thread Dongjoon Hyun
Thank you for the update, Jungtaek. Dongjoon. On Tue, Feb 13, 2024 at 7:29 AM Jungtaek Lim wrote: > Hi, > > Just a head-up since I didn't give an update for a week after the last > update from the discussion thread. > > I've been following the automated release process and encountered several

Heads-up: Update on Spark 3.5.1 RC

2024-02-13 Thread Jungtaek Lim
Hi, Just a head-up since I didn't give an update for a week after the last update from the discussion thread. I've been following the automated release process and encountered several issues. Maybe I will file JIRA tickets and follow PRs. Issues I figured out so far are 1) python library

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-05 Thread Jungtaek Lim
Thanks all for the positive feedback! Will figure out time to go through the RC process. Stay tuned! On Mon, Feb 5, 2024 at 7:46 AM Gengliang Wang wrote: > +1 > > On Sun, Feb 4, 2024 at 1:57 PM Hussein Awala wrote: > >> +1 >> >> On Sun, Feb 4, 2024 at 10:13 PM John Zhuge wrote: >> >>> +1 >>>

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread Gengliang Wang
+1 On Sun, Feb 4, 2024 at 1:57 PM Hussein Awala wrote: > +1 > > On Sun, Feb 4, 2024 at 10:13 PM John Zhuge wrote: > >> +1 >> >> John Zhuge >> >> >> On Sun, Feb 4, 2024 at 11:23 AM Santosh Pingale >> wrote: >> >>> +1 >>> >>> On Sun, Feb 4, 2024, 8:18 PM Xiao Li >>> wrote: >>> +1

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread Hussein Awala
+1 On Sun, Feb 4, 2024 at 10:13 PM John Zhuge wrote: > +1 > > John Zhuge > > > On Sun, Feb 4, 2024 at 11:23 AM Santosh Pingale > wrote: > >> +1 >> >> On Sun, Feb 4, 2024, 8:18 PM Xiao Li >> wrote: >> >>> +1 >>> >>> On Sun, Feb 4, 2024 at 6:07 AM beliefer wrote: >>> +1

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread John Zhuge
+1 John Zhuge On Sun, Feb 4, 2024 at 11:23 AM Santosh Pingale wrote: > +1 > > On Sun, Feb 4, 2024, 8:18 PM Xiao Li > wrote: > >> +1 >> >> On Sun, Feb 4, 2024 at 6:07 AM beliefer wrote: >> >>> +1 >>> >>> >>> >>> 在 2024-02-04 15:26:13,"Dongjoon Hyun" 写道: >>> >>> +1 >>> >>> On Sat, Feb 3,

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread Santosh Pingale
+1 On Sun, Feb 4, 2024, 8:18 PM Xiao Li wrote: > +1 > > On Sun, Feb 4, 2024 at 6:07 AM beliefer wrote: > >> +1 >> >> >> >> 在 2024-02-04 15:26:13,"Dongjoon Hyun" 写道: >> >> +1 >> >> On Sat, Feb 3, 2024 at 9:18 PM yangjie01 >> wrote: >> >>> +1 >>> >>> 在 2024/2/4 13:13,“Kent

Re: Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread Xiao Li
+1 On Sun, Feb 4, 2024 at 6:07 AM beliefer wrote: > +1 > > > > 在 2024-02-04 15:26:13,"Dongjoon Hyun" 写道: > > +1 > > On Sat, Feb 3, 2024 at 9:18 PM yangjie01 > wrote: > >> +1 >> >> 在 2024/2/4 13:13,“Kent Yao”mailto:y...@apache.org>> 写入: >> >> >> +1 >> >> >> Jungtaek Lim >

Re:Re: [DISCUSS] Release Spark 3.5.1?

2024-02-04 Thread beliefer
+1 在 2024-02-04 15:26:13,"Dongjoon Hyun" 写道: +1 On Sat, Feb 3, 2024 at 9:18 PM yangjie01 wrote: +1 在 2024/2/4 13:13,“Kent Yao”mailto:y...@apache.org>> 写入: +1 Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> 于2024年2月3日周六 21:14写道: > > Hi dev, > > looks like there are a huge

Re: [DISCUSS] Release Spark 3.5.1?

2024-02-03 Thread Dongjoon Hyun
+1 On Sat, Feb 3, 2024 at 9:18 PM yangjie01 wrote: > +1 > > 在 2024/2/4 13:13,“Kent Yao”mailto:y...@apache.org>> 写入: > > > +1 > > > Jungtaek Lim kabhwan.opensou...@gmail.com>> 于2024年2月3日周六 21:14写道: > > > > Hi dev, > > > > looks like there are a huge number of commits being pushed to branch-3.5

Re: [DISCUSS] Release Spark 3.5.1?

2024-02-03 Thread yangjie01
+1 在 2024/2/4 13:13,“Kent Yao”mailto:y...@apache.org>> 写入: +1 Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> 于2024年2月3日周六 21:14写道: > > Hi dev, > > looks like there are a huge number of commits being pushed to branch-3.5 > after 3.5.0 was released, 200+ commits. > > $ git log --oneline

Re: [DISCUSS] Release Spark 3.5.1?

2024-02-03 Thread Kent Yao
+1 Jungtaek Lim 于2024年2月3日周六 21:14写道: > > Hi dev, > > looks like there are a huge number of commits being pushed to branch-3.5 > after 3.5.0 was released, 200+ commits. > > $ git log --oneline v3.5.0..HEAD | wc -l > 202 > > Also, there are 180 JIRA tickets containing 3.5.1 as fixed version, and

[DISCUSS] Release Spark 3.5.1?

2024-02-03 Thread Jungtaek Lim
Hi dev, looks like there are a huge number of commits being pushed to branch-3.5 after 3.5.0 was released, 200+ commits. $ git log --oneline v3.5.0..HEAD | wc -l 202 Also, there are 180 JIRA tickets containing 3.5.1 as fixed version, and 10 resolved issues are either marked as blocker (even

Re: Spark 3.5.1

2024-01-31 Thread Jungtaek Lim
Hi, I agreed it's time to release 3.5.1. 10 resolved issues are either marked as blocker (even correctness issues) or critical, which justifies the release. I had been trying to find the time to take a step, but had no luck with it. I'll give it another try this week (it needs some time as I'm

Spark 3.5.1

2024-01-30 Thread Santosh Pingale
Hey there Spark 3.5 branch has accumulated 199 commits with quite a few bug fixes related to correctness. Are there any plans for releasing 3.5.1? Kind regards Santosh