Re: [VOTE] SPIP: Stored Procedures API for Catalogs

+1 On Mon, May 13, 2024 at 12:30 PM Zhou Jiang wrote: > +1 (non-binding) > > On Sat, May 11, 2024 at 2:10 PM L. C. Hsieh wrote: > >> Hi all, >> >> I’d like to start a vote for SPIP: Stored Procedures API for Catalogs. >> >> Please also refer to: >> >>- Discussion thread: >>

Re: [VOTE] SPARK-46122: Set spark.sql.legacy.createHiveTableByDefault to false

+1 On Fri, Apr 26, 2024 at 10:01 AM Dongjoon Hyun wrote: > I'll start with my +1. > > Dongjoon. > > On 2024/04/26 16:45:51 Dongjoon Hyun wrote: > > Please vote on SPARK-46122 to set > spark.sql.legacy.createHiveTableByDefault > > to `false` by default. The technical scope is defined in the

Re: [VOTE] Release Spark 3.4.3 (RC2)

+1 On Tue, Apr 16, 2024 at 11:57 AM L. C. Hsieh wrote: > +1 > > On Tue, Apr 16, 2024 at 4:08 AM Wenchen Fan wrote: > > > > +1 > > > > On Mon, Apr 15, 2024 at 12:31 PM Dongjoon Hyun > wrote: > >> > >> I'll start with my +1. > >> > >> - Checked checksum and signature > >> - Checked

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

+1 On Sat, Apr 13, 2024 at 3:26 PM Dongjoon Hyun wrote: > I'll start from my +1. > > Dongjoon. > > On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > > Please vote on SPARK-4 to use ANSI SQL mode by default. > > The technical scope is defined in the following PR which is > > one line of code

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

+1, enabling Spark's ANSI SQL mode in version 4.0 will significantly enhance data quality and integrity. I fully support this initiative. > In other words, the current Spark ANSI SQL implementation becomes the first implementation for Spark SQL users to face at first while providing

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

+1 On Sun, Mar 31, 2024 at 8:24 PM Dongjoon Hyun wrote: > +1 > > Thank you, Hyukjin. > > Dongjoon > > On Sun, Mar 31, 2024 at 19:07 Haejoon Lee > wrote: > >> +1 >> >> On Mon, Apr 1, 2024 at 10:15 AM Hyukjin Kwon >> wrote: >> >>> Hi all, >>> >>> I'd like to start the vote for SPIP: Pure Python

Re: Allowing Unicode Whitespace in Lexer

+1, this is a reasonable change. Gengliang On Wed, Mar 27, 2024 at 9:54 AM serge rielau.com wrote: > Going once, going twice, …. last call for objections > On Mar 23, 2024 at 5:29 PM -0700, serge rielau.com , > wrote: > > Hello, > > I have a PR https://github.com/apache/spark/pull/45620 ready

[VOTE][RESULT] SPIP: Structured Logging Framework for Apache Spark

(*) - Scott - Jungtaek Lim - Reynold Xin (*) - Holden Karau (*) - Xiao Li (*) - Chao Sun (*) - Liang-Chi Hsieh (*) - rhatlnux - Robyn Nameth - John Zhuge - Ruifeng Zheng (*) - Tom Graves (*) - Bo Yang +0: None -1: None Thanks, Gengliang Wang

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

ult is worth one-thousand >> expert opinions (Werner >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >> >> >> On Mon, 11 Mar 2024 at 09:27, Hyukjin Kwon wrote: >> >> +1

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

ote: >> >>> +1 >>> >>> On Mon, 11 Mar 2024 at 18:11, yangjie01 >>> wrote: >>> >>>> +1 >>>> >>>> >>>> >>>> Jie Yang >>>> >>>> >>>> >>>

[VOTE] SPIP: Structured Logging Framework for Apache Spark

p=sharing> - Discussion thread <https://lists.apache.org/thread/gocslhbfv1r84kbcq3xt04nx827ljpxq> Please vote on the SPIP for the next 72 hours: [ ] +1: Accept the proposal as an official SPIP [ ] +0 [ ] -1: I don’t think this is a good idea because … Thanks! Gengliang Wang

Re: [DISCUSS] SPIP: Structured Spark Logging

wikipedia.org/wiki/Wernher_von_Braun>Von > Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". > > > On Sat, 9 Mar 2024 at 18:10, Gengliang Wang wrote: > >> Hi Mich, >> >> Thanks for your suggestions. I agree that we should avoid confusion

Re: [DISCUSS] SPIP: Structured Spark Logging

The information provided is correct to the best of my > knowledge but of course cannot be guaranteed . It is essential to note > that, as with any advice, quote "one test result is worth one-thousand > expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von >

[DISCUSS] SPIP: Structured Spark Logging

Hi All, I propose to enhance our logging system by transitioning to structured logs. This initiative is designed to tackle the challenges of analyzing distributed logs from drivers, workers, and executors by allowing them to be queried using a fixed schema. The goal is to improve the

Re: Re: [DISCUSS] Release Spark 3.5.1?

+1 On Sun, Feb 4, 2024 at 1:57 PM Hussein Awala wrote: > +1 > > On Sun, Feb 4, 2024 at 10:13 PM John Zhuge wrote: > >> +1 >> >> John Zhuge >> >> >> On Sun, Feb 4, 2024 at 11:23 AM Santosh Pingale >> wrote: >> >>> +1 >>> >>> On Sun, Feb 4, 2024, 8:18 PM Xiao Li >>> wrote: >>> +1

Re: Algolia search on website is broken

Hi Nick, Thank you for reporting the issue with our web crawler. I've found that the issue was due to a change(specifically, pull request #40269 ) in the website's HTML structure, where the JavaScript selector ".container-wrapper" is now ".container".

Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript files

+1 On Sat, Nov 25, 2023 at 2:50 AM yangjie01 wrote: > +1 > > > > *发件人**: *Reynold Xin > *日期**: *2023年11月25日 星期六 14:35 > *收件人**: *Dongjoon Hyun > *抄送**: *Ye Zhou , Mridul Muralidharan < > mri...@gmail.com>, Kent Yao , dev > *主题**: *Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript

Re: Welcome to Our New Apache Spark Committer and PMCs

Congratulations to all! Well deserved! On Mon, Oct 2, 2023 at 10:16 PM Xiao Li wrote: > Hi all, > > The Spark PMC is delighted to announce that we have voted to add one new > committer and two new PMC members. These individuals have consistently > contributed to the project and have clearly

Re: [VOTE] Release Apache Spark 3.5.0 (RC5)

+1 On Mon, Sep 11, 2023 at 11:28 AM Xiao Li wrote: > +1 > > Xiao > > Yuanjian Li 于2023年9月11日周一 10:53写道： > >> @Peter Toth I've looked into the details of this >> issue, and it appears that it's neither a regression in version 3.5.0 nor a >> correctness issue. It's a bug related to a new

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

+1 On Wed, Sep 6, 2023 at 9:46 PM Yuanjian Li wrote: > +1 (non-binding) > > Xiao Li 于2023年9月6日周三 15:27写道： > >> +1 >> >> Xiao >> >> Herman van Hovell 于2023年9月6日周三 22:08写道： >> >>> Tested connect, and everything looks good. >>> >>> +1 >>> >>> On Wed, Sep 6, 2023 at 8:11 AM Yuanjian Li >>>

Re: Welcome two new Apache Spark committers

Congratulations! Peter and Xiduo! On Sun, Aug 6, 2023 at 7:37 PM Jungtaek Lim wrote: > Congrats Peter and Xiduo! > > On Mon, Aug 7, 2023 at 11:33 AM yangjie01 > wrote: > >> Congratulations, Peter and Xiduo ~ >> >> >> >> *发件人**: *Hyukjin Kwon >> *日期**: *2023年8月7日 星期一 10:30 >> *收件人**: *Ruifeng

Re: [Reminder] Spark 3.5 Branch Cut

Hi Yuanjian, Besides the abovementioned changes, it would be great to include the UI page for Spakr Connect: SPARK-44394 . Best Regards, Gengliang On Fri, Jul 14, 2023 at 11:44 AM Julek Sompolski wrote: > Thank you, > My changes that you

Introducing English SDK for Apache Spark - Seeking Your Feedback and Contributions

ser-friendly. Thank you in advance for your attention and involvement. We look forward to hearing your thoughts and seeing your contributions! Best, Gengliang Wang

Re: [VOTE] Release Spark 3.4.1 (RC1)

+1 On Thu, Jun 22, 2023 at 11:14 AM Driesprong, Fokko wrote: > Thank you for running the release Dongjoon > > +1 > > Tested against Iceberg and it looks good. > > > Op do 22 jun 2023 om 18:03 schreef yangjie01 : > >> +1 >> >> >> >> *发件人**: *Dongjoon Hyun >> *日期**: *2023年6月22日 星期四 23:35 >>

Re: [ANNOUNCE] Apache Spark 3.4.0 released

Congratulations everyone! Thank you Xinrong for driving the release! On Fri, Apr 14, 2023 at 12:47 PM Xinrong Meng wrote: > Hi All, > > We are happy to announce the availability of *Apache Spark 3.4.0*! > > Apache Spark 3.4.0 is the fifth release of the 3.x line. > > To download Spark 3.4.0,

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

+1 On Sun, Apr 9, 2023 at 3:17 PM Dongjoon Hyun wrote: > +1 > > I verified the same steps like previous RCs. > > Dongjoon. > > > On Sat, Apr 8, 2023 at 7:47 PM Mridul Muralidharan > wrote: > >> >> +1 >> >> Signatures, digests, etc check out fine. >> Checked out tag and build/tested with -Phive

Re: Apache Spark 3.2.4 EOL Release?

+1 On Wed, Apr 5, 2023 at 11:27 AM kazuyuki tanimura wrote: > +1 > > On Apr 5, 2023, at 6:53 AM, Tom Graves > wrote: > > +1 > > Tom > > On Tuesday, April 4, 2023 at 12:25:13 PM CDT, Dongjoon Hyun < > dongjoon.h...@gmail.com> wrote: > > > Hi, All. > > Since Apache Spark 3.2.0 passed RC7 vote on

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

Hi Anton, +1 for adding the old constructors back! Could you raise a PR for this? I will review it ASAP. Thanks Gengliang On Wed, Apr 5, 2023 at 9:37 AM Anton Okolnychyi wrote: > Sorry, I think my last message did not land on the list. > > I have a question about changes to exceptions used in

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

Thanks for creating the RC1, Xinrong! Besides the blockers mentioned by Tom, let's include the following bug fix in Spark 3.4.0 as well: [SPARK-42406][SQL] Fix check for missing required fields of to_protobuf

Re: Time for Spark 3.4.0 release?

+1, thanks for driving the release! Gengliang On Tue, Jan 3, 2023 at 10:55 PM Dongjoon Hyun wrote: > +1 > > Thank you! > > Dongjoon > > On Tue, Jan 3, 2023 at 9:44 PM Rui Wang wrote: > >> +1 to cut the branch starting from a workday! >> >> Great to see this is happening! >> >> Thanks Xinrong!

[VOTE][RESULT] SPIP: Better Spark UI scalability and Driver stability for large applications

The vote passes with 11 +1s(3 binding +1s) +1: Kent Yao Mridul Muralidharan* Jie Yang Yuming Wang Maciej Szymkiewicz* Chris Nauroth Jungtaek Lim Ye Zhou Wenchen Fan* Ruifeng Zheng Peter Toth 0: None -1: None (* = binding) Thank you all for chiming in and for your votes! Cheers, Gengliang

[VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications

Hi all, I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver stability for large applications" The goal of the SPIP is to improve the Driver's stability by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the read and write operations on RocksDB, it

Re: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications

With the positive feedback from Mridul and Wenchen, I will officially start the vote. On Tue, Nov 15, 2022 at 8:57 PM Wenchen Fan wrote: > This looks great! UI stability/scalability has been a pain point for a > long time. > > On Sat, Nov 12, 2022 at 5:24 AM Gengliang Wang wr

Re: Apache Spark 3.2.3 Release?

+1. Thanks Chao! On Tue, Oct 18, 2022 at 11:45 AM huaxin gao wrote: > +1 Thanks Chao! > > Huaxin > > On Tue, Oct 18, 2022 at 11:29 AM Dongjoon Hyun > wrote: > >> +1 >> >> Thank you for volunteering, Chao! >> >> Dongjoon. >> >> >> On Tue, Oct 18, 2022 at 9:55 AM Sean Owen wrote: >> >>> OK by

Re: [VOTE] Release Spark 3.3.1 (RC4)

+1 from me, same as last time. On Tue, Oct 18, 2022 at 11:45 AM L. C. Hsieh wrote: > +1 > > Thanks Yuming! > > On Tue, Oct 18, 2022 at 11:28 AM Dongjoon Hyun > wrote: > > > > +1 > > > > Thank you, Yuming and all! > > > > Dongjoon. > > > > > > On Tue, Oct 18, 2022 at 9:22 AM Yang,Jie(INF) >

Re: Welcome Yikun Jiang as a Spark committer

Congratulations, Yikun! On Sun, Oct 9, 2022 at 12:33 AM 416161...@qq.com wrote: > Congrats, Yikun! > > -- > Ruifeng Zheng > ruife...@foxmail.com > >

Re: [VOTE] Release Spark 3.3.1 (RC2)

+1. I ran some simple tests and also verified that SPARK-40389 is fixed. Gengliang On Mon, Oct 3, 2022 at 8:56 AM Thomas Graves wrote: > +1. ran out internal tests and everything looks good. > > Tom Graves > > On Wed, Sep 28, 2022 at 12:20 AM Yuming Wang wrote: > > > > Please vote on

Re: [VOTE] SPIP: Support Docker Official Image for Spark

+1 On Wed, Sep 21, 2022 at 7:26 PM Xiangrui Meng wrote: > +1 > > On Wed, Sep 21, 2022 at 6:53 PM Kent Yao wrote: > >> +1 >> >> *Kent Yao * >> @ Data Science Center, Hangzhou Research Institute, NetEase Corp. >> *a spark enthusiast* >> *kyuubi is a >> unified

Re: [DISCUSS] SPIP: Support Docker Official Image for Spark

+1, thanks for the work! On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon wrote: > +1 > > On Mon, 19 Sept 2022 at 09:15, Yikun Jiang wrote: > >> Hi, all >> >> I would like to start the discussion for supporting Docker Official Image >> for Spark. >> >> This SPIP is proposed to add Docker Official

Re: Time for Spark 3.3.1 release?

+1. Thank you, Yuming! On Mon, Sep 12, 2022 at 12:10 PM L. C. Hsieh wrote: > +1 > > Thanks Yuming! > > On Mon, Sep 12, 2022 at 11:50 AM Dongjoon Hyun > wrote: > > > > +1 > > > > Thanks, > > Dongjoon. > > > > On Mon, Sep 12, 2022 at 6:38 AM Yuming Wang wrote: > >> > >> Hi, All. > >> > >> > >>

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! Well deserved. On Tue, Aug 9, 2022 at 7:09 AM Yi Wu wrote: > Congrats Xinrong!! > > > On Tue, Aug 9, 2022 at 7:07 PM Maxim Gekk > wrote: > >> Congratulations, Xinrong! >> >> Maxim Gekk >> >> Software Engineer >> >> Databricks, Inc. >> >> >> On Tue, Aug 9, 2022 at

Re: [VOTE] Release Spark 3.2.2 (RC1)

Hi Bruce, FYI we had further discussions on https://github.com/apache/spark/pull/35313#issuecomment-1185195455. Thanks for pointing that out, but this document issue should not be a blocker of the release. +1 on the RC. Gengliang On Thu, Jul 14, 2022 at 10:22 PM sarutak wrote: > Hi Dongjoon

Re: Apache Spark 3.2.2 Release?

+1. Thank you, Dongjoon. On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan wrote: > +1 > > On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng > wrote: > >> +1 >> >> Thanks! >> >> >> Xinrong Meng >> >> Software Engineer >> >> Databricks >> >> >> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li wrote: >> >>> +1 >>>

Docker images for Spark 3.3.0 release are now available

Hi all, The official Docker images for Spark 3.3.0 release are now available! - To run Spark with Scala/Java API only: https://hub.docker.com/r/apache/spark - To run Python on Spark: https://hub.docker.com/r/apache/spark-py - To run R on Spark: https://hub.docker.com/r/apache/spark-r

Re: Re: [VOTE][SPIP] Spark Connect

+1 (non-binding) On Wed, Jun 15, 2022 at 9:32 AM Dongjoon Hyun wrote: > +1 > > On Wed, Jun 15, 2022 at 9:22 AM Xiao Li wrote: > >> +1 >> >> Xiao >> >> beliefer 于2022年6月14日周二 03:35写道： >> >>> +1 >>> Yeah, I tried to use Apache Livy, so as we can runing interactive query. >>> But the Spark

Re: Stickers and Swag

FYI now you can find the shopping information on https://spark.apache.org/community as well :) Gengliang > On Jun 14, 2022, at 7:47 PM, Hyukjin Kwon wrote: > > Woohoo > > On Tue, 14 Jun 2022 at 15:04, Xiao Li > wrote: >

Re: [VOTE] Release Spark 3.3.0 (RC6)

+1 (non-binding) On Mon, Jun 13, 2022 at 10:20 AM Herman van Hovell wrote: > +1 > > On Mon, Jun 13, 2022 at 12:53 PM Wenchen Fan wrote: > >> +1, tests are all green and there are no more blocker issues AFAIK. >> >> On Fri, Jun 10, 2022 at 12:27 PM Maxim Gekk >> wrote: >> >>> Please vote on

Re: [VOTE] Release Spark 3.3.0 (RC2)

Hi Kent and Wenchen, Thanks for reporting. I just created https://github.com/apache/spark/pull/36609 to fix the issue. Gengliang On Thu, May 19, 2022 at 5:40 PM Wenchen Fan wrote: > I think it should have been fixed by >

Re: SIGMOD System Award for Apache Spark

Congratulations to the whole spark community! On Fri, May 13, 2022 at 10:14 AM Jungtaek Lim wrote: > Congrats Spark community! > > On Fri, May 13, 2022 at 10:40 AM Qian Sun wrote: > >> Congratulations !!! >> >> 2022年5月13日 上午3:44，Matei Zaharia 写道： >> >> Hi all, >> >> We recently found out that

Re: [VOTE] Release Spark 3.3.0 (RC1)

Hi Maxim, Thanks for the work! There is a bug fix from Bruce merged on branch-3.3 right after the RC1 is cut: SPARK-39093: Dividing interval by integral can result in codegen compilation error So -1 from me. We

Re: Apache Spark 3.3 Release

I'd like to add the following new SQL functions in the 3.3 release. These functions are useful when overflow or encoding errors occur: - [SPARK-38548][SQL] New SQL function: try_sum - [SPARK-38589][SQL] New SQL function: try_avg

Re: [VOTE] Spark 3.1.3 RC4

+1 (non-binding) On Wed, Feb 16, 2022 at 1:28 PM Wenchen Fan wrote: > +1 > > On Tue, Feb 15, 2022 at 3:59 PM Yuming Wang wrote: > >> +1 (non-binding). >> >> On Tue, Feb 15, 2022 at 10:22 AM Ruifeng Zheng >> wrote: >> >>> +1 (non-binding) >>> >>> checked the release script issue Dongjoon

Re: [ANNOUNCE] Apache Spark 3.2.1 released

Thanks to Huaxin for driving the release! Fengyu, this is a known issue that will be fixed in the 3.3 release. Currently, the "hadoop3.2" means 3.2 or higher. See the thread https://lists.apache.org/thread/yov8xsggo3g2qr2p1rrr2xtps25wkbvj for more details. On Sat, Jan 29, 2022 at 3:26 PM

Re: [VOTE] Release Spark 3.2.1 (RC2)

+1 (non-binding) On Mon, Jan 24, 2022 at 6:26 PM Dongjoon Hyun wrote: > +1 > > Dongjoon. > > On Sat, Jan 22, 2022 at 7:19 AM Mridul Muralidharan > wrote: > >> >> +1 >> >> Signatures, digests, etc check out fine. >> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes >> >>

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thanks for the works, Shane! On Wed, Dec 8, 2021 at 9:19 AM shane knapp ☠ wrote: > created an issue to track stuff: > > https://issues.apache.org/jira/browse/SPARK-37571 > > On Tue, Dec 7, 2021 at 8:25 AM shane knapp ☠ wrote: > >> Will you be nuking all the Jenkins-related code in the repo

Re: Time for Spark 3.2.1?

+1 for new maintenance releases for all 3.x branches as well. On Wed, Dec 8, 2021 at 8:19 AM Hyukjin Kwon wrote: > SGTM! > > On Wed, 8 Dec 2021 at 09:07, huaxin gao wrote: > >> I prefer to start rolling the release in January if there is no need to >> publish it sooner :) >> >> On Tue, Dec 7,

Re: [VOTE] SPIP: Row-level operations in Data Source V2

+1 (non-binding) On Tue, Nov 16, 2021 at 9:03 PM Wenchen Fan wrote: > +1 > > On Mon, Nov 15, 2021 at 2:54 AM John Zhuge wrote: > >> +1 (non-binding) >> >> On Sun, Nov 14, 2021 at 10:33 AM Chao Sun wrote: >> >>> +1 (non-binding). Thanks Anton for the work! >>> >>> On Sun, Nov 14, 2021 at 10:01

Re: Update Spark 3.3 release window?

+1, Mid-March 2022 sounds good. Gengliang On Thu, Oct 28, 2021 at 10:54 PM Tom Graves wrote: > +1 for updating, mid march sounds good. I'm also fine with EOL 2.x. > > Tom > > On Thursday, October 28, 2021, 09:37:00 AM CDT, Mridul Muralidharan < > mri...@gmail.com> wrote: > > > > +1 to EOL 2.x

Re: [ANNOUNCE] Apache Spark 3.2.0

/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.3.tgz > > FYI, unable to download from this location. > Also, I don’t see Hadoop 3.3 version in the dist > > > On Oct 19, 2021, at 9:39 AM, Bode, Meikel, NMA-CFD < > meikel.b...@bertelsmann.de> wrote: > >  > > Man

[ANNOUNCE] Apache Spark 3.2.0

Hi all, Apache Spark 3.2.0 is the third release of the 3.x line. With tremendous contribution from the open-source community, this release managed to resolve in excess of 1,700 Jira tickets. We'd like to thank our contributors and users for their contributions and early feedback to this release.

Re: [VOTE][RESULT] Release Spark 3.2.0 (RC7)

> Yes. Genliang. Many thanks. > > > > *From:* Mich Talebzadeh > *Sent:* Dienstag, 12. Oktober 2021 09:25 > *To:* Gengliang Wang > *Cc:* dev > *Subject:* Re: [VOTE][RESULT] Release Spark 3.2.0 (RC7) > > > > great work Gengliang. Thanks for your tremendous contribu

[VOTE][RESULT] Release Spark 3.2.0 (RC7)

The vote passes with 28 +1s (10 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Gengliang Wang - Michael Heuer - Mridul Muralidharan * - Sean Owen * - Ruifeng Zheng - Dongjoon Hyun * - Yuming Wang - Reynold Xin * - Cheng Su - Peter Toth - Mich Talebzadeh - Maxim Gekk

Please take a look at the draft of the Spark 3.2.0 release notes

Hi all, I am preparing to publish and announce Spark 3.2.0 This is the draft of the release note, and I plan to edit a bit more and use it as the final release note. Please take a look and let me know if I missed any major changes or something.

Re: [VOTE] Release Spark 3.2.0 (RC7)

Starting with my +1(non-binding) Thanks, Gengliang On Thu, Oct 7, 2021 at 12:48 AM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time October 11 and passes if a > majori

[VOTE] Release Spark 3.2.0 (RC7)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time October 11 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC6)

t; > > > > >> PySpark smoke tests pass, I'm going to do a last pass through the > JIRAs > > >> before my vote though. > > >> > > >> On Wed, Sep 29, 2021 at 8:54 AM Sean Owen wrote: > > >> > > >>>

Re: [VOTE] Release Spark 3.2.0 (RC6)

Starting with my +1(non-binding) Thanks, Gengliang On Tue, Sep 28, 2021 at 11:45 PM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time September 30 and passes if a > majori

[VOTE] Release Spark 3.2.0 (RC6)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 30 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC5)

sting/spark-3.2.0/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:34: >>>>> package com.google.common.collect does not exist >>>>> ... >>>>> >>>>> I didn't see this in RC4, so, I wonder if a recent change affe

Re: [VOTE] Release Spark 3.2.0 (RC5)

Hi Kousuke, I tend to agree with Sean. It only affects the macOS developers when building Spark with the released Spark 3.2 code tarball without setting JAVA_HOME. I can mention this one as a known issue in the release note if this vote passes. Thanks, Gengliang On Mon, Sep 27, 2021 at 11:47 PM

Re: [VOTE] Release Spark 3.2.0 (RC5)

Starting with my +1(non-binding) Thanks, Gengliang On Mon, Sep 27, 2021 at 8:55 PM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time September 29 and passes if a > majori

[VOTE] Release Spark 3.2.0 (RC5)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 29 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC4)

ent-tabpanel#comment-17419285 > I think SPARK-35672 is a breaking change. > > Peter > > > On Thu, Sep 23, 2021 at 5:32 PM Yi Wu wrote: > >> +1 (non-binding) >> >> Thanks for the work, Gengliang! >> >> Bests, >> Yi >> >> On Th

Re: [VOTE] Release Spark 3.2.0 (RC4)

Starting with my +1(non-binding) Thanks, Gengliang On Thu, Sep 23, 2021 at 10:02 PM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time September 27 and passes if a > majori

[VOTE] Release Spark 3.2.0 (RC4)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 27 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC3)

et? I think Parquet currently >>>>>> uses Hadoop compression codec while Hadoop 2.7 still depends on native >>>>>> lib >>>>>> for the LZ4. Maybe we should run the test only for Hadoop 3.2 profile. >>>>>> &g

Re: [VOTE] Release Spark 3.2.0 (RC3)

this by changing my settings.xml file. >> >> Anyway, I can see this biting other people so I thought that I would >> mention it. >> >> Steve C >> >> On 19 Sep 2021, at 1:18 pm, Gengliang Wang wrote: >> >> Please vote on releasing the foll

Re: [VOTE] Release Spark 3.2.0 (RC3)

Starting with my +1(non-binding) Thanks, Gengliang On Sun, Sep 19, 2021 at 11:18 AM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time September 24 and passes if a > majori

[VOTE] Release Spark 3.2.0 (RC3)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 24 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC2)

/SPARK-36705> which will need to be > addressed. > > Regards, > Mridul > > > On Sun, Sep 5, 2021 at 8:47 AM Gengliang Wang wrote: > > Hi all, > > the voting fails. > Liang-Chi reported a new block SPARK-36669 > <https://issues.apache.org/jira/browse/SPA

Re: [VOTE] Release Spark 3.2.0 (RC2)

.13 support is experimental as of 3.2.0 anyway. > > > On Wed, Sep 1, 2021 at 2:08 AM Gengliang Wang wrote: > >> Please vote on releasing the following candidate as >> Apache Spark version 3.2.0. >> >> The vote is open until 11:59pm Pacific time September 3 and passes if a

Re: [VOTE] Release Spark 3.2.0 (RC2)

t - PARQUET-2078 <https://issues.apache.org/jira/browse/PARQUET-2078>: Failed to read parquet file after writing with the same parquet version if `spark.sql.hive.convertMetastoreParquet` is false - SPARK-36629 <https://issues.apache.org/jira/browse/SPARK-36629>: U

[VOTE] Release Spark 3.2.0 (RC2)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 3 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this

Re: [VOTE] Release Spark 3.2.0 (RC1)

2 -Pyarn -Phadoop-cloud -Phive-thriftserver >>>>>>>> -Phive-2.3 -Pscala-2.13 -Dhadoop.version=3.2.2 >>>>>>>> >>>>>>>> >>>>>>>> And then attempted to build my Java based spark application. >>>>>

Re: spark 3.2 release date

browse/SPARK-36619 <https://issues.apache.org/jira/browse/SPARK-36619> is resolved. Gengliang Wang > On Aug 31, 2021, at 12:06 PM, infa elance wrote: > > What is the expected ballpark release date of spark 3.2 ? > > Thanks and Regards, > Ajay.

Re: [VOTE] Release Spark 3.2.0 (RC1)

4:58 AM Yi Wu wrote: >> >>> -1. I found a bug (https://issues.apache.org/jira/browse/SPARK-36558) >>> in the push-based shuffle, which could lead to job hang. >>> >>> Bests, >>> Yi >>> >>> On Sat, Aug 21, 2021 at 1:05 AM G

Re: Add option to Spark UI to proxy to the executors?

Hi Holden, FYI there are already some related features in Spark: - Spark Master UI to reverse proxy Application and Workers UI - Support Spark UI behind front-end reverse proxy using a path prefix Revert proxy URL

Re: [VOTE] Release Spark 3.2.0 (RC1)

test to pass. > > Given the failure, and as the fix is already in the branch, will -1 the RC. > > Regards, > Mridul > > > On Fri, Aug 20, 2021 at 12:05 PM Gengliang Wang wrote: > >> Please vote on releasing the following candidate as Apache Spark version >>

Re: [VOTE] Release Spark 3.2.0 (RC1)

TF-8 > OS name: "mac os x", version: "11.5", arch: "x86_64", family: "mac" > > $ echo $MAVEN_OPTS > -Xmx8g -XX:ReservedCodeCacheSize=1g > > Pozdrawiam, > Jacek Laskowski > > https://about.me/JacekLaskowski > "The Interna

[VOTE] Release Spark 3.2.0 (RC1)

Please vote on releasing the following candidate as Apache Spark version 3.2 .0. The vote is open until 11:59pm Pacific time Aug 25 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this package

Re: Spark 3.2.0 first RC next week

e to push-based shuffle to improve > code robustness and performance, and is almost ready to be committed. > Because of the protocol change, it’s best to include it with 3.2.0 > release. > > Best, > Min > > On Tue, Aug 10, 2021 at 01:13 Gengliang Wang wrote: > >> Hi

Spark 3.2.0 first RC next week

Hi all, As of now, there are still some open/in-progress blockers for Spark 3.2.0 release: - Prohibit update mode in native support of session window (SPARK-36463 ) - Avoid inlining non-deterministic With-CTEs(SPARK-36447

Re: Apache Spark 3.2 Expectation

: > Thank you, Gengliang! > > On Wed, Jun 30, 2021 at 10:56 PM Gengliang Wang wrote: > >> Hi all, >> >> Just as a gentle reminder, I will do the branch cut tomorrow. Please >> focus on finalizing the works to land in Spark 3.2.0. >> After the branch cut, we

Re: Apache Spark 3.2 Expectation

t; >> GA period ideally we should focus on bug fixes and polishing. >> >> It would be great if we can speed up on these items in the list too. >> >> >> On Thu, 17 Jun 2021, 15:08 Gengliang Wang, wrote: >> >>> Thanks for the suggestions from Dongjoo

Re: [DISCUSS] Rename hadoop-3.2/hadoop-2.7 profile to hadoop-3/hadoop-2?

+1 for targeting the renaming for Apache Spark 3.3 at the current phase. On Fri, Jun 25, 2021 at 6:55 AM DB Tsai wrote: > +1 on renaming. > > DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > > On Jun 24, 2021, at 11:41 AM, Chao Sun wrote: > > Hi, > > As Spark master has upgraded

Re: [VOTE] Release Spark 3.0.3 (RC1)

+1 (non-binding) > On Jun 21, 2021, at 1:33 PM, Hyukjin Kwon wrote: > > +1 > > 2021년 6월 21일 (월) 오후 2:19, Dongjoon Hyun >님이 작성: > +1 > > Thank you, Yi. > > Bests, > Dongjoon. > > > On Sat, Jun 19, 2021 at 6:57 PM Yuming Wang >

Re: Apache Spark 3.2 Expectation

a >>>> soft >>>> > cut and the committers still are able to commit to `branch-3.3` >>>> according >>>> > to their decisions. >>>> > >>>> > Given that Apache Spark had 115 commits in a week in various areas >>&

Re: Apache Spark 3.2 Expectation

ra/browse/SPARK-34198> I wonder whether we should postpone the branch cut date. cc Min Shen, Yi Wu, Max Gekk, Huaxin Gao, Jungtaek Lim, Yuanjian Li, Liang-Chi Hsieh, who work on the projects above. On Tue, Jun 15, 2021 at 4:34 PM Hyukjin Kwon wrote: > +1, thanks. > > On Tue, 15 Ju

Re: Apache Spark 3.2 Expectation

Hi, As the expected release date is close, I would like to volunteer as the release manager for Apache Spark 3.2.0. Thanks, Gengliang On Mon, Apr 12, 2021 at 1:59 PM Wenchen Fan wrote: > An update: we found a mistake that we picked the Spark 3.2 release date > based on the scheduled release

Re: Apache Spark 3.0.3 Release?

+1, thanks Yi Gengliang Wang > On Jun 9, 2021, at 6:03 PM, 郑瑞峰 wrote: > > +1, thanks Yi

  1   2   >