Re: Welcoming some new Apache Spark committers

wrote: >>>> >>>>> Welcome, Huaxin, Jungtaek, and Dilip! >>>>> >>>>> Congratulations! >>>>> >>>>> On Tue, Jul 14, 2020 at 10:37 AM Matei Zaharia < >>>>> matei.zaha...@gmail.com> wrote: >>

Re: [vote] Apache Spark 3.0 RC3

+1 (non-binding)     - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [DISCUSS] SPIP: Row-level operations in Data Source V2

I took a quick look at the PR and it looks like a great feature to have. It provides unified APIs for data sources to perform the commonly used operations easily and efficiently, so users don't have to implement customer extensions on their own. Thanks Anton for the work! On Thu, Jun 24, 2021 at

Re: Welcoming six new Apache Spark committers

Congratulations to you all!! On Fri, Mar 26, 2021 at 4:22 PM Yuming Wang wrote: > Congrats! > > On Sat, Mar 27, 2021 at 7:13 AM Takeshi Yamamuro > wrote: > >> Congrats, all~ >> >> On Sat, Mar 27, 2021 at 7:46 AM Jungtaek Lim < >> kabhwan.opensou...@gmail.com> wrote: >> >>> Congrats all! >>>

Re: [Discuss][SPIP] DataSource V2 SQL push down

I5NGYzYWMzMWYwNDliOWIwM2ZkODllODk4Njk2NzEiLCJwIjoiYyJ9 >> >> This SPIP aims to make pushdown more extendable. >> >> I would like to thank huaxin gao, my prototype is based on her PR. I will >> submit a PR ASAP >> >> Thanks >> >> Chang. >> >

Re: [VOTE] SPIP: Add FunctionCatalog

+1 (non-binding) On Tue, Mar 9, 2021 at 1:12 AM Kent Yao wrote: > +1, looks great! > > *Kent Yao * > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > *a spark enthusiast* > *kyuubi is a unified multi-tenant JDBC > interface for large-scale

Re: Apache Spark 3.2 Expectation

Thanks Dongjoon and Xiao for the discussion. I would like to add Data Source V2 Aggregate push down to the list. I am currently working on JDBC Data Source V2 Aggregate push down, but the common code can be used for the file based V2 Data Source as well. For example, MAX and MIN can be pushed down

Re: [Discuss][SPIP] DataSource V2 SQL push down

File Based Source and SQL Based Source are quite different on >> push down capabilities. I am not sure they can be consolidated into one API. >> >> I will push my PR tomorrow, and after that, could we schedule a meeting >> to discuss the API? >> >> huaxin gao 于2021年4

Re: [Discuss][SPIP] DataSource V2 SQL push down

the file source > and SQL source. > > What's the time difference between Beijing and your timezone? I prefer > next Monday night or Tuesday morning. > > I can share zoom. > > huaxin gao 于2021年4月8日周四 上午7:10写道： > >> Hi Chang, >> >> Thanks for working on

Re: [VOTE] Release Spark 3.2.0 (RC7)

+1 (non-binding) On Fri, Oct 8, 2021 at 8:27 AM Xinli shang wrote: > +1 (non-binding) > > > On Fri, Oct 8, 2021 at 7:59 AM Chao Sun wrote: > >> +1 (non-binding) >> >> On Fri, Oct 8, 2021 at 1:01 AM Maxim Gekk >> wrote: >> >>> +1 (non-binding) >>> >>> On Fri, Oct 8, 2021 at 10:44 AM Mich

Re: [DISCUSS] SPIP: Storage Partitioned Join for Data Source V2

+1. Thanks for lifting the current restrictions on bucket join and making this more generalized. On Sun, Oct 24, 2021 at 9:33 AM Ryan Blue wrote: > +1 from me as well. Thanks Chao for doing so much to get it to this point! > > On Sat, Oct 23, 2021 at 11:29 PM DB Tsai wrote: > >> +1 on this

Re: Time for Spark 3.2.1?

release, and we have resolved many >> bug fixes and regressions. What do you guys think about rolling Spark 3.2.1 >> release? >> >> cc @huaxin gao FYI who I happened to overhear >> that is interested in rolling the maintenance release :-). >> >

Re: [VOTE] SPIP: Row-level operations in Data Source V2

+1 On Fri, Nov 12, 2021 at 6:44 PM Yufei Gu wrote: > +1 > > > On Nov 12, 2021, at 6:25 PM, L. C. Hsieh wrote: > > > > Hi all, > > > > I’d like to start a vote for SPIP: Row-level operations in Data Source > V2. > > > > The proposal is to add support for executing row-level operations > > such

Re: [VOTE] SPIP: Storage Partitioned Join for Data Source V2

+1 On Fri, Oct 29, 2021 at 10:59 AM Dongjoon Hyun wrote: > +1 > > Dongjoon > > On 2021/10/29 17:48:59, Russell Spitzer > wrote: > > +1 This is a great idea, (I have no Apache Spark voting points) > > > > On Fri, Oct 29, 2021 at 12:41 PM L. C. Hsieh wrote: > > > > > > > > I'll start with my

Re: Time for Spark 3.2.1?

se around next January? I would > leave it to @huaxin gao :-). > > On Wed, 8 Dec 2021 at 06:19, Dongjoon Hyun > wrote: > >> +1 for new releases. >> >> Dongjoon. >> >> On Mon, Dec 6, 2021 at 8:51 PM Wenchen Fan wrote: >> >>> +1 to make

Re: Time for Spark 3.2.1?

or new maintenance releases for all 3.x branches as well. >>> >>> On Wed, Dec 8, 2021 at 8:19 AM Hyukjin Kwon wrote: >>> >>>> SGTM! >>>> >>>> On Wed, 8 Dec 2021 at 09:07, huaxin gao wrote: >>>> >>>>> I pre

Re: [VOTE] Release Spark 3.2.1 (RC1)

The two regressions have been fixed. I will cut RC2 tomorrow late afternoon. Thanks, Huaxin On Wed, Jan 12, 2022 at 9:11 AM huaxin gao wrote: > Thank you all for testing and voting! > > I will -1 this RC because > https://issues.apache.org/jira/browse/SPARK-37855

Re: [VOTE] Release Spark 3.2.1 (RC1)

> +1 (non-binding) >>> >>> Thanks, ruifeng zheng >>> >>> -- Original -- >>> *From:* "Cheng Su" ; >>> *Date:* Wed, Jan 12, 2022 02:54 PM >>> *To:* "Qian Sun";"huaxin gao"< >>>

[VOTE] Release Spark 3.2.1 (RC1)

Please vote on releasing the following candidate as Apache Spark version 3.2.1. The vote is open until Jan. 13th at 12 PM PST (8 PM UTC) and passes if a majority +1 PMC votes are cast, with a minimum of 3 + 1 votes. [ ] +1 Release this package as Apache Spark 3.2.1 [ ] -1 Do not release this

Re: [VOTE] SPIP: Catalog API for view metadata

+1 (non-binding) On Fri, Feb 4, 2022 at 11:40 AM L. C. Hsieh wrote: > +1 > > On Thu, Feb 3, 2022 at 7:25 PM Chao Sun wrote: > > > > +1 (non-binding). Looking forward to this feature! > > > > On Thu, Feb 3, 2022 at 2:32 PM Ryan Blue wrote: > >> > >> +1 for the SPIP. I think it's well designed

[ANNOUNCE] Apache Spark 3.2.1 released

over to the download page: https://spark.apache.org/downloads.html To view the release notes: https://spark.apache.org/releases/spark-release-3-2-1.html We would like to acknowledge all community members for contributing to this release. This release would not have been possible without you. Huaxin

Re: [VOTE] Release Spark 3.2.1 (RC1)

che.spark.serializer.KryoSerializer") \ >> .set("spark.sql.repl.eagerEval.maxNumRows", "1") >> >> return >> SparkSession.builder.appName(app_name).config(conf=conf).getOrCreate() >> >> spark = get_spark_session("Falk&qu

[VOTE] Release Spark 3.2.1 (RC2)

Please vote on releasing the following candidate as Apache Spark version 3.2.1. The vote is open until 8:00pm Pacific time January 25 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.1[ ] -1 Do not release this package

[VOTE][RESULT] Release Spark 3.2.1 (RC2)

The vote passes with 13 +1s (4 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Sean Owen * - Mridul Muralidharan * - Dongjoon Hyun * - Gengliang Wang - Michael Heuer - Chao Sun - Cheng Su - John Zhuge - Kent Yao - Ruifeng Zheng - XiDuo You - Wenchen Fan * - Yuming

Re: Welcome to Our New Apache Spark Committer and PMCs

Congratulations! On Wed, Oct 4, 2023 at 7:39 AM Chao Sun wrote: > Congratulations! > > On Wed, Oct 4, 2023 at 5:11 AM Jungtaek Lim > wrote: > >> Congrats! >> >> 2023년 10월 4일 (수) 오후 5:04, yangjie01 님이 작성: >> >>> Congratulations! >>> >>> >>> >>> Jie Yang >>> >>> >>> >>> *发件人**: *Dongjoon Hyun

Re: [DISCUSSION] SPIP: An Official Kubernetes Operator for Apache Spark

+1 On Thu, Nov 9, 2023 at 3:14 PM DB Tsai wrote: > +1 > > To be completely transparent, I am employed in the same department as Zhou > at Apple. > > I support this proposal, provided that we witness community adoption > following the release of the Flink Kubernetes operator, streamlining Flink

Re: [VOTE] Release Spark 3.3.0 (RC5)

Thanks Dongjoon for opening a jira to track this issue. I agree this is a flaky test. I have seen the flakiness in our internal tests. I also agree this is a non-blocker because the feature is disabled by default. I will try to take a look to see if I can find the root cause. Thanks, Huaxin On

Re: [VOTE] Release Spark 3.3.0 (RC5)

I agree with Prashant, -1 from me too because this may break iceberg usage. Thanks, Huaxin On Wed, Jun 8, 2022 at 10:07 AM Prashant Singh wrote: > -1 from my side as well, found this today. > > While testing Apache iceberg with 3.3 found this bug where a table with > partitions with null

Re: [VOTE][SPIP] Spark Connect

+1 On Mon, Jun 13, 2022 at 5:42 PM L. C. Hsieh wrote: > +1 > > On Mon, Jun 13, 2022 at 5:41 PM Chao Sun wrote: > > > > +1 (non-binding) > > > > On Mon, Jun 13, 2022 at 5:11 PM Hyukjin Kwon > wrote: > >> > >> +1 > >> > >> On Tue, 14 Jun 2022 at 08:50, Yuming Wang wrote: > >>> > >>> +1. > >>>

Re: 回复： [VOTE] Release Spark 3.3.0 (RC6)

+1 (non-binding) On Mon, Jun 13, 2022 at 10:47 PM Kent Yao wrote: > +1, non-binding > > Xiao Li 于2022年6月14日周二 13:11写道： > > > > +1 > > > > Xiao > > > > beliefer 于2022年6月13日周一 20:04写道： > >> > >> +1 AFAIK, no blocking issues now. > >> Glad to hear to release 3.3.0 ! > >> > >> > >> 在 2022-06-14

Re: Apache Spark 3.2.3 Release?

+1 Thanks Chao! Huaxin On Tue, Oct 18, 2022 at 11:29 AM Dongjoon Hyun wrote: > +1 > > Thank you for volunteering, Chao! > > Dongjoon. > > > On Tue, Oct 18, 2022 at 9:55 AM Sean Owen wrote: > >> OK by me, if someone is willing to drive it. >> >> On Tue, Oct 18, 2022 at 11:47 AM Chao Sun

Re: Welcome Yikun Jiang as a Spark committer

Congratulations! On Fri, Oct 7, 2022 at 11:22 PM Yang,Jie(INF) wrote: > Congratulations Yikun! > > Regards, > Yang Jie > -- > *发件人:* Mridul Muralidharan > *发送时间:* 2022年10月8日 14:16:02 > *收件人:* Yuming Wang > *抄送:* Hyukjin Kwon; dev; Yikun Jiang > *主题:* Re: Welcome

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! On Tue, Aug 9, 2022 at 12:47 PM Dongjoon Hyun wrote: > Congrat! :) > > Dongjoon. > > On Tue, Aug 9, 2022 at 10:40 AM Takuya UESHIN > wrote: > > > > Congratulations, Xinrong! > > > > On Tue, Aug 9, 2022 at 10:07 AM Gengliang Wang wrote: > >> > >> Congratulations, Xinrong! Well

Re: Time for Spark 3.4.0 release?

+1 Thanks! On Wed, Jan 4, 2023 at 10:19 AM L. C. Hsieh wrote: > +1 > > Thank you! > > On Wed, Jan 4, 2023 at 9:13 AM Chao Sun wrote: > >> +1, thanks! >> >> Chao >> >> On Wed, Jan 4, 2023 at 1:56 AM Mridul Muralidharan >> wrote: >> >>> >>> +1, Thanks ! >>> >>> Regards, >>> Mridul >>> >>> On

Re: [ANNOUNCE] Apache Spark 3.2.3 released

Thanks Chao for driving the release! On Wed, Nov 30, 2022 at 9:24 AM Dongjoon Hyun wrote: > Thank you, Chao! > > On Wed, Nov 30, 2022 at 8:16 AM Yang,Jie(INF) wrote: > >> Thanks, Chao! >> >> >> >> *发件人**: *Maxim Gekk >> *日期**: *2022年11月30日 星期三 19:40 >> *收件人**: *Jungtaek Lim >> *抄送**:

Re: Time for release v3.3.2

+1 Thanks Liang-Chi! On Mon, Jan 30, 2023 at 6:01 PM Dongjoon Hyun wrote: > +1 > > Thank you so much, Liang-Chi. > 3.3.2 release will help 3.4.0 release too because they share many bug > fixes. > > Dongjoon > > > On Mon, Jan 30, 2023 at 5:56 PM Hyukjin Kwon wrote: > >> +100! >> >> On Tue, 31

Re: [VOTE] Release Spark 3.2.3 (RC1)

+1 Thanks Chao! On Mon, Nov 14, 2022 at 9:37 PM L. C. Hsieh wrote: > +1 > > Thanks Chao. > > On Mon, Nov 14, 2022 at 6:55 PM Dongjoon Hyun > wrote: > > > > +1 > > > > Thank you, Chao. > > > > On Mon, Nov 14, 2022 at 4:12 PM Chao Sun wrote: > >> > >> Please vote on releasing the following

Re: Apache Spark 3.2.4 EOL Release?

+1 On Tue, Apr 4, 2023 at 11:17 AM Chao Sun wrote: > +1 > > On Tue, Apr 4, 2023 at 11:12 AM Holden Karau wrote: > >> +1 >> >> On Tue, Apr 4, 2023 at 11:04 AM L. C. Hsieh wrote: >> >>> +1 >>> >>> Sounds good and thanks Dongjoon for driving this. >>> >>> On 2023/04/04 17:24:54 Dongjoon Hyun

Re: [VOTE] Release Apache Spark 3.2.4 (RC1)

+1 On Mon, Apr 10, 2023 at 8:17 AM Chao Sun wrote: > +1 (non-binding) > > On Mon, Apr 10, 2023 at 7:07 AM yangjie01 wrote: > >> +1 (non-binding) >> >> >> >> *发件人**: *Sean Owen >> *日期**: *2023年4月10日 星期一 21:19 >> *收件人**: *Dongjoon Hyun >> *抄送**: *"dev@spark.apache.org" >> *主题**: *Re: [VOTE]

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

+1 On Mon, Apr 10, 2023 at 8:18 AM Chao Sun wrote: > +1 (non-binding) > > On Mon, Apr 10, 2023 at 12:41 AM Ruifeng Zheng > wrote: > >> +1 (non-binding) >> >> -- >> Ruifeng Zheng >> ruife...@foxmail.com >> >>

Re: [VOTE][SPIP] Lazy Materialization for Parquet Read Performance Improvement

+1 On Mon, Feb 13, 2023 at 3:09 PM Dongjoon Hyun wrote: > +1 > > Dongjoon > > On 2023/02/13 22:52:59 "L. C. Hsieh" wrote: > > Hi all, > > > > I'd like to start the vote for SPIP: Lazy Materialization for Parquet > > Read Performance Improvement. > > > > The high summary of the SPIP is that it

Re: [DISCUSS] SPIP: Lazy Materialization for Parquet Read Performance Improvement

+1 On Tue, Jan 31, 2023 at 6:10 PM DB Tsai wrote: > +1 > > Sent from my iPhone > > On Jan 31, 2023, at 4:16 PM, Yuming Wang wrote: > >  > +1. > > On Wed, Feb 1, 2023 at 7:42 AM kazuyuki tanimura > wrote: > >> Great! Much appreciated, Mitch! >> >> Kazu >> >> On Jan 31, 2023, at 3:07 PM, Mich

Re: [VOTE][SPIP] Python Data Source API

+1 On Fri, Jul 7, 2023 at 8:59 AM Mich Talebzadeh wrote: > +1 for me > > Mich Talebzadeh, > Solutions Architect/Engineering Lead > Palantir Technologies Limited > London > United Kingdom > > >view my Linkedin profile > > > >

Re: Apache Spark 3.4.1 Release?

+1 On Thu, Jun 8, 2023 at 2:25 PM Dongjoon Hyun wrote: > Hi, All. > > `branch-3.4` already has 77 commits since v3.4.0 tag. > > https://github.com/apache/spark/releases/v3.4.0 (Tagged on April 6th) > > $ git log --oneline v3.4.0..HEAD | wc -l > 77 > > I'd like to propose to have

Re: Welcome two new Apache Spark committers

Congratulations! Peter and Xiduo! On Mon, Aug 7, 2023 at 9:40 AM Dongjoon Hyun wrote: > Congratulations, Peter and Xiduo. :) > > Dongjoon. > > On Sun, Aug 6, 2023 at 10:08 PM XiDuo You wrote: > >> Thank you all ! >> >> Jia Fan 于2023年8月7日周一 11:31写道： >> > >> > Congratulations! >> >

Re: [VOTE] Release Spark 3.4.1 (RC1)

+1 On Tue, Jun 20, 2023 at 11:21 PM Hyukjin Kwon wrote: > +1 > > On Wed, 21 Jun 2023 at 14:23, yangjie01 wrote: > >> +1 >> >> >> 在 2023/6/21 13:20，“L. C. Hsieh”> vii...@gmail.com>> 写入: >> >> >> +1 >> >> >> On Tue, Jun 20, 2023 at 8:48 PM Dongjoon Hyun > > wrote: >>

Re: [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)

+1 On Mon, Jun 12, 2023 at 11:05 AM Dongjoon Hyun wrote: > +1 > > Dongjoon > > On 2023/06/12 18:00:38 Dongjoon Hyun wrote: > > Please vote on the release plan for Apache Spark 4.0.0. > > > > The vote is open until June 16th 1AM (PST) and passes if a majority +1 > PMC > > votes are cast, with a

Re: [VOTE] SPIP: An Official Kubernetes Operator for Apache Spark

+1 On Tue, Nov 14, 2023 at 10:45 AM Holden Karau wrote: > +1 > > On Tue, Nov 14, 2023 at 10:21 AM DB Tsai wrote: > >> +1 >> >> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 >> >> On Nov 14, 2023, at 10:14 AM, Vakaris Baškirov < >> vakaris.bashki...@gmail.com> wrote: >> >> +1

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

+1 On Mon, Mar 11, 2024 at 7:02 AM Wenchen Fan wrote: > +1 > > On Mon, Mar 11, 2024 at 5:26 PM Hyukjin Kwon wrote: > >> +1 >> >> On Mon, 11 Mar 2024 at 18:11, yangjie01 >> wrote: >> >>> +1 >>> >>> >>> >>> Jie Yang >>> >>> >>> >>> *发件人**: *Haejoon Lee >>> *日期**: *2024年3月11日 星期一 17:09 >>>

Re: [VOTE] Add new `Versions` in Apache Spark JIRA for Versioning of Spark Operator

+1 On Fri, Apr 12, 2024 at 9:07 AM Dongjoon Hyun wrote: > +1 > > Thank you! > > I hope we can customize `dev/merge_spark_pr.py` script per repository > after this PR. > > Dongjoon. > > On 2024/04/12 03:28:36 "L. C. Hsieh" wrote: > > Hi all, > > > > Thanks for all discussions in the thread of

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

+1 On Thu, Apr 11, 2024 at 11:18 PM L. C. Hsieh wrote: > +1 > > I believe ANSI mode is well developed after many releases. No doubt it > could be used. > Since it is very easy to disable it to restore to current behavior, I > guess the impact could be limited. > Do we have known the possible

Re: [VOTE] Release Spark 3.4.3 (RC2)

+1 On Tue, Apr 16, 2024 at 6:55 PM Kent Yao wrote: > +1(non-binding) > > Thanks, > Kent Yao > > bo yang 于2024年4月17日周三 09:49写道： > > > > +1 > > > > On Tue, Apr 16, 2024 at 1:38 PM Hyukjin Kwon > wrote: > >> > >> +1 > >> > >> On Wed, Apr 17, 2024 at 3:57 AM L. C. Hsieh wrote: > >>> > >>> +1 >

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

+1 On Sat, Apr 13, 2024 at 4:36 PM L. C. Hsieh wrote: > +1 > > On Sat, Apr 13, 2024 at 4:12 PM Hyukjin Kwon wrote: > > > > +1 > > > > On Sun, Apr 14, 2024 at 7:46 AM Chao Sun wrote: > >> > >> +1. > >> > >> This feature is very helpful for guarding against correctness issues, > such as null