Re: [FYI] SPARK-47993: Drop Python 3.8

+1 On Fri, Apr 26, 2024 at 10:26 AM Xinrong Meng wrote: > +1 > > On Thu, Apr 25, 2024 at 2:08 PM Holden Karau > wrote: > >> +1 >> >> Twitter: https://twitter.com/holdenkarau >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 >>

Re: [PySpark]: DataFrameWriterV2.overwrite fails with spark connect

error? > If not, create one. > > Thanks, > Toki Takahashi > > ----- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- Ruifeng Zheng E-mail: zrfli...@gmail.com

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

hours: >> >> [ ] +1: Accept the proposal as an official SPIP >> [ ] +0 >> [ ] -1: I don’t think this is a good idea because … >> >> Thanks. >> > -- Ruifeng Zheng E-mail: zrfli...@gmail.com

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

+1 On Wed, Mar 13, 2024 at 4:32 AM John Zhuge wrote: > +1 (non-binding) > > On Tue, Mar 12, 2024 at 8:45 AM L. C. Hsieh wrote: > >> +1 >> >> >> On Tue, Mar 12, 2024 at 8:20 AM Chao Sun wrote: >> >>> +1 >>> >>> On Tue, Mar 12, 2024 at 8:03 AM Xiao Li >>> wrote: >>> +1 On Tue,

Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript files

read/5rqrho4ldgmqlc173y2229pfll5sgkff >> <https://mailshield.baidu.com/check?q=A5eIk13PzbvR5UCVFABDK4GRUTDfo284IuNUsoBhT99%2fS%2boFEdCRAzrqN9WHLc6WlpnhXlOglydnrAZRonZnnNSbT%2fY%3d> >> [2] >> https://docs.google.com/document/d/1hWl5Q2CNNOjN5Ubyoa28XmpJtDyD9BtGtiEG2TT94rg/edit?usp=sharing >> <https://mailshield.baidu.com/check?q=%2blIPGMGbYRuMEyUGx9ZQr5J1dIG1UlQn%2foKhALCdvpTABcdfsCF%2feqphsUqfpMIo7PacgdBDy6l9QC%2bTgZsqyACtpv4nZolAb0la8ThaeT5qcuXbdAnaKqgLCfTm8MZMdthX2w%3d%3d> >> >> - >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> >> > -- Ruifeng Zheng E-mail: zrfli...@gmail.com

Re: [VOTE] SPIP: An Official Kubernetes Operator for Apache Spark

>> > >> Please also refer to: >>>>>>> > >> >>>>>>> > >>- Discussion thread: >>>>>>> > >> >>>>>>> https://lists.apache.org/thread/wdy7jfhf7m8jy74p6s0npjfd15ym5rxz >>>>>>> > >>- JIRA ticket: >>>>>>> https://issues.apache.org/jira/browse/SPARK-45923 >>>>>>> > >>- SPIP doc: >>>>>>> https://docs.google.com/document/d/1f5mm9VpSKeWC72Y9IiKN2jbBn32rHxjWKUfLRaGEcLE >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> Please vote on the SPIP for the next 72 hours: >>>>>>> > >> >>>>>>> > >> [ ] +1: Accept the proposal as an official SPIP >>>>>>> > >> [ ] +0 >>>>>>> > >> [ ] -1: I don’t think this is a good idea because … >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> Thank you! >>>>>>> > >> >>>>>>> > >> Liang-Chi Hsieh >>>>>>> > >> >>>>>>> > >> >>>>>>> - >>>>>>> > >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>>>> > >> >>>>>>> > > >>>>>>> > > >>>>>>> > > -- >>>>>>> > > >>>>>>> > > Zhou, Ye 周晔 >>>>>>> > >>>>>>> > >>>>>>> - >>>>>>> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>>>> > >>>>>>> >>>>>>> - >>>>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>>>> >>>>>>> >>>>>> -- Ruifeng Zheng E-mail: zrfli...@gmail.com

Re: [VOTE] Updating documentation hosted for EOL and maintenance releases

+1 On Tue, Sep 26, 2023 at 12:51 PM Hyukjin Kwon wrote: > Hi all, > > I would like to start the vote for updating documentation hosted for EOL > and maintenance releases to improve the usability here, and in order for > end users to read the proper and correct documentation. > > For discussion

Re: [ANNOUNCE] Apache Spark 3.5.0 released

view the release notes: >> https://spark.apache.org/releases/spark-release-3-5-0.html >> >> We would like to acknowledge all community members for contributing to >> this >> release. This release would not have been possible without you. >> >> Best, >> Yuanjian >> >> -- Ruifeng Zheng E-mail: zrfli...@gmail.com

Re: [VOTE] Release Apache Spark 3.5.0 (RC5)

+1 On Tue, Sep 12, 2023 at 7:24 AM Hyukjin Kwon wrote: > +1 > > On Tue, Sep 12, 2023 at 7:05 AM Xiao Li wrote: > >> +1 >> >> Xiao >> >> Yuanjian Li 于2023年9月11日周一 10:53写道： >> >>> @Peter Toth I've looked into the details of this >>> issue, and it appears that it's neither a regression in

Re: Welcome two new Apache Spark committers

Congratulations! Peter and Xiduo! On Mon, Aug 7, 2023 at 10:13 AM Xiao Li wrote: > Congratulations, Peter and Xiduo! > > > > Debasish Das 于2023年8月6日周日 19:08写道： > >> Congratulations Peter and Xidou. >> >> On Sun, Aug 6, 2023, 7:05 PM Wenchen Fan wrote: >> >>> Hi all, >>> >>> The Spark PMC

Re: LLM script for error message improvement

+1 from my side, I'm fine to have it as a helper script On Thu, Aug 3, 2023 at 10:53 AM Hyukjin Kwon wrote: > I think adding that dev tool script to improve the error message is fine. > > On Thu, 3 Aug 2023 at 10:24, Haejoon Lee > wrote: > >> Dear contributors, I hope you are doing well! >> >>

Re: Time for Spark 3.3.3 release?

+1, thank you Yuming On Tue, Aug 1, 2023 at 10:40 AM Yuming Wang wrote: > Thank you. I will prepare 3.3.3-rc1 soon. > > On Sun, Jul 30, 2023 at 12:15 AM Dongjoon Hyun > wrote: > >> +1 >> >> Thank you for volunteering, Yuming. >> >> Dongjoon >> >> >> On Fri, Jul 28, 2023 at 11:35 AM Yuming Wang

Re: Spark Docker Official Image is now available

for Spark: >> https://lists.apache.org/thread/l1793y5224n8bqkp3s6ltgkykso4htb3 >> [2] [VOTE] SPIP: Support Docker Official Image for Spark: >> https://lists.apache.org/thread/ro6olodm1jzdffwjx4oc7ol7oh6kshbl >> [3] https://github.com/docker-library/official-images/pull/130

Re: [VOTE][SPIP] Python Data Source API

+1 On Mon, Jul 10, 2023 at 8:20 AM Jungtaek Lim wrote: > +1 > > On Sat, Jul 8, 2023 at 4:13 AM Reynold Xin > wrote: > >> +1! >> >> >> On Fri, Jul 7 2023 at 11:58 AM, Holden Karau >> wrote: >> >>> +1 >>> >>> On Fri, Jul 7, 2023 at 9:55 AM huaxin gao >>> wrote: >>> +1 On Fri,

Re: [VOTE][SPIP] PySpark Test Framework

+1 On Thu, Jun 22, 2023 at 1:11 PM Dongjoon Hyun wrote: > +1 > > Dongjoon > > On Wed, Jun 21, 2023 at 8:56 PM Hyukjin Kwon wrote: > >> +1 >> >> On Thu, 22 Jun 2023 at 02:20, Jacek Laskowski wrote: >> >>> +0 >>> >>> Pozdrawiam, >>> Jacek Laskowski >>> >>> "The Internals Of" Online Books

Re: [VOTE] Release Spark 3.4.1 (RC1)

+1 On Wed, Jun 21, 2023 at 2:26 PM huaxin gao wrote: > +1 > > On Tue, Jun 20, 2023 at 11:21 PM Hyukjin Kwon > wrote: > >> +1 >> >> On Wed, 21 Jun 2023 at 14:23, yangjie01 wrote: >> >>> +1 >>> >>> >>> 在 2023/6/21 13:20，“L. C. Hsieh”>> vii...@gmail.com>> 写入: >>> >>> >>> +1 >>> >>> >>> On Tue,

Re: [DISCUSS] SPIP: Add PySpark Test Framework

+1 from my side sounds good, it will be helpful to both users and contributors to improve the test coverage On Wed, Jun 14, 2023 at 8:27 AM Hyukjin Kwon wrote: > Yeah, I have been thinking about this too, and Holden did some work here > that this SPIP will reuse. I support this. > > On Wed, 14

Re: Apache Spark 3.4.1 Release?

+1 Thank you Dongjoon! On Fri, Jun 9, 2023 at 11:54 PM Xiao Li wrote: > +1 > > On Fri, Jun 9, 2023 at 08:30 Wenchen Fan wrote: > >> +1 >> >> On Fri, Jun 9, 2023 at 8:52 PM Xinrong Meng wrote: >> >>> +1. Thank you Doonjoon! >>> >>> Thanks, >>> >>> Xinrong Meng >>> >>> Mridul Muralidharan

Re: [VOTE] Release Apache Spark 3.2.4 (RC1)

+1 (non-binding) Thank you for driving this release! RuifengZheng ruife...@foxmail.com --Original-- From: "Yuming Wang"

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

+1 (non-binding) RuifengZheng ruife...@foxmail.com --Original-- From: "Kent Yao"

Re: Apache Spark 3.2.2 Release?

+1 thank you Dongjoon! RuifengZheng ruife...@foxmail.com --Original-- From: "Yikun Jiang"

?????? [VOTE] Release Spark 3.3.0 (RC6)

+1 (non-binding) Maxim, thank you for driving this release! thanks, ruifeng ---- ??: "Chao Sun"

?????? [VOTE][SPIP] Spark Connect

+1 ---- ??: "huaxin gao"

回复：Introducing "Pandas API on Spark" component in JIRA, and use "PS" PR title component

+1, I think it is a good idea --原始邮件-- 发件人: "Hyukjin Kwon"

回复： [VOTE] Spark 3.1.3 RC4

+1 (non-binding) checked the release script issue Dongjoon mentioned: curl -s https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc4-bin/spark-3.1.3-bin-hadoop2.7.tgz | tar tz | grep hadoop-common spark-3.1.3-bin-hadoop2.7/jars/hadoop-common-2.7.4

回复：[ANNOUNCE] Apache Spark 3.2.1 released

It's Great! Congrats and thanks, huaxin! -- 原始邮件 -- 发件人: "huaxin gao"

回复： [VOTE] Release Spark 3.2.1 (RC2)

+1 (non-binding) --原始邮件-- 发件人: "Kent Yao"

Re: [VOTE] Release Spark 3.2.1 (RC1)

+1 (non-binding) Thanks, ruifeng zheng --Original-- From: "Che

Re: [Core][Suggestion] sortWithinPartitions and aggregateWithinPartitions for RDD

Do you mean in-memory processing? It works fine if all partitions are small. But when some partition don’t fit in memory, it will cause OOM. 发件人: Reynold Xin <r...@databricks.com> 日期: 2018年2月1日 星期四 下午3:14 收件人: Ruifeng Zheng <ruife...@foxmail.com> 抄送: <dev@spark.apache.org

[Core][Suggestion] sortWithinPartitions and aggregateWithinPartitions for RDD

HI all: 1, Dataset API supports operation “sortWithinPartitions”, but in RDD API there is no counterpart (I know there is “repartitionAndSortWithinPartitions”, but I don’t want to repartition the RDD), I have to convert RDD to Dataset for this function. Would it make sense to add a