Re: [ANNOUNCE] New Celeborn PMC Member: Mridul Muralidharan

d welcome, Mridul! > > > > > > > > > > > > > > > > Regards, > > > > > > > > Nicholas Jiang > > > > > > > > > > > > > > > > > > > > At 2024-09-20 09:30:19, "rexxiong" wrote: > >

[jira] [Assigned] (SPARK-49502) Avoid NPE in SparkEnv.get.shuffleManager.unregisterShuffle

[ https://issues.apache.org/jira/browse/SPARK-49502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-49502: --- Assignee: dzcxzl > Avoid NPE

Re: [VOTE] Move Variant to Parquet

Hi, What was the conclusions of discussions with Parquet and Iceberg communities on this ? Thanks, Mridul On Mon, Sep 2, 2024 at 12:48 PM Gene Pang wrote: > Hi all, > > I’d like to start a vote for moving the Variant specification and library > to the Parquet project. This allows the Variant

[jira] [Resolved] (SPARK-49217) Support separate buffer size configuration in UnsafeShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-49217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-49217. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-49217) Support separate buffer size configuration in UnsafeShuffleWriter

[ https://issues.apache.org/jira/browse/SPARK-49217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-49217: --- Assignee: dzcxzl > Support separate buffer size configuration

[jira] [Assigned] (SPARK-49268) Log IO exceptions in SHS history provider

[ https://issues.apache.org/jira/browse/SPARK-49268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-49268: --- Assignee: Rob Reeves > Log IO exceptions in SHS history provi

[jira] [Resolved] (SPARK-49268) Log IO exceptions in SHS history provider

[ https://issues.apache.org/jira/browse/SPARK-49268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-49268. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

Re: [VOTE] Deprecate SparkR

+1 Regards, Mridul On Wed, Aug 21, 2024 at 11:46 AM Reynold Xin wrote: > +1 > > On Wed, Aug 21, 2024 at 6:42 PM Shivaram Venkataraman < > shivaram.venkatara...@gmail.com> wrote: > >> Hi all >> >> Based on the previous discussion thread [1], I hereby call a vote to >> deprecate the SparkR modu

Re: Welcome new Apache Spark committers

Congratulations ! Regards, Mridul On Mon, Aug 12, 2024 at 7:27 PM Dongjoon Hyun wrote: > Congratulations, Martin, Haejoon, Allison. :) > > Dongjoon > > On Mon, Aug 12, 2024 at 5:19 PM Hyukjin Kwon wrote: > >> Hi all, >> >> The Spark PMC recently voted to add three new committers. Please join m

Re: Welcoming a new PMC member

Congratulations Kent ! Regards, Mridul On Mon, Aug 12, 2024 at 8:46 PM Dongjoon Hyun wrote: > Congratulations, Kent. > > Dongjoon. > > On Mon, Aug 12, 2024 at 5:22 PM Xiao Li wrote: > >> Congratulations ! >> >> Hyukjin Kwon 于2024年8月12日周一 17:20写道： >> >>> Hi all, >>> >>> The Spark PMC recently

Re: [DISCUSS] Deprecating SparkR

+1 Regards, Mridul On Mon, Aug 12, 2024 at 6:18 PM Hyukjin Kwon wrote: > +1 > > On Tue, Aug 13, 2024 at 7:04 AM Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> And just for the record, the stats that I screenshotted >>

Re: ASF board report for August 2024

Looks good to me, thanks Matei. Regards, Mridul On Sat, Aug 10, 2024 at 7:02 PM Matei Zaharia wrote: > Hi all, > > It’s time to submit our board report soon on August 14th. Here’s a draft, > feel free to leave any comments: > > = > > Description: > > Apache Spark is a fa

Re: [DRAFT] Celeborn Board Report

Looks good to me, thanks Keyong ! Regards, Mridul On Sun, Aug 11, 2024 at 9:59 PM Keyong Zhou wrote: > Hi community, > > The board report is due on August 14th, following is the draft I made, any > comments > will be appreciated, thanks! > > ## Description: > The mission of Apache Celeborn is t

Re: [VOTE] CIP-10: Introduce Celeborn Chaos Testing Framework

+1 This will be very useful for hardening Celeborn as we evolve it ! Regards, Mridul On Fri, Aug 2, 2024 at 9:37 AM Nicholas Jiang wrote: > Hi all, > > Thanks for all the feedback about the CIP-10: Introduce Celeborn Chaos > Testing Framework[1]. The discussion thread is here [2]. > > I'd lik

Re: [外部邮件] [VOTE] Release Spark 3.5.2 (RC2)

+1 Signatures, digests, etc check out fine. Checked out tag and build/tested with -Phive -Pyarn -Pkubernetes Regards, Mridul On Tue, Jul 23, 2024 at 9:51 PM Kent Yao wrote: > +1(non-binding), I have checked: > > - Download links are OK > - Signatures, Checksums, and the KEYS file are OK > - L

[jira] [Updated] (RATIS-2131) Configuring Ratis fails when hostname is used, and is an IPv6 host

[ https://issues.apache.org/jira/browse/RATIS-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated RATIS-2131: --- Affects Version/s: 3.1.0 > Configuring Ratis fails when hostname is used, and is

[jira] [Created] (RATIS-2131) Configuring Ratis fails when hostname is used, and is an IPv6 host

Mridul Muralidharan created RATIS-2131: -- Summary: Configuring Ratis fails when hostname is used, and is an IPv6 host Key: RATIS-2131 URL: https://issues.apache.org/jira/browse/RATIS-2131 Project

[jira] [Resolved] (SPARK-48928) Log Warning for Calling .unpersist() on Locally Checkpointed RDDs

[ https://issues.apache.org/jira/browse/SPARK-48928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-48928. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

Re: [ANNOUNCE] New Celeborn PMC Member: Nicholas Jiang

Congratulations ! Regards, Mridul On Tue, Jul 23, 2024 at 12:37 PM Fei Wang wrote: > Congrats! > > Regards, > Fei Wang > > On 2024/07/23 10:19:37 Keyong Zhou wrote: > > Congrats! > > > > Regards, > > Keyong Zhou > > > > angers zhu 于2024年7月23日周二 18:09写道： > > > > > Congrats! > > > > > > > > > T

Re: Re: [ANNOUNCE] New Celeborn Committer: Fei Wang

Congratulations ! Regards, Mridul On Tue, Jul 23, 2024 at 12:54 AM rexxiong wrote: > Congratulations! > > Regards, > Jiashu Xiong > > Nicholas Jiang 于2024年7月23日周二 13:02写道： > > > Congratulations!Regards, > > > > Nicholas Jiang > > > > > > 在 2024-07-23 12:21:19，"Yihe Li" 写道： > > >Congratulation

Re: [DISCUSS] Why do we remove RDD usage and RDD-backed code?

It is not necessary for users to create a SparkSession Martin - they can initialize a SparkContext and work with RDD api: which would be what Dongjoon is referring to IMO. Even after Spark Connect GA, I am not in favor of deprecating RDD Api at least until we have parity between both (which we don

Re: Question regarding TLS support in Celeborn

Hi, Yes, it is supported. Note that in addition to TLS communications specifically within Celeborn, the Apache Ratis message exchange (for Raft HA) requires use of grpc - TLS is not supported with raft rpc type = netty. Regards, Mridul On Thu, Jul 11, 2024 at 8:57 PM lohit wrote: > Hello Cel

Re: [DISCUSS] Release Apache Spark 3.5.2

+1 Thanks for volunteering ! Regards, Mridul On Thu, Jul 11, 2024 at 4:03 AM Kent Yao wrote: > Hi dev, > > It's been approximately 5 months since Feb 23, 2024, when > we released version 3.5.1 for branch-3.5. The patchset differing > from 3.5.1 has grown significantly, now consisting of over 16

Re: [DISCUSS] CIP-10: Introduce Celeborn Chaos Testing Framework

Hi, This is a great idea - and would go a long way in flushing out bugs and issues - and improving the overall robustness of Celeborn ! It would also be good to have: a) Capture a (replay) log of all events which were triggered. b) Ability to 'replay' the log and deterministically reach the same

Re: [VOTE] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 Regards, Mridul On Tue, Jul 9, 2024 at 10:19 AM Xianjin YE wrote: > +1 > > > On Jul 9, 2024, at 22:41, L. C. Hsieh wrote: > > > > +1 > > > > On Tue, Jul 9, 2024 at 1:13 AM Wenchen Fan wrote: > >> > >> +1 > >> > >> On Tue, Jul 9, 2024 at 10:47 AM Reynold Xin > wrote: > >>> > >>> +1 > >>>

Re: Jira version update

Thanks Keyong ! Regards, Mridul On Thu, Jul 4, 2024 at 9:22 PM Keyong Zhou wrote: > Thanks Mridul for pointing this out, I just modified 0.5.0 as released :) > > Regards, > Keyong Zhou > > Mridul Muralidharan 于2024年7月5日周五 03:13写道： > > > Hi, > > > >

Re: [DISCUSS] Allow GitHub Actions runs for contributors' PRs without approvals in apache/spark-connect-go

+1 Regards, Mridul On Mon, Jul 8, 2024 at 8:04 PM Ruifeng Zheng wrote: > +1 > > On Sat, Jul 6, 2024 at 4:45 AM bo yang wrote: > >> +1 This is a great suggestion, thanks Hyukjin! >> >> >> On Thu, Jul 4, 2024 at 4:11 AM Hyukjin Kwon wrote: >> >>> Alright! let me start the vote! >>> >>> On Thu,

Jira version update

Hi, While updating an issue manually, I noticed that 0.5.0 is still mentioned as an unreleased version in jira. Given 0.5 release, we should be getting it updated ? Regards, Mridul

Re: [VOTE] Release Apache Celeborn 0.5.0-rc3

Forgot to update here. Signatures, digests, etc check out fine. Checked out tag and build/tested with "-Pspark3.1" I keep getting the following error: - metrics/prometheus *** FAILED *** 200 did not equal 404 (ApiBaseResourceSuite.scala:90) - metrics/json *** FAILED *** 200 did not equal 404

[jira] [Resolved] (SPARK-47910) Memory leak when interrupting shuffle write using zstd compression

[ https://issues.apache.org/jira/browse/SPARK-47910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-47910. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-47910) Memory leak when interrupting shuffle write using zstd compression

[ https://issues.apache.org/jira/browse/SPARK-47910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-47910: --- Assignee: JacobZheng > Memory leak when interrupting shuffle write us

Re: Re: [VOTE] Contrinute Apache Celeborn CLI

+1 Regards, Mridul On Wed, Jun 12, 2024 at 1:08 AM Shaoyun Chen wrote: > +1 > > Keyong Zhou 于2024年6月12日周三 13:47写道： > > > > +1 > > > > Thanks for the proposal! > > > > Regards, > > Keyong Zhou > > > > Nicholas Jiang 于2024年6月12日周三 13:02写道： > > > > > +1. Looking forward to Celeborn CLI. > > > >

Re: Re: [Discussion] Proposal Management in Celeborn Community

On Thu, May 30, 2024 at 12:33 PM Nicholas wrote: > > > > > Hi Jiashu, > > > > > > > > > > > > > > > +1 for me. According to my experience in the Flink community, the > > > discussion of the CIP is commented in dev maillist instead of commented > &g

Re: [DISCUSS] Celeborn CLI Proposal

Hi, Looks good to me as well, I had reviewed this proposal internally already :-) Regards, Mridul On Fri, Jun 7, 2024 at 11:32 PM Keyong Zhou wrote: > Hi Aravind, > > Thanks for the proposal! The proposal LGTM, I think it's very valuable. > > Regards, > Keyong Zhou > > Aravind Patnam 于2024

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

Hi, Versioned documentation has the benefit that users can have reasonable confidence that features, functionality and examples mentioned will work with that released Spark version. A versionless guide runs into potential issues with deprecation, behavioral changes and new features. My concern

Re: [Discussion] Proposal Management in Celeborn Community

Inline comments, discussions are invaluable for design docs - this is not yet supported in confluence right ? Another option would be to iterate and discuss through other means (like google docs), and before vote, move it to the wiki - so that the community is deciding/voting on artifacts which a

Re: [VOTE] Release Apache Celeborn 0.4.1-rc1

+1 Signatures, digests, etc check out fine. Checked out tag and build/tested with "-Pspark3.1" Regards, Mridul On Sun, May 19, 2024 at 10:19 PM rexxiong wrote: > +1 (binding) > I checked > - Download links are valid. > - git commit hash is correct > - Checksums and signatures are valid. > - N

Re: [DRAFT] Celeborn Board Report

e... > > Regards, > Keyong Zhou > > Mridul Muralidharan 于2024年5月3日周五 23:38写道： > > > Hi, > > > > I meant call it out as part of the board report, so that it is captured > > in our updates to board. > > > > This is the first update post TLP, righ

Re: [DRAFT] Celeborn Board Report

doJW-f3BZAvxciDbI3mTw > <https://mp.weixin.qq.com/s/DdoJW-f3BZAvxciDbI3mTw> > > It'll be great if we can call out louder, do you have any idea? : ) > > Regards, > Keyong Zhou > > Mridul Muralidharan 于2024年5月3日周五 07:40写道： > > > Hi, > > > > Do we want to c

Re: [DRAFT] Celeborn Board Report

ct graduated recently). > - Chandni Singh was added as committer on 2024-03-21. > - Mridul Muralidharan was added as committer on 2024-04-29. > > ## Project Activity: > Software development activity: > > - We are preparing to release 0.4.1 in May. > - We are preparing to r

Re: [ANNOUNCE] Add Mridul Muralidharan as new committer

r 29, 2024, at 09:21, Keyong Zhou wrote: > > > > > > Hi Celeborn Community, > > > > > > The Project Management Committee (PMC) for Apache Celeborn > > > has invited Mridul Muralidharan to become a committer and we are > pleased > > > to announ

Re: [ANNOUNCE] Add Mridul Muralidharan as new committer

r 29, 2024, at 09:21, Keyong Zhou wrote: > > > > > > Hi Celeborn Community, > > > > > > The Project Management Committee (PMC) for Apache Celeborn > > > has invited Mridul Muralidharan to become a committer and we are > pleased > > > to announ

[jira] [Updated] (SPARK-47919) credulousTrustStoreManagers should return empty array for accepted issuers

[ https://issues.apache.org/jira/browse/SPARK-47919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-47919: Priority: Minor (was: Major) > credulousTrustStoreManagers should return em

[jira] [Created] (SPARK-47919) credulousTrustStoreManagers should return empty array for accepted issuers

Mridul Muralidharan created SPARK-47919: --- Summary: credulousTrustStoreManagers should return empty array for accepted issuers Key: SPARK-47919 URL: https://issues.apache.org/jira/browse/SPARK-47919

Re: [DISCUSS] Time for 0.4.1

+1 Regards, Mridul On Thu, Apr 18, 2024 at 11:50 PM Ethan Feng wrote: > +1 > > Thanks, > Ethan Feng > > Yu Li 于2024年4月16日周二 17:20写道： > > > > +1, thanks for driving this and volunteering as our RM, Nicholas! > > > > Best Regards, > > Yu > > > > On Sat, 13 Apr 2024 at 10:31, Keyong Zhou wrote:

[jira] [Comment Edited] (SPARK-47172) Upgrade Transport block cipher mode to GCM

[ https://issues.apache.org/jira/browse/SPARK-47172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838474#comment-17838474 ] Mridul Muralidharan edited comment on SPARK-47172 at 4/18/24 5:4

[jira] [Commented] (SPARK-47172) Upgrade Transport block cipher mode to GCM

[ https://issues.apache.org/jira/browse/SPARK-47172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838474#comment-17838474 ] Mridul Muralidharan commented on SPARK-47172: - We do not backport feat

[jira] [Commented] (SPARK-47172) Upgrade Transport block cipher mode to GCM

[ https://issues.apache.org/jira/browse/SPARK-47172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837927#comment-17837927 ] Mridul Muralidharan commented on SPARK-47172: - Given we have addre

Re: [VOTE] Release Spark 3.4.3 (RC2)

+1 Signatures, digests, etc check out fine. Checked out tag and build/tested with -Phive -Pyarn -Pkubernetes Regards, Mridul On Sun, Apr 14, 2024 at 11:31 PM Dongjoon Hyun wrote: > I'll start with my +1. > > - Checked checksum and signature > - Checked Scala/Java/R/Python/SQL Document's Spark

[jira] [Assigned] (SPARK-47253) Allow LiveEventBus to stop without the completely draining of event queue

[ https://issues.apache.org/jira/browse/SPARK-47253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-47253: --- Assignee: TakawaAkirayo > Allow LiveEventBus to stop without

[jira] [Resolved] (SPARK-47253) Allow LiveEventBus to stop without the completely draining of event queue

[ https://issues.apache.org/jira/browse/SPARK-47253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-47253. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-47318: Affects Version/s: 3.5.0 3.4.0 > AuthEngine key excha

[jira] [Resolved] (SPARK-47318) AuthEngine key exchange needs additional KDF round

[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-47318. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Comment Edited] (SPARK-47759) Apps being stuck after JavaUtils.timeStringAs fails to parse a legitimate time string

[ https://issues.apache.org/jira/browse/SPARK-47759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835587#comment-17835587 ] Mridul Muralidharan edited comment on SPARK-47759 at 4/10/24 4:0

[jira] [Comment Edited] (SPARK-47759) Apps being stuck after JavaUtils.timeStringAs fails to parse a legitimate time string

[ https://issues.apache.org/jira/browse/SPARK-47759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835587#comment-17835587 ] Mridul Muralidharan edited comment on SPARK-47759 at 4/10/24 4:0

[jira] [Commented] (SPARK-47759) Apps being stuck after JavaUtils.timeStringAs fails to parse a legitimate time string

[ https://issues.apache.org/jira/browse/SPARK-47759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17835587#comment-17835587 ] Mridul Muralidharan commented on SPARK-47759: - In order to validat

Re: Versioning of Spark Operator

I am trying to understand if we can simply align with Spark's version for this ? Makes the release and jira management much more simpler for developers and intuitive for users. Regards, Mridul On Tue, Apr 9, 2024 at 10:09 AM Dongjoon Hyun wrote: > Hi, Liang-Chi. > > Thank you for leading Apa

Re: Apache Spark 3.4.3 (?)

Hi Dongjoon, Thanks for volunteering ! I would suggest to wait for SPARK-47318 to be merged as well for 3.4 Regards, Mridul On Sat, Apr 6, 2024 at 6:49 PM Dongjoon Hyun wrote: > Hi, All. > > Apache Spark 3.4.2 tag was created on Nov 24th and `branch-3.4` has 85 > commits including important

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

--- >>> *From:* Denny Lee >>> *Sent:* Monday, April 1, 2024 10:06:14 AM >>> *To:* Hussein Awala >>> *Cc:* Chao Sun ; Hyukjin Kwon ; >>> Mridul Muralidharan ; dev >>> *Subject:* Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect) >&

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

Can you point me to the SPIP’s discussion thread please ? I was not able to find it, but I was on vacation, and so might have missed this … Regards, Mridul On Sun, Mar 31, 2024 at 9:08 PM Haejoon Lee wrote: > +1 > > On Mon, Apr 1, 2024 at 10:15 AM Hyukjin Kwon wrote: > >> Hi all, >> >> I'd li

Re: [ANNOUNCE] Apache Celeborn is graduated to Top Level Project

Congratulations !! Regards, Mridul On Tue, Mar 26, 2024 at 11:54 PM Nicholas Jiang wrote: > Congratulations! Witness the continuous development of the > community.Regards, > Nicholas Jiang > At 2024-03-25 20:49:36, "Ethan Feng" wrote: > >Hello Celeborn community, > > > >I am glad to share tha

Re: Maven 'stuck' in service test compilation ?

nvironmental information, JDK version, etc. > > Regards, > Ethan Feng > > Mridul Muralidharan 于2024年3月21日周四 15:25写道： > > > > Hi, > > > > > > I am observing that a maven build gets 'stuck' when compiling > "services" > > for

Re: [ANNOUNCE] Add Chandni Singh as new committer

Congratulations Chandni ! Great job :-) Regards, Mridul On Thu, Mar 21, 2024 at 3:30 AM Keyong Zhou wrote: > Hi Celeborn Community, > > The Podling Project Management Committee (PPMC) for Apache Celeborn > has invited Chandni Singh to become a committer and we are pleased > to announce that sh

Re: [ANNOUNCE] Add Chandni Singh as new committer

Congratulations Chandni ! Great job :-) Regards, Mridul On Thu, Mar 21, 2024 at 3:30 AM Keyong Zhou wrote: > Hi Celeborn Community, > > The Podling Project Management Committee (PPMC) for Apache Celeborn > has invited Chandni Singh to become a committer and we are pleased > to announce that sh

Maven 'stuck' in service test compilation ?

Hi, I am observing that a maven build gets 'stuck' when compiling "services" for running tests. Without tests, this goes through: $ ARGS="-Pspark-3.1"; ./build/mvn $ARGS clean 2>&1 | tee clean_output.txt && ./build/mvn -DskipTests $ARGS package 2>&1 | tee build_output.txt This gets stuck inde

[jira] [Assigned] (SPARK-45375) [CORE] Mark connection as timedOut in TransportClient.close

[ https://issues.apache.org/jira/browse/SPARK-45375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-45375: --- Assignee: Hasnain Lakhani > [CORE] Mark connection as timedOut

[jira] [Comment Edited] (SPARK-45374) [CORE] Add test keys for SSL functionality

[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828087#comment-17828087 ] Mridul Muralidharan edited comment on SPARK-45374 at 3/18/24 8:2

[jira] [Assigned] (SPARK-45374) [CORE] Add test keys for SSL functionality

[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-45374: --- Assignee: Hasnain Lakhani > [CORE] Add test keys for SSL functional

[jira] [Comment Edited] (SPARK-45374) [CORE] Add test keys for SSL functionality

[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828087#comment-17828087 ] Mridul Muralidharan edited comment on SPARK-45374 at 3/18/24 8:0

[jira] [Commented] (SPARK-45374) [CORE] Add test keys for SSL functionality

[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828087#comment-17828087 ] Mridul Muralidharan commented on SPARK-45374: - Missed your query, you

Re: [Spark-Core] Improving Reliability of spark when Executors OOM

are some > open discussion threads on the doc you shared. > > @Mridul Muralidharan In what state are your efforts > along this? Is it something that your team is actively pursuing/ building > or are mostly planning right now? Asking so that we can align efforts on > this. >

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

I am supportive of the proposal - this is a step in the right direction ! Additional metadata (explicit and inferred) for log records, and exposing them for indexing is extremely useful. The specifics of the API still need some work IMO and does not need to be this disruptive, but I consider tha

[jira] [Updated] (SPARK-47146) Possible thread leak when doing sort merge join

[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-47146: Fix Version/s: 3.5.2 3.4.3 > Possible thread leak when do

[jira] [Commented] (SPARK-47146) Possible thread leak when doing sort merge join

[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823832#comment-17823832 ] Mridul Muralidharan commented on SPARK-47146: - Backported to 3.5 and 3.

[jira] [Assigned] (SPARK-47146) Possible thread leak when doing sort merge join

[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-47146: --- Assignee: JacobZheng > Possible thread leak when doing sort merge j

[jira] [Resolved] (SPARK-47146) Possible thread leak when doing sort merge join

[ https://issues.apache.org/jira/browse/SPARK-47146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-47146. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

Re: [DISCUSS] SPIP: Structured Spark Logging

Hi Gengling, Thanks for sharing this ! I added a few queries to the proposal doc, and we can continue discussing there, but overall I am in favor of this. Regards, Mridul On Fri, Mar 1, 2024 at 1:35 AM Gengliang Wang wrote: > Hi All, > > I propose to enhance our logging system by transition

Re: [VOTE] Graduate Apache Celeborn (incubating) as a TLP - Community

+1 Regards, Mridul On Fri, Mar 1, 2024 at 4:35 AM Nicholas wrote: > > +1. > > > Regards, > Nicholas Jiang > > > > > -- > 发自我的网易邮箱手机智能版 > > > > - Original Message - > From: "Yu Li" > To: dev@celeborn.apache.org > Sent: Fri, 1 Mar 2024 16:52:10 +0800 > Subject: [VOTE] Graduate Apache C

Re: [DISCUSS] Graduate Celeborn as TLP

+1 Looking forward to Celeborn as a TLP ! Best wishes to the community :-) Regards, Mridul On Tue, Feb 27, 2024 at 5:23 AM Willem Jiang wrote: > Thanks for the clarification. Now we are good to go. > > Willem Jiang > > > > On Tue, Feb 27, 2024 at 7:15 PM Keyong Zhou wrote: > > > > Thanks Wil

Re: [ANNONCE] New PPMC member: Fu Chen

Congratulations ! Regards, Mridul On Mon, Feb 19, 2024 at 7:46 PM Cheng Pan wrote: > Congrats! > > Thanks, > Cheng Pan > > > > On Feb 20, 2024, at 08:22, Nicholas wrote: > > > > Congratulations to Fu Chen!Regards, > > Nicholas Jiang > > > > > > > > > > At 2024-02-20 00:23:06, "Shaoyun Chen" w

Re: Large number of incubator-celeb...@noreply.github.com emails

cided to choose JIRA at last. Seems > different projects have different preferences. Maybe > newer projects tends to use Github. > > To me, I'm actually fine with both. JIRA works well so far, will using > Github be more beneficial? Glad to hear about your opinion. > > Th

Re: Large number of incubator-celeb...@noreply.github.com emails

Looks like I am wrong, github issues can be used [1]. Is Celeborn planning to use github issues going forward ? Regards, Mridul [1] https://www.apache.org/dev/#issues On Wed, Feb 7, 2024 at 12:00 AM Mridul Muralidharan wrote: > Hi, > > I received a fairly large number of

Large number of incubator-celeb...@noreply.github.com emails

Hi, I received a fairly large number of emails to incubator-celeb...@noreply.github.com, which typically are for PR's. They appear to be github issues - are we trying to move to github issues instead of Apache jira ? IIRC there is a policy to use jira for tracking bugs/improvements, right ? Reg

[jira] [Resolved] (SPARK-46512) Optimize shuffle reading when both sort and combine are used.

[ https://issues.apache.org/jira/browse/SPARK-46512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46512. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-46512) Optimize shuffle reading when both sort and combine are used.

[ https://issues.apache.org/jira/browse/SPARK-46512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46512: --- Assignee: Chenyu Zheng > Optimize shuffle reading when both sort

[jira] [Assigned] (SPARK-46733) Simplify the ContextCleaner|BlockManager by the exit operation only depend on interrupt thread

[ https://issues.apache.org/jira/browse/SPARK-46733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46733: --- Assignee: Jiaan Geng > Simplify the ContextCleaner|BlockManager by

[jira] [Resolved] (SPARK-46733) Simplify the ContextCleaner|BlockManager by the exit operation only depend on interrupt thread

[ https://issues.apache.org/jira/browse/SPARK-46733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46733. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Commented] (SPARK-46623) Replace SimpleDateFormat with DateTimeFormatter

[ https://issues.apache.org/jira/browse/SPARK-46623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17808080#comment-17808080 ] Mridul Muralidharan commented on SPARK-46623: - Issue resolved by

[jira] [Resolved] (SPARK-46623) Replace SimpleDateFormat with DateTimeFormatter

[ https://issues.apache.org/jira/browse/SPARK-46623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46623. - Fix Version/s: 4.0.0 Assignee: Jiaan Geng Resolution: Fixed

[jira] [Resolved] (SPARK-46696) In ResourceProfileManager, function calls should occur after variable declarations.

[ https://issues.apache.org/jira/browse/SPARK-46696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46696. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-46696) In ResourceProfileManager, function calls should occur after variable declarations.

[ https://issues.apache.org/jira/browse/SPARK-46696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46696: --- Assignee: liangyongyuan > In ResourceProfileManager, function calls sho

Re: [Spark-Core] Improving Reliability of spark when Executors OOM

Hi, We are internally exploring adding support for dynamically changing the resource profile of a stage based on runtime characteristics. This includes failures due to OOM and the like, slowness due to excessive GC, resource wastage due to excessive overprovisioning, etc. Essentially handles sca

[jira] [Assigned] (SPARK-46399) Add exit status to the Application End event for the use of Spark Listener

[ https://issues.apache.org/jira/browse/SPARK-46399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46399: --- Assignee: Reza Safi > Add exit status to the Application End event for

[jira] [Resolved] (SPARK-46399) Add exit status to the Application End event for the use of Spark Listener

[ https://issues.apache.org/jira/browse/SPARK-46399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46399. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

Re: [VOTE] Release Apache Celeborn(Incubating) 0.3.2-incubating-rc0

+1 Signatures, digests, license, etc check out fine. Checked out tag and build/tested with -Pspark3.1 and -Pflink-1.17 Regards, Mridul On Tue, Dec 19, 2023 at 8:06 PM rexxiong wrote: > +1 (binding) > I checked > - Download links are valid. > - git commit hash is correct > - Checksums and sign

[jira] [Resolved] (SPARK-46132) [CORE] Support key password for JKS keys for RPC SSL

[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46132. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-46132) [CORE] Support key password for JKS keys for RPC SSL

[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46132: --- Assignee: Hasnain Lakhani > [CORE] Support key password for JKS keys

Re: [VOTE] Release Spark 3.3.4 (RC1)

I am seeing a bunch of python related (43) failures in the sql module (for example [1]) ... I am currently on Python 3.11.6, java 8. Not sure if ubuntu modified anything from under me, thoughts ? I am currently testing this against an older branch to make sure it is not an issue with my desktop.

Re: [DISCUSS] Time for 0.3.2

+1 on 0.3.2, thanks Nicholas ! Regards, Mridul On Thu, Dec 7, 2023 at 12:51 AM Cheng Pan wrote: > +1, thanks for volunteering. > > Feel free to ping me if you encounter permission issues during the release > phase. > > Thanks, > Cheng Pan > > > > On Dec 7, 2023, at 14:31, Nicholas wrote: > >

[jira] [Updated] (SPARK-46058) [CORE] Add separate flag for privateKeyPassword

[ https://issues.apache.org/jira/browse/SPARK-46058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated SPARK-46058: Labels: pull-request-available (was: pull-request-available releasenotes

  1   2   3   4   5   6   7   8   9   10   >