Re: Conda Python Env in K8S

Hi Meikel, Well the short answer is it is what it is, for one reason or other. if someone else managed to make it work, then no doubt will be delighted to hear it. Until then I prefer the built- in docker image. Also by centralising this in the docker image, it will be available if a node fails

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Shane you kick ass thank you for everything you’ve done for us :) Keep on rocking :) On Mon, Dec 6, 2021 at 4:24 PM Hyukjin Kwon wrote: > Thanks, Shane. > > On Tue, 7 Dec 2021 at 09:19, Dongjoon Hyun > wrote: > >> I really want to thank you for all your help. >> You've done so many things for

Re: Time for Spark 3.2.1?

Always fine by me if someone wants to roll a release. It's been ~6 months since the last 3.0.x and 3.1.x releases, too; a new release of those wouldn't hurt either, if any of our release managers have the time or inclination. 3.0.x is reaching unofficial end-of-life around now anyway. On Mon,

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thanks, Shane. On Tue, 7 Dec 2021 at 09:19, Dongjoon Hyun wrote: > I really want to thank you for all your help. > You've done so many things for the Apache Spark community. > > Sincerely, > Dongjoon > > > On Mon, Dec 6, 2021 at 12:02 PM shane knapp ☠ wrote: > >> hey everyone! >> >> after a

Time for Spark 3.2.1?

Hi all, It's been two months since Spark 3.2.0 release, and we have resolved many bug fixes and regressions. What do you guys think about rolling Spark 3.2.1 release? cc @huaxin gao FYI who I happened to overhear that is interested in rolling the maintenance release :-).

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thank you, Shane. On Mon, Dec 6, 2021 at 4:27 PM Holden Karau wrote: > > Shane you kick ass thank you for everything you’ve done for us :) Keep on > rocking :) > > On Mon, Dec 6, 2021 at 4:24 PM Hyukjin Kwon wrote: >> >> Thanks, Shane. >> >> On Tue, 7 Dec 2021 at 09:19, Dongjoon Hyun wrote:

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

I really want to thank you for all your help. You've done so many things for the Apache Spark community. Sincerely, Dongjoon On Mon, Dec 6, 2021 at 12:02 PM shane knapp ☠ wrote: > hey everyone! > > after a marathon run of nearly a decade, we're finally going to be > shutting down

Re: Time for Spark 3.2.1?

Thanks for pinging me. I am happy to take care of Spark 3.2.1 release :) On Mon, Dec 6, 2021 at 4:57 PM Sean Owen wrote: > Always fine by me if someone wants to roll a release. > > It's been ~6 months since the last 3.0.x and 3.1.x releases, too; a new > release of those wouldn't hurt either,

[Apache Spark Jenkins] build system shutting down Dec 23th, 2021

hey everyone! after a marathon run of nearly a decade, we're finally going to be shutting down {amp|rise}lab jenkins at the end of this month... the earliest snapshot i could find is from 2013 with builds for spark 0.7:

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Hi, Shane, Thank you for your work on it! Xiao On Mon, Dec 6, 2021 at 6:20 PM L. C. Hsieh wrote: > Thank you, Shane. > > On Mon, Dec 6, 2021 at 4:27 PM Holden Karau wrote: > > > > Shane you kick ass thank you for everything you’ve done for us :) Keep > on rocking :) > > > > On Mon, Dec 6,

Re: Time for Spark 3.2.1?

+1 to make new maintenance releases for all 3.x branches. On Tue, Dec 7, 2021 at 8:57 AM Sean Owen wrote: > Always fine by me if someone wants to roll a release. > > It's been ~6 months since the last 3.0.x and 3.1.x releases, too; a new > release of those wouldn't hurt either, if any of our

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thanks, Shane! Really appreciate it! Wenchen On Tue, Dec 7, 2021 at 12:38 PM Xiao Li wrote: > Hi, Shane, > > Thank you for your work on it! > > Xiao > > > > > On Mon, Dec 6, 2021 at 6:20 PM L. C. Hsieh wrote: > >> Thank you, Shane. >> >> On Mon, Dec 6, 2021 at 4:27 PM Holden Karau wrote: >>

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Farewell to Jenkins and its classic weather forecast build status icons: [image: health-80plus.png][image: health-60to79.png][image: health-40to59.png][image: health-20to39.png][image: health-00to19.png] And thank you Shane for all the help over these years. Will you be nuking all the

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Wow! end of an era Thanks so much to you Shane for all you work over 10 (!!) years. And to Amplab also! Farewell Spark Jenkins! N On Tue, Dec 7, 2021 at 6:49 AM Nicholas Chammas wrote: > Farewell to Jenkins and its classic weather forecast build status icons: > > [image:

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thanks for all the hard work you have been doing, Shane! On Tue, Dec 7, 2021 at 2:17 PM Nick Pentreath wrote: > Wow! end of an era > > Thanks so much to you Shane for all you work over 10 (!!) years. And to > Amplab also! > > Farewell Spark Jenkins! > > N > > On Tue, Dec 7, 2021 at 6:49 AM

RE: Conda Python Env in K8S

Hi Mich, Thanks for your response. Yes -py-files options works. I also tested it. The question is why the -archives option doesn't? >From Jira I can see that it should be available since 3.1.0: https://issues.apache.org/jira/browse/SPARK-33530 https://issues.apache.org/jira/browse/SPARK-33615