Re: Stable Release for Spark-on-k8s

in K8s area at 3.1.1 and > want more contribution in the future. Please try to use 3.1.1-rc1 and let > us know your feedback. 3.1.1-rc2 will start soon (next Monday). > > https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-33005 > > Bests, > Dongjoon > > On T

Stable Release for Spark-on-k8s

Hey Everyone When are we planning to release the stable release/ Generally Available release for Spark-on-k8s ? I see that it is still experimental. Regards Ankit Prakash Gupta

Re: Add option to Spark UI to proxy to the executors?

Hey Holden I can probably help you with that. We should probably pass the same configs to the application also,for the reverse proxy to work. Try setting the spark.ui.reverseProxy=true while running your application. I hope this will solve your issue. Best Regards Ankit Prakash Gupta On Thu,

Re: Spark docker image convention proposal for docker repository

Hey Mich Adding to what Mich is suggesting, how about having the base OS version in the image tag as well, like --- 3.1.2-scala_2.12-java11-slim 3.1.2_sparkpy-scala_2.12-java11-buster 3.1.2_sparkR-scala_2.12-java11-slim Regards. Ankit Prakash Gupta info.ank...@gmail.com LinkedIn :

Re: [DISCUSS] SPIP: Support Docker Official Image for Spark

Hi Yikun Thanks for all your efforts! This is very much needed. But I have the below three questions: 1. How do we support deployments of spark-standalone clusters in case the users wants to use the same image for spark-standalone clusters ? Since that is also widely used. 2. I am not sure about

Re: [VOTE] SPIP: Support Docker Official Image for Spark

+1 Regards, Ankit Prakash Gupta On Thu, Sep 22, 2022 at 10:38 AM Yang,Jie(INF) wrote: > +1 (non-binding) > > > > Regards, > > Yang Jie > > > > *发件人**: *Gengliang Wang > *日期**: *2022年9月22日 星期四 12:22 > *收件人**: *Xiangrui Meng > *抄送**: *Kent Yao , Hyukjin Kwon , > dev > *主题**: *Re: [VOTE]

Re: [DISCUSS] SPIP: Support Docker Official Image for Spark

; rehl/centos/rocky/openEuler serious, yum/dnf software manager. But as you >> know, different OS's have various package versions and upgrade policies, so >> it's perhaps not very easy work for maintenance, but I think it's possible. >> >> Regards, >> Yikun >> &g

Re: Contributions needed: 4 higher order functions

Hey Hyukjin / Devs Please do review the changes for below mentioned is the pull PR for the same. https://issues.apache.org/jira/browse/SPARK-41232 https://github.com/apache/spark/pull/38865 Thanks and Regards Ankit Prakash Gupta On Wed, Nov 30, 2022 at 4:04 PM Hyukjin Kwon wrote: > Hi all,

Re: Spark Multiple Hive Metastore Catalog Support

++ User Mailing List Just a reminder, anyone who can help on this. Thanks a lot ! Ankit Prakash Gupta On Wed, Apr 12, 2023 at 8:22 AM Ankit Gupta wrote: > Hi All > > The question is regarding the support of multiple Remote Hive Metastore > catalogs with Spark. Starting Spark

Re: Spark Multiple Hive Metastore Catalog Support

ps://github.com/ExpediaGroup/waggle-dance > > This may be useful or perhaps serve as inspiration. > > Thanks, > > Elliot. > > On Mon, 17 Apr 2023 at 16:38, Ankit Gupta wrote: > >> ++ >> User Mailing List >> >> Just a reminder, anyone who can help on this. &

Spark Multiple Hive Metastore Catalog Support

Hi All The question is regarding the support of multiple Remote Hive Metastore catalogs with Spark. Starting Spark 3, multiple catalog support is added in spark, but have we implemented any CatalogPlugin that can help us configure multiple Remote Hive Metastore Catalogs ? If yes, can anyone help