I’ve run Jupyter w/Spark on K8s, haven’t tried it with Dataproc personally.
The Spark K8s pod scheduler is now more pluggable for Yunikorn and Volcano can be used with less effort. On Mon, Sep 5, 2022 at 7:44 AM Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Hi, > > > Has anyone got experience of running Jupyter on dataproc versus Jupyter > notebook on GKE (k8). > > > I have not looked at this for a while but my understanding is that Spark > on GKE/k8 is not yet performed. This is classic Spark with Python/Pyspark. > > > Also I would like to know the state of spark with Volcano. Has progress > made on that front. > > > Regards, > > > Mich > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > https://en.everybodywiki.com/Mich_Talebzadeh > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
