Re: Jupyter notebook on Dataproc versus GKE

2022-09-06 Thread Holden Karau 
I’ve used Argo for K8s scheduling, for awhile it’s also what Kubeflow used
underneath for scheduling.

On Tue, Sep 6, 2022 at 10:01 AM Mich Talebzadeh 
wrote:

> Thank you all.
>
> Has anyone used Argo for k8s scheduler by any chance?
>
> On Tue, 6 Sep 2022 at 13:41, Bjørn Jørgensen 
> wrote:
>
>> "*JupyterLab is the next-generation user interface for Project Jupyter
>> offering all the familiar building blocks of the classic Jupyter Notebook
>> (notebook, terminal, text editor, file browser, rich outputs, etc.) in a
>> flexible and powerful user interface.*"
>> https://github.com/jupyterlab/jupyterlab
>>
>> You will find them both at https://jupyter.org
>>
>> man. 5. sep. 2022 kl. 23:40 skrev Mich Talebzadeh <
>> mich.talebza...@gmail.com>:
>>
>>> Thanks Bjorn,
>>>
>>> What are the differences and the functionality Jupyerlab brings in on
>>> top of Jupyter notebook?
>>>
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Mon, 5 Sept 2022 at 20:58, Bjørn Jørgensen 
>>> wrote:
>>>
 Jupyter notebook is replaced with jupyterlab :)

 man. 5. sep. 2022 kl. 21:10 skrev Holden Karau :

>
>
> On Mon, Sep 5, 2022 at 9:00 AM Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
>> Thanks for that.
>>
>> How do you rate the performance of Jupyter W/Spark on K8s compared to
>> the same on  a cluster of VMs (example Dataproc).
>>
>> Also somehow a related question (may be naive as well). For example,
>> Google offers a lot of standard ML libraries for example built into a 
>> data
>> warehouse like BigQuery. What does the Jupyter notebook offer that others
>> don't?
>>
> Jupyter notebook doesn’t offer any particular set of libraries,
> although you can add your own to the container etc.
>
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>> for any loss, damage or destruction of data or any other property which 
>> may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 5 Sept 2022 at 12:47, Holden Karau 
>> wrote:
>>
>>> I’ve run Jupyter w/Spark on K8s, haven’t tried it with Dataproc
>>> personally.
>>>
>>> The Spark K8s pod scheduler is now more pluggable for Yunikorn and
>>> Volcano can be used with less effort.
>>>
>>> On Mon, Sep 5, 2022 at 7:44 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>

 Hi,


 Has anyone got experience of running Jupyter on dataproc versus
 Jupyter notebook on GKE (k8).


 I have not looked at this for a while but my understanding is that
 Spark on GKE/k8 is not yet performed. This is classic Spark with
 Python/Pyspark.


 Also I would like to know the state of spark with Volcano. Has
 progress made on that front.


 Regards,


 Mich


view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility
 for any loss, damage or destruction of data or any other property 
 which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary 
 damages
 arising from such loss, damage or destruction.



>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams:

Re: Jupyter notebook on Dataproc versus GKE

2022-09-06 Thread Mich Talebzadeh 
Thank you all.

Has anyone used Argo for k8s scheduler by any chance?

On Tue, 6 Sep 2022 at 13:41, Bjørn Jørgensen 
wrote:

> "*JupyterLab is the next-generation user interface for Project Jupyter
> offering all the familiar building blocks of the classic Jupyter Notebook
> (notebook, terminal, text editor, file browser, rich outputs, etc.) in a
> flexible and powerful user interface.*"
> https://github.com/jupyterlab/jupyterlab
>
> You will find them both at https://jupyter.org
>
> man. 5. sep. 2022 kl. 23:40 skrev Mich Talebzadeh <
> mich.talebza...@gmail.com>:
>
>> Thanks Bjorn,
>>
>> What are the differences and the functionality Jupyerlab brings in on top
>> of Jupyter notebook?
>>
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Mon, 5 Sept 2022 at 20:58, Bjørn Jørgensen 
>> wrote:
>>
>>> Jupyter notebook is replaced with jupyterlab :)
>>>
>>> man. 5. sep. 2022 kl. 21:10 skrev Holden Karau :
>>>


 On Mon, Sep 5, 2022 at 9:00 AM Mich Talebzadeh <
 mich.talebza...@gmail.com> wrote:

> Thanks for that.
>
> How do you rate the performance of Jupyter W/Spark on K8s compared to
> the same on  a cluster of VMs (example Dataproc).
>
> Also somehow a related question (may be naive as well). For example,
> Google offers a lot of standard ML libraries for example built into a data
> warehouse like BigQuery. What does the Jupyter notebook offer that others
> don't?
>
 Jupyter notebook doesn’t offer any particular set of libraries,
 although you can add your own to the container etc.

>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for
> any loss, damage or destruction of data or any other property which may
> arise from relying on this email's technical content is explicitly
> disclaimed. The author will in no case be liable for any monetary damages
> arising from such loss, damage or destruction.
>
>
>
>
> On Mon, 5 Sept 2022 at 12:47, Holden Karau 
> wrote:
>
>> I’ve run Jupyter w/Spark on K8s, haven’t tried it with Dataproc
>> personally.
>>
>> The Spark K8s pod scheduler is now more pluggable for Yunikorn and
>> Volcano can be used with less effort.
>>
>> On Mon, Sep 5, 2022 at 7:44 AM Mich Talebzadeh <
>> mich.talebza...@gmail.com> wrote:
>>
>>>
>>> Hi,
>>>
>>>
>>> Has anyone got experience of running Jupyter on dataproc versus
>>> Jupyter notebook on GKE (k8).
>>>
>>>
>>> I have not looked at this for a while but my understanding is that
>>> Spark on GKE/k8 is not yet performed. This is classic Spark with
>>> Python/Pyspark.
>>>
>>>
>>> Also I would like to know the state of spark with Volcano. Has
>>> progress made on that front.
>>>
>>>
>>> Regards,
>>>
>>>
>>> Mich
>>>
>>>
>>>view my Linkedin profile
>>> 
>>>
>>>
>>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>>> for any loss, damage or destruction of data or any other property which 
>>> may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary 
>>> damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9  
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
> --
 Twitter: https://twitter.com/holdenkarau
 Books (Learning Spark, High Performance Spark, etc.):
 https://amzn.to/2MaRAG9  
 YouTube Live Streams: https://www.youtube.com/user/holdenkarau

>>>
>>>
>>> --
>>> Bjørn Jørgensen
>>> Vestre Aspehaug 4
>>> ,
>>> 6010 Ålesund
>>> Norge
>>>
>>> +47 480 94 297
>>>
>>
>
> --
> Bjørn Jørgensen
> Vestre Aspehaug 4
> ,
> 6010 Ålesund
>

Re: Jupyter notebook on Dataproc versus GKE

2022-09-06 Thread Bjørn Jørgensen 
"*JupyterLab is the next-generation user interface for Project Jupyter
offering all the familiar building blocks of the classic Jupyter Notebook
(notebook, terminal, text editor, file browser, rich outputs, etc.) in a
flexible and powerful user interface.*"
https://github.com/jupyterlab/jupyterlab

You will find them both at https://jupyter.org

man. 5. sep. 2022 kl. 23:40 skrev Mich Talebzadeh :

> Thanks Bjorn,
>
> What are the differences and the functionality Jupyerlab brings in on top
> of Jupyter notebook?
>
>
>
>view my Linkedin profile
> 
>
>
>  https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 5 Sept 2022 at 20:58, Bjørn Jørgensen 
> wrote:
>
>> Jupyter notebook is replaced with jupyterlab :)
>>
>> man. 5. sep. 2022 kl. 21:10 skrev Holden Karau :
>>
>>>
>>>
>>> On Mon, Sep 5, 2022 at 9:00 AM Mich Talebzadeh <
>>> mich.talebza...@gmail.com> wrote:
>>>
 Thanks for that.

 How do you rate the performance of Jupyter W/Spark on K8s compared to
 the same on  a cluster of VMs (example Dataproc).

 Also somehow a related question (may be naive as well). For example,
 Google offers a lot of standard ML libraries for example built into a data
 warehouse like BigQuery. What does the Jupyter notebook offer that others
 don't?

>>> Jupyter notebook doesn’t offer any particular set of libraries, although
>>> you can add your own to the container etc.
>>>



view my Linkedin profile
 


  https://en.everybodywiki.com/Mich_Talebzadeh



 *Disclaimer:* Use it at your own risk. Any and all responsibility for
 any loss, damage or destruction of data or any other property which may
 arise from relying on this email's technical content is explicitly
 disclaimed. The author will in no case be liable for any monetary damages
 arising from such loss, damage or destruction.




 On Mon, 5 Sept 2022 at 12:47, Holden Karau 
 wrote:

> I’ve run Jupyter w/Spark on K8s, haven’t tried it with Dataproc
> personally.
>
> The Spark K8s pod scheduler is now more pluggable for Yunikorn and
> Volcano can be used with less effort.
>
> On Mon, Sep 5, 2022 at 7:44 AM Mich Talebzadeh <
> mich.talebza...@gmail.com> wrote:
>
>>
>> Hi,
>>
>>
>> Has anyone got experience of running Jupyter on dataproc versus
>> Jupyter notebook on GKE (k8).
>>
>>
>> I have not looked at this for a while but my understanding is that
>> Spark on GKE/k8 is not yet performed. This is classic Spark with
>> Python/Pyspark.
>>
>>
>> Also I would like to know the state of spark with Volcano. Has
>> progress made on that front.
>>
>>
>> Regards,
>>
>>
>> Mich
>>
>>
>>view my Linkedin profile
>> 
>>
>>
>>  https://en.everybodywiki.com/Mich_Talebzadeh
>>
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility
>> for any loss, damage or destruction of data or any other property which 
>> may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
 --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>>
>> --
>> Bjørn Jørgensen
>> Vestre Aspehaug 4, 6010 Ålesund
>> Norge
>>
>> +47 480 94 297
>>
>

-- 
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge

+47 480 94 297