Hi, Min
Thanks for the proposed possible solution. And you mean it may can be
implemented at the library level, could you help give more details about
the library level that I can dig into.
I am wondering if I can implement a plugin based on things like
KernelManager to achieve this, like opening another configurable parameter
in jupyter_notebook_config.py to help me mange the maximum kernel instances
number. Any advice about this is appreciated.
Thanks again.
在 2017年2月7日星期二 UTC+8下午7:03:08，Min RK写道：
>
>
>
> On Mon, Feb 6, 2017 at 3:54 PM, <[email protected] <javascript:>> wrote:
>
>> Hi,
>> We know user can create as many notebook instances as they can in
>> one jupyter notebook server client, and when we want to integrate spark's
>> pyspark into jupyter notebook, which means using the ipykernel, when every
>> each notebook instance is created, then a pyspark shell(or to say a driver)
>> is initialized, since we run that in spark client mode, so all the started
>> drivers would run on same host. And when some user in crazy mode, like
>> creating many many notebook instances, then many drivers would all start in
>> one host, which will lead to being lack of available resource easily.
>> So I am wondering if Jupyter notebook has some mechanism to deal
>> with or avoid such kind of that issue?
>>
>
> Jupyter itself has no mechanism to deal with this. It would have to be
> done at the library level - i.e. check for other instances and do one of:
>
> - connect to running instances if possible
> - refuse to start if too many are running
> - etc.
>
> -Min
>
>
>>
>> Many thanks in advance!
>>
>> Best Regards
>> Sherry
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Project Jupyter" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jupyter/3fa00644-1ea0-419d-8bbe-3dfc0d67fe95%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jupyter/3fa00644-1ea0-419d-8bbe-3dfc0d67fe95%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jupyter/1fcbb6f2-bbfd-426b-b521-d3d09a8c3846%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
