Looking through the code a bit more I found this code snippet in 
jupyter_client/manager.py 
<https://github.com/jupyter/jupyter_client/blob/master/jupyter_client/manager.py#L188-L193>
:

def _launch_kernel(self, kernel_cmd, **kw):
    """actually launch the kernel

    override in a subclass to launch kernel subprocesses differently
    """
    return launch_kernel(kernel_cmd, **kw)

I know the PID of the running kernel process. If I could somehow do a 
subprocess.Popen() on that existing process and return a reference to that, 
I think that would work.

Unfortunately I can't anything related to doing a subprocess.Popen() on an 
*existing* process...

On Tuesday, February 13, 2018 at 1:50:04 PM UTC-8, Edward Banner wrote:
>
> *Arg! I can't seem to get formatting down on google forums. I apologize. 
> Here is my response in full.*
>
> Hi Roland,
>
> Thanks for your response. Your suggestion makes sense. But does that mean 
> going with a custom server extension 
> <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> is 
> the way to go?
>
> I spent yesterday attempting to accomplish this solely though jupyter 
> configuration options. For a running kernel with information
>
> {
>   "shell_port": 63391,
>   "iopub_port": 63394,
>   "stdin_port": 63392,
>   "control_port": 63393,
>   "hb_port": 63396,
>   "ip": "127.0.0.1",
>   "key": "fff108f8-b1bf9b17aa337ce7d3d14565",
>   "transport": "tcp",
>   "signature_scheme": "hmac-sha256",
>   "kernel_name": ""
> }
>
> I set the following variables in jupyter_notebook_config.py.
>
> c.ConnectionFileMixin.connection_file = 
> '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json'
> c.ConnectionFileMixin.control_port = 63393
> c.ConnectionFileMixin.iopub_port = 63394
> c.ConnectionFileMixin.stdin_port = 63392
> c.ConnectionFileMixin.hb_port = 63396
> c.ConnectionFileMixin.ip = '127.0.0.1'
> c.ConnectionFileMixin.shell_port = 63391
> c.ConnectionFileMixin.transport = 'tcp'
>
> However when I created a new notebook I get errors like
>
> [I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5)
> [D 12:53:44.982 NotebookApp] Starting kernel: [
> '/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 
> 'ipykernel_launcher', '-f', 
> '/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json'
> ]
> [D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393
> Traceback (most recent call last):
>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
> 193, in _run_module_as_main
>     "__main__", mod_spec)
>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
> 85, in _run_code
>     exec(code, run_globals)
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py"
> , line 16, in <module>
>     app.launch_new_instance()
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
> , line 657, in launch_instance
>     app.initialize(argv)
>   File "<decorator-gen-123>", line 2, in initialize
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
> , line 87, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 448, in initialize
>     self.init_sockets()
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 238, in init_sockets
>     self.shell_port = self._bind_socket(self.shell_socket, self.shell_port
> )
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 180, in _bind_socket
>     s.bind("tcp://%s:%i" % (self.ip, port))
>   File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.
> socket.Socket.bind (zmq/backend/cython/socket.c:5653)
>   File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.
> checkrc._check_rc (zmq/backend/cython/socket.c:10014)
> zmq.error.ZMQError: Address already in use
>
> until it hits the maximum number of retries. Presumably this is because 
> the ipykernel command attempts to create a *new* socket but since it 
> already exists it fails to do so.
>
> Thinking that I could "trick" jupyter into not using ipykernel and just 
> use the existing sockets I set
>
> c.KernelManager.kernel_cmd = [
> '/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py']
>
> where bar.py contains
>
> while True:
>     pass
>
> After creating another new notebook I get
>
> [D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4-
> 4d98-a0c6-1f227627f784/channels
> [D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391
> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394
> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392
>
> which gave me hope! But then when I tried to execute a cell it just hangs. 
> I did notice that http://localhost:8888/api/kernels has the following 
> information.
>
> [
>     {
>         id: "9b260abf-072f-48e2-b5f0-213d934418a2",
>         name: "python3",
>         last_activity: "2018-02-13T21:10:41.460182Z",
>         execution_state: "starting",
>         connections: 0
>     }
> ]
>
> However the execution_state being stuck at "starting" and connections 
> being at 0 make me think that bar.py isn't quite doing enough.
>
> Digging into the code a bit I found the object returned from the call to 
> ipykernel is a subprocess.Popen() object. But I couldn't find anywhere else 
> in the code except KernelManager.is_alive() where Popen.poll() is called 
> on it 
> <https://github.com/jupyter/jupyter_client/blob/38b81cc07ddf70c2f534f11c4f4ef0103bfd5fe2/jupyter_client/multikernelmanager.py#L204-L205>
> .
>
> I also tried creating a custom kernel manager class and set
>
> c.NotebookApp.kernel_manager_class = 
> 'kernelmanager.ExistingMappingKernelManager'
>
> where kernelmanager.ExistingMappingKernelManager is the class
>
> from tornado import gen, web
> from jupyter_client import KernelManager
> from notebook.services.kernels.kernelmanager import MappingKernelManager
>
> class ExistingMappingKernelManager(MappingKernelManager):
>     """A KernelManager that just connects to an existing kernel."""
>
>     @gen.coroutine
>     def start_kernel(self, kernel_id=None, path=None, **kwargs):
>         kernel_id = 1
>         km = KernelManager(kernel_name='python3')
>         kc = km.client()
>         kc.load_connection_file(
> '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')
>         kc.start_channels()
>         try:
>             kc.wait_for_ready()
>         except RuntimeError:
>             kc.stop_channels()
>             raise
>         raise gen.Return(kernel_id)
>
> but when I create a new notebook I get the error
>
> [E 13:08:17.169 NotebookApp] Unhandled error in API request               
>                                                                             
>                                                  
>     Traceback (most recent call last):                                   
>                                                                             
>      
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/base/handlers.py"
> , line 516, in wrapper                                                   
>                           
>         result = yield gen.maybe_future(method(self, *args, **kwargs))   
>                                                                             
>                    
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", 
> line 1055, in run                                                         
>                   
>         value = future.result()                                           
>                                                                             
>                                                  
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
> , line 238, in result                                                     
>                                       
>         raise_exc_info(self._exc_info)                                   
>                                                                             
>        
>       File "<string>", line 4, in raise_exc_info                         
>                                                                             
>      
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", 
> line 1063, in run                             
>         yielded = self.gen.throw(*exc_info)                               
>                                                                      
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/handlers.py"
> , line 75, in post                                                       
>                       
>         type=mtype))                                                     
>                                                                             
>                                                
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", 
> line 1055, in run                                                         
>           
>         value = future.result()                                           
>                                                                             
>                                                  
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
> , line 238, in result                                                     
>  
>         raise_exc_info(self._exc_info)                                   
>                          
>       File "<string>", line 4, in raise_exc_info                         
>                                                                             
>                          
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", 
> line 1069, in run                                                        
>         yielded = self.gen.send(value)                                   
>                           
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
> , line 81, in create_session                                             
>                 
>         self.save_session(session_id, path=path, name=name, type=type, 
> kernel_id=kernel_id)                                
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
> , line 125, in save_session                                               
>                
>         return self.get_session(session_id=session_id)                   
>                             
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
> , line 170, in get_session
>         return self.row_to_model(row)                                     
>                
>       File 
> "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
> , line 209, in row_to_model                                               
>                
>         raise KeyError                                                   
>                                                  
>     KeyError                                                             
>                                                                             
>             
> [E 13:08:17.171 NotebookApp] {                                           
>                                    
>       "Host": "localhost:8888",                                           
>                                                
>       "Connection": "keep-alive",                                         
>                          
>       "Content-Length": "96",                                             
>                
>       "Accept": "application/json, text/javascript, */*; q=0.01",         
>                                         
>       "Origin": "http://localhost:8888";,                                 
>                           
>       "X-Requested-With": "XMLHttpRequest",                               
>                                                                             
>             
>       "X-Xsrftoken": 
> "2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005",                 
>                                                                             
>                                                          
>       "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) 
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
> ,                                                                         
>              
>       "Content-Type": "application/json",                                 
>                                                                             
>     
>       "Referer": "http://localhost:8888/notebooks/Kernel%20Client.ipynb";, 
>                                                                             
>                                         
>       "Accept-Encoding": "gzip, deflate, br",                             
>                                                                             
>                   
>       "Accept-Language": "en-US,en;q=0.9",                               
>                                                                             
>                          
>       "Cookie": 
> "_xsrf=2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005; 
> username-localhost-8889=\"2|1:0|10:1517089627|23:username-localhost-8889|44:M2NiNzEzOGRjOGVhNDhlMzk0YjRlZGI2OWQ4M2RkZjk=|fcb4100a390a3d9608150659dc573ec135609a1e778f386e293b029e1fe016e3\";
>  
> user
> name-localhost-8890=\"2|1:0|10:1518302103|23:username-localhost-8890|44:OTdjNGRhZGRjYTRlNGMxZjg0YzY4ZDIzZjU1NjEzYTQ=|6c9c7647d95641d6f2d7375d9d45130a96e9ed9ec690b2bca01b6652c23ccd05\";
>  
> username-localhost-8888=\"2|1:0|10:1518555822|23:username-localhost-8888|44:M2M3OTdhYTh
>
> iNjQ2NDE3ZWEyZjA3Mjk2MzdjMjZlMmI=|e2c3fe789c16638c5fa4573a9d7626b87a00cb4d33ed1488122c6543b8832398\""
>  
>                                                       
>     }                                   
>
> presumably because I did not call super(MappingKernelManager, 
> self).start_kernel(**kwargs) which presumably does some important setup 
> work.
>
> I *did* get a hacky solution "working" by the end of the day, though! I 
> put the following code snippet in a jupyter notebook.
>
> from jupyter_client import BlockingKernelClient
>
> client = BlockingKernelClient()
> client.load_connection_file(
> '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')
> client.start_channels()
>
> Then I am able to do client.execute_interactive(code) as many times as I 
> want!
>
> Using this approach one way forward would be to create a magic command 
> that wraps code in a call to client.execute_interactive(). However for a 
> single notebook that would require having to put the magic inside every 
> code cell, which I would rather avoid if possible.
>
> So my question is can I start a new jupyter notebook and attach it to a 
> running IPython kernel with configurables alone? If so am I close? If not 
> what would be the most promising direction to proceed? The closest example 
> I can think to study is nb2kg <https://github.com/jupyter-incubator/nb2kg> 
> which 
> allows a jupyter notebook to attach to a remote kernel. However that kernel 
> was started by a jupyter notebook so it still might not be close enough for 
> my use case to solve my problem
>
> I truly appreciate any help I can get as I am running out of ideas.
>
> On Tuesday, February 13, 2018 at 1:28:46 PM UTC-8, Edward Banner wrote:
>>
>> Hi Roland,
>>
>> Thanks for your response. Your suggestion makes sense. But does that mean 
>> going with a custom server extension 
>> <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> 
>> is the way to go?
>>
>> I spent yesterday attempting to accomplish this solely though jupyter 
>> configuration options. For a running kernel with information
>>
>> {
>>   "shell_port": 63391,
>>   "iopub_port": 63394,
>>   "stdin_port": 63392,
>>   "control_port": 63393,
>>   "hb_port": 63396,
>>   "ip": "127.0.0.1",
>>   "key": "fff108f8-b1bf9b17aa337ce7d3d14565",
>>   "transport": "tcp",
>>   "signature_scheme": "hmac-sha256",
>>   "kernel_name": ""
>> }
>>
>> I set the following variables in jupyter_notebook_config.py.
>>
>> c.ConnectionFileMixin.connection_file = 
>> '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json'
>> c.ConnectionFileMixin.control_port = 63393
>> c.ConnectionFileMixin.iopub_port = 63394
>> c.ConnectionFileMixin.stdin_port = 63392
>> c.ConnectionFileMixin.hb_port = 63396
>> c.ConnectionFileMixin.ip = '127.0.0.1'
>> c.ConnectionFileMixin.shell_port = 63391
>> c.ConnectionFileMixin.transport = 'tcp'
>>
>> However when I created a new notebook I get errors like
>>
>> [I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5)
>> [D 12:53:44.982 NotebookApp] Starting kernel: [
>> '/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 
>> 'ipykernel_launcher', '-f', 
>> '/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json'
>> ]
>> [D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393
>> Traceback (most recent call last):
>>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
>> 193, in _run_module_as_main
>>     "__main__", mod_spec)
>>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
>> 85, in _run_code
>>     exec(code, run_globals)
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py"
>> , line 16, in <module>
>>     app.launch_new_instance()
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
>> , line 657, in launch_instance
>>     app.initialize(argv)
>>   File "<decorator-gen-123>", line 2, in initialize
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
>> , line 87, in catch_config_error
>>     return method(app, *args, **kwargs)
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
>> , line 448, in initialize
>>     self.init_sockets()
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
>> , line 238, in init_sockets
>>     self.shell_port = self._bind_socket(self.shell_socket, self.
>> shell_port)
>>   File 
>> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
>> , line 180, in _bind_socket
>>     s.bind("tcp://%s:%i" % (self.ip, port))
>>   File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.
>> socket.Socket.bind (zmq/backend/cython/socket.c:5653)
>>   File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.
>> checkrc._check_rc (zmq/backend/cython/socket.c:10014)
>> zmq.error.ZMQError: Address already in use
>>
>> until it hits the maximum number of retries. Presumably this is because 
>> the ipykernel command attempts to create a *new* socket but since it 
>> already exists it fails to do so.
>>
>> Thinking that I could "trick" jupyter into not using ipykernel and just 
>> use the existing sockets I set
>>
>> c.KernelManager.kernel_cmd = [
>> '/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py']
>>
>> where bar.py contains
>>
>> while True:
>>     pass
>>
>> After creating another new notebook I get
>>
>> [D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4
>> -4d98-a0c6-1f227627f784/channels
>> [D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391
>> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394
>> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392
>>
>> which gave me hope! But then when I tried to execute a cell it just 
>> hangs. I did notice that http://localhost:8888/api/kernels has the 
>> following information.
>>
>> <span
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/eb394188-0acc-4c17-ab4b-2ecd8e4dbc8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to