*Arg! I can't seem to get formatting down on google forums. I apologize. 
Here is my response in full.*

Hi Roland,

Thanks for your response. Your suggestion makes sense. But does that mean 
going with a custom server extension 
<http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> is 
the way to go?

I spent yesterday attempting to accomplish this solely though jupyter 
configuration options. For a running kernel with information

{
  "shell_port": 63391,
  "iopub_port": 63394,
  "stdin_port": 63392,
  "control_port": 63393,
  "hb_port": 63396,
  "ip": "127.0.0.1",
  "key": "fff108f8-b1bf9b17aa337ce7d3d14565",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

I set the following variables in jupyter_notebook_config.py.

c.ConnectionFileMixin.connection_file = 
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json'
c.ConnectionFileMixin.control_port = 63393
c.ConnectionFileMixin.iopub_port = 63394
c.ConnectionFileMixin.stdin_port = 63392
c.ConnectionFileMixin.hb_port = 63396
c.ConnectionFileMixin.ip = '127.0.0.1'
c.ConnectionFileMixin.shell_port = 63391
c.ConnectionFileMixin.transport = 'tcp'

However when I created a new notebook I get errors like

[I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5)
[D 12:53:44.982 NotebookApp] Starting kernel: [
'/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 'ipykernel_launcher',
 '-f', 
'/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json'
]
[D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393
Traceback (most recent call last):
  File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 193
, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 85,
 in _run_code
    exec(code, run_globals)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py"
, line 16, in <module>
    app.launch_new_instance()
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
, line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-123>", line 2, in initialize
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
, line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 448, in initialize
    self.init_sockets()
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 238, in init_sockets
    self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 180, in _bind_socket
    s.bind("tcp://%s:%i" % (self.ip, port))
  File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.
socket.Socket.bind (zmq/backend/cython/socket.c:5653)
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.
checkrc._check_rc (zmq/backend/cython/socket.c:10014)
zmq.error.ZMQError: Address already in use

until it hits the maximum number of retries. Presumably this is because the 
ipykernel command attempts to create a *new* socket but since it already 
exists it fails to do so.

Thinking that I could "trick" jupyter into not using ipykernel and just use 
the existing sockets I set

c.KernelManager.kernel_cmd = [
'/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py']

where bar.py contains

while True:
    pass

After creating another new notebook I get

[D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4-
4d98-a0c6-1f227627f784/channels
[D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391
[D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394
[D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392

which gave me hope! But then when I tried to execute a cell it just hangs. 
I did notice that http://localhost:8888/api/kernels has the following 
information.

[
    {
        id: "9b260abf-072f-48e2-b5f0-213d934418a2",
        name: "python3",
        last_activity: "2018-02-13T21:10:41.460182Z",
        execution_state: "starting",
        connections: 0
    }
]

However the execution_state being stuck at "starting" and connections being 
at 0 make me think that bar.py isn't quite doing enough.

Digging into the code a bit I found the object returned from the call to 
ipykernel is a subprocess.Popen() object. But I couldn't find anywhere else 
in the code except KernelManager.is_alive() where Popen.poll() is called on 
it 
<https://github.com/jupyter/jupyter_client/blob/38b81cc07ddf70c2f534f11c4f4ef0103bfd5fe2/jupyter_client/multikernelmanager.py#L204-L205>
.

I also tried creating a custom kernel manager class and set

c.NotebookApp.kernel_manager_class = 
'kernelmanager.ExistingMappingKernelManager'

where kernelmanager.ExistingMappingKernelManager is the class

from tornado import gen, web
from jupyter_client import KernelManager
from notebook.services.kernels.kernelmanager import MappingKernelManager

class ExistingMappingKernelManager(MappingKernelManager):
    """A KernelManager that just connects to an existing kernel."""

    @gen.coroutine
    def start_kernel(self, kernel_id=None, path=None, **kwargs):
        kernel_id = 1
        km = KernelManager(kernel_name='python3')
        kc = km.client()
        kc.load_connection_file(
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')
        kc.start_channels()
        try:
            kc.wait_for_ready()
        except RuntimeError:
            kc.stop_channels()
            raise
        raise gen.Return(kernel_id)

but when I create a new notebook I get the error

[E 13:08:17.169 NotebookApp] Unhandled error in API request                 
                                                                            
                                               
    Traceback (most recent call last):                                     
                                                                            
   
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/base/handlers.py"
, line 516, in wrapper                                                     
                        
        result = yield gen.maybe_future(method(self, *args, **kwargs))     
                                                                            
                 
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1055, in run                                                               
            
        value = future.result()                                             
                                                                            
                                               
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
, line 238, in result                                                       
                                    
        raise_exc_info(self._exc_info)                                     
                                                                            
     
      File "<string>", line 4, in raise_exc_info                           
                                                                            
   
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1063, in run                             
        yielded = self.gen.throw(*exc_info)                                 
                                                                   
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/handlers.py"
, line 75, in post                                                         
                    
        type=mtype))                                                       
                                                                            
                                             
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1055, in run                                                               
    
        value = future.result()                                             
                                                                            
                                               
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
, line 238, in result                                                      
        raise_exc_info(self._exc_info)                                     
                       
      File "<string>", line 4, in raise_exc_info                           
                                                                            
                       
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1069, in run                                                        
        yielded = self.gen.send(value)                                     
                        
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 81, in create_session                                               
              
        self.save_session(session_id, path=path, name=name, type=type, 
kernel_id=kernel_id)                                
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 125, in save_session                                                 
             
        return self.get_session(session_id=session_id)                     
                          
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 170, in get_session
        return self.row_to_model(row)                                       
             
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 209, in row_to_model                                                 
             
        raise KeyError                                                     
                                               
    KeyError                                                               
                                                                            
          
[E 13:08:17.171 NotebookApp] {                                             
                                 
      "Host": "localhost:8888",                                             
                                             
      "Connection": "keep-alive",                                           
                       
      "Content-Length": "96",                                               
             
      "Accept": "application/json, text/javascript, */*; q=0.01",           
                                      
      "Origin": "http://localhost:8888";,                                   
                        
      "X-Requested-With": "XMLHttpRequest",                                 
                                                                            
          
      "X-Xsrftoken": 
"2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005",                   
                                                                            
                                                       
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36", 
                                                                            
         
      "Content-Type": "application/json",                                   
                                                                            
  
      "Referer": "http://localhost:8888/notebooks/Kernel%20Client.ipynb";,   
                                                                            
                                      
      "Accept-Encoding": "gzip, deflate, br",                               
                                                                            
                
      "Accept-Language": "en-US,en;q=0.9",                                 
                                                                            
                       
      "Cookie": "_xsrf=2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005; 
username-localhost-8889=\"2|1:0|10:1517089627|23:username-localhost-8889|44:M2NiNzEzOGRjOGVhNDhlMzk0YjRlZGI2OWQ4M2RkZjk=|fcb4100a390a3d9608150659dc573ec135609a1e778f386e293b029e1fe016e3\";
 
user
name-localhost-8890=\"2|1:0|10:1518302103|23:username-localhost-8890|44:OTdjNGRhZGRjYTRlNGMxZjg0YzY4ZDIzZjU1NjEzYTQ=|6c9c7647d95641d6f2d7375d9d45130a96e9ed9ec690b2bca01b6652c23ccd05\";
 
username-localhost-8888=\"2|1:0|10:1518555822|23:username-localhost-8888|44:M2M3OTdhYTh
iNjQ2NDE3ZWEyZjA3Mjk2MzdjMjZlMmI=|e2c3fe789c16638c5fa4573a9d7626b87a00cb4d33ed1488122c6543b8832398\""
 
                                                      
    }                                   

presumably because I did not call super(MappingKernelManager, 
self).start_kernel(**kwargs) which presumably does some important setup 
work.

I *did* get a hacky solution "working" by the end of the day, though! I put 
the following code snippet in a jupyter notebook.

from jupyter_client import BlockingKernelClient

client = BlockingKernelClient()
client.load_connection_file(
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')
client.start_channels()

Then I am able to do client.execute_interactive(code) as many times as I 
want!

Using this approach one way forward would be to create a magic command that 
wraps code in a call to client.execute_interactive(). However for a single 
notebook that would require having to put the magic inside every code cell, 
which I would rather avoid if possible.

So my question is can I start a new jupyter notebook and attach it to a 
running IPython kernel with configurables alone? If so am I close? If not 
what would be the most promising direction to proceed? The closest example 
I can think to study is nb2kg <https://github.com/jupyter-incubator/nb2kg> 
which 
allows a jupyter notebook to attach to a remote kernel. However that kernel 
was started by a jupyter notebook so it still might not be close enough for 
my use case to solve my problem

I truly appreciate any help I can get as I am running out of ideas.

On Tuesday, February 13, 2018 at 1:28:46 PM UTC-8, Edward Banner wrote:
>
> Hi Roland,
>
> Thanks for your response. Your suggestion makes sense. But does that mean 
> going with a custom server extension 
> <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> 
> is the way to go?
>
> I spent yesterday attempting to accomplish this solely though jupyter 
> configuration options. For a running kernel with information
>
> {
>   "shell_port": 63391,
>   "iopub_port": 63394,
>   "stdin_port": 63392,
>   "control_port": 63393,
>   "hb_port": 63396,
>   "ip": "127.0.0.1",
>   "key": "fff108f8-b1bf9b17aa337ce7d3d14565",
>   "transport": "tcp",
>   "signature_scheme": "hmac-sha256",
>   "kernel_name": ""
> }
>
> I set the following variables in jupyter_notebook_config.py.
>
> c.ConnectionFileMixin.connection_file = 
> '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json'
> c.ConnectionFileMixin.control_port = 63393
> c.ConnectionFileMixin.iopub_port = 63394
> c.ConnectionFileMixin.stdin_port = 63392
> c.ConnectionFileMixin.hb_port = 63396
> c.ConnectionFileMixin.ip = '127.0.0.1'
> c.ConnectionFileMixin.shell_port = 63391
> c.ConnectionFileMixin.transport = 'tcp'
>
> However when I created a new notebook I get errors like
>
> [I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5)
> [D 12:53:44.982 NotebookApp] Starting kernel: [
> '/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 
> 'ipykernel_launcher', '-f', 
> '/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json'
> ]
> [D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393
> Traceback (most recent call last):
>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
> 193, in _run_module_as_main
>     "__main__", mod_spec)
>   File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 
> 85, in _run_code
>     exec(code, run_globals)
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py"
> , line 16, in <module>
>     app.launch_new_instance()
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
> , line 657, in launch_instance
>     app.initialize(argv)
>   File "<decorator-gen-123>", line 2, in initialize
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
> , line 87, in catch_config_error
>     return method(app, *args, **kwargs)
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 448, in initialize
>     self.init_sockets()
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 238, in init_sockets
>     self.shell_port = self._bind_socket(self.shell_socket, self.shell_port
> )
>   File 
> "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
> , line 180, in _bind_socket
>     s.bind("tcp://%s:%i" % (self.ip, port))
>   File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.
> socket.Socket.bind (zmq/backend/cython/socket.c:5653)
>   File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.
> checkrc._check_rc (zmq/backend/cython/socket.c:10014)
> zmq.error.ZMQError: Address already in use
>
> until it hits the maximum number of retries. Presumably this is because 
> the ipykernel command attempts to create a *new* socket but since it 
> already exists it fails to do so.
>
> Thinking that I could "trick" jupyter into not using ipykernel and just 
> use the existing sockets I set
>
> c.KernelManager.kernel_cmd = [
> '/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py']
>
> where bar.py contains
>
> while True:
>     pass
>
> After creating another new notebook I get
>
> [D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4-
> 4d98-a0c6-1f227627f784/channels
> [D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391
> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394
> [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392
>
> which gave me hope! But then when I tried to execute a cell it just hangs. 
> I did notice that http://localhost:8888/api/kernels has the following 
> information.
>
> <span
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/64cc3540-1cea-4260-9d85-a5122300419d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to