Hi Roland,

Thanks for your response. Your suggestion makes sense. But does that mean 
going with a custom server extension 
<http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> 
is the way to go?

I spent yesterday attempting to accomplish this solely though jupyter 
configuration options. For a running kernel with information

{
  "shell_port": 63391,
  "iopub_port": 63394,
  "stdin_port": 63392,
  "control_port": 63393,
  "hb_port": 63396,
  "ip": "127.0.0.1",
  "key": "fff108f8-b1bf9b17aa337ce7d3d14565",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

I set the following variables in jupyter_notebook_config.py.

c.ConnectionFileMixin.connection_file = 
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json'
c.ConnectionFileMixin.control_port = 63393
c.ConnectionFileMixin.iopub_port = 63394
c.ConnectionFileMixin.stdin_port = 63392
c.ConnectionFileMixin.hb_port = 63396
c.ConnectionFileMixin.ip = '127.0.0.1'
c.ConnectionFileMixin.shell_port = 63391
c.ConnectionFileMixin.transport = 'tcp'

However when I created a new notebook I get errors like

[I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5)
[D 12:53:44.982 NotebookApp] Starting kernel: [
'/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 'ipykernel_launcher', 
'-f', 
'/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json'
]
[D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393
Traceback (most recent call last):
  File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 193
, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 85, 
in _run_code
    exec(code, run_globals)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py"
, line 16, in <module>
    app.launch_new_instance()
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
, line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-123>", line 2, in initialize
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py"
, line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 448, in initialize
    self.init_sockets()
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 238, in init_sockets
    self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
  File 
"/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py"
, line 180, in _bind_socket
    s.bind("tcp://%s:%i" % (self.ip, port))
  File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython.
socket.Socket.bind (zmq/backend/cython/socket.c:5653)
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.
checkrc._check_rc (zmq/backend/cython/socket.c:10014)
zmq.error.ZMQError: Address already in use

until it hits the maximum number of retries. Presumably this is because the 
ipykernel command attempts to create a *new* socket but since it already 
exists it fails to do so.

Thinking that I could "trick" jupyter into not using ipykernel and just use 
the existing sockets I set

c.KernelManager.kernel_cmd = [
'/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py']

where bar.py contains

while True:
    pass

After creating another new notebook I get

[D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4-
4d98-a0c6-1f227627f784/channels
[D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391
[D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394
[D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392

which gave me hope! But then when I tried to execute a cell it just hangs. 
I did notice that http://localhost:8888/api/kernels has the following 
information.

[
    {
        id: "9b260abf-072f-48e2-b5f0-213d934418a2",
        name: "python3",
        last_activity: "2018-02-13T21:10:41.460182Z",
        execution_state: "starting",
        connections: 0
    }
]

The execution_state being "starting" and connections being at 0 makes me 
think that bar.py isn't quite doing enough.

Digging into the code I discovered that the object returned from the call 
to ipykernel is a subprocess.Popen() object but all I could find is that 
Popen.poll() 
is called on it 
<https://github.com/jupyter/jupyter_client/blob/38b81cc07ddf70c2f534f11c4f4ef0103bfd5fe2/jupyter_client/multikernelmanager.py#L204-L205>
 but 
I couldn't find any other interfaces it has to support.

I also tried setting c.NotebookApp.kernel_manager_class to my own custom 
kernel manager class

from tornado import gen, web
from jupyter_client import KernelManager
from notebook.services.kernels.kernelmanager import MappingKernelManager

class ExistingMappingKernelManager(MappingKernelManager):
    """A KernelManager that just connects to an existing kernel."""

    @gen.coroutine
    def start_kernel(self, kernel_id=None, path=None, **kwargs):
        kernel_id = 1
        km = KernelManager(kernel_name='python3')
        kc = km.client()
        kc.load_connection_file(
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')
        kc.start_channels()
        try:
            kc.wait_for_ready()
        except RuntimeError:
            kc.stop_channels()
            raise
        raise gen.Return(kernel_id)

but when I create a new notebook I get

[E 13:03:58.692 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/base/handlers.py"
, line 516, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1055, in run
        value = future.result()
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
, line 238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1063, in run
        yielded = self.gen.throw(*exc_info)
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/handlers.py"
, line 75, in post
        type=mtype))
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1055, in run
        value = future.result()
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py"
, line 238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 
1069, in run
        yielded = self.gen.send(value)
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 81, in create_session
        self.save_session(session_id, path=path, name=name, type=type, 
kernel_id=kernel_id)
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 125, in save_session
        return self.get_session(session_id=session_id)
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 170, in get_session
        return self.row_to_model(row)
      File 
"/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py"
, line 209, in row_to_model
        raise KeyError
    KeyError
[E 13:03:58.708 NotebookApp] {
      "Host": "localhost:8888",
      "Connection": "keep-alive",
      "Content-Length": "93",
      "Accept": "application/json, text/javascript, */*; q=0.01",
      "Origin": "http://localhost:8888";,
      "X-Requested-With": "XMLHttpRequest",
      "X-Xsrftoken": 
"2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36",
      "Content-Type": "application/json",
      "Referer": 
"http://localhost:8888/notebooks/Untitled74.ipynb?kernel_name=python3";,
      "Accept-Encoding": "gzip, deflate, br",
      "Accept-Language": "en-US,en;q=0.9",
      "Cookie": "_xsrf=2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005; 
username-localhost-8889=\"2|1:0|10:1517089627|23:username-localhost-8889|44:M2NiNzEzOGRjOGVhNDhlMzk0YjRlZGI2OWQ4M2RkZjk=|fcb4100a390a3d9608150659dc573ec135609a1e778f386e293b029e1fe016e3\";
 
username-localhost-8890=\"2|1:0|10:1518302103|23:username-localhost-8890|44:OTdjNGRhZGRjYTRlNGMxZjg0YzY4ZDIzZjU1NjEzYTQ=|6c9c7647d95641d6f2d7375d9d45130a96e9ed9ec690b2bca01b6652c23ccd05\";
 
username-localhost-8888=\"2|1:0|10:1518555822|23:username-localhost-8888|44:M2M3OTdhYThiNjQ2NDE3ZWEyZjA3Mjk2MzdjMjZlMmI=|e2c3fe789c16638c5fa4573a9d7626b87a00cb4d33ed1488122c6543b8832398\""
    }
[E 13:03:58.710 NotebookApp] 500 POST /api/sessions (::1) 289.25ms referer=
http://localhost:8888/notebooks/Untitled74.ipynb?kernel_name=python3

Presumably this is because I failed to call super(MappingKernelManager, 
self).start_kernel(**kwargs) which presumably does some necessary setup 
work that I skipped.

At the end of the day I *did* get a very hacky solution working though with 
the following code snippet in a jupyter notebook

from jupyter_client import BlockingKernelClient

client = BlockingKernelClient()
client.load_connection_file(
'/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json')

Then I can do client.execute_interactive(code) as many times as I want!

One way to go is to make a magic command out of this which wraps cell code 
and passes it to client.execute_interactive() so I don't have to type 
client.execute_interactive() each time. However I still need to insert the 
magic into every single code cell, which I would rather not do.

So my question is can I attach a new jupyter notebook to an external 
IPython kernel solely though configurables? Am I close? If not what would 
be the best way to go? The closest example I can find to study is nb2kg 
<https://github.com/jupyter-incubator/nb2kg> which allows one to attach a 
jupyter notebook to a remote kernel. However the kernels were still created 
by a jupyter notebook server so I am afraid it is still not close enough to 
help me solve my use case.

I truly appreciate any help as I think I am running out of ideas to try.

On Sunday, February 11, 2018 at 10:56:03 PM UTC-8, Roland Weber wrote:
>
> Hello Edward,
>
> through either making a custom server extension 
>> <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> 
>> or using the jupyter kernel gateway 
>> <https://github.com/jupyter/kernel_gateway>.
>>
>
> The kernel gateway in its websocket personality doesn't work with 
> notebooks at all.
> In its notebook personality, it runs a configured notebook with one kernel 
> to serve HTTP requests.
>
> Since your declared intention is to associate a notebook with a kernel you 
> started, I don't think that KG is of any use in your scenario.
>
> cheers,
>   Roland
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/beeee20a-a82c-4857-9199-420469c5bf0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to