The container is shutting down because the jupyterhub it is issuing the
shutdown command after it fails connect to the container for 30
seconds.Here are the logs for the container.
root@jupyterhub:/srv/jupyterhub# docker logs -f
jupyter-zozo6015.1.u5030s66gvd7rkt4af4bn6kux
Executing the command: jupyterhub-singleuser --ip="0.0.0.0" --port=8888
[W 2020-02-18 14:27:50.746 SingleUserNotebookApp configurable:168] Config
option `open_browser` not recognized by `SingleUserNotebookApp`. Did you
mean `browser`?
[I 2020-02-18 14:27:52.760 SingleUserNotebookApp extension:158] JupyterLab
extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 2020-02-18 14:27:52.761 SingleUserNotebookApp extension:159] JupyterLab
application directory is /opt/conda/share/jupyter/lab
[I 2020-02-18 14:27:53.666 SingleUserNotebookApp singleuser:406] Starting
jupyterhub-singleuser server version 0.9.6
[I 2020-02-18 14:27:53.694 SingleUserNotebookApp notebookapp:1924] Serving
notebooks from local directory: /home/jovyan
[I 2020-02-18 14:27:53.696 SingleUserNotebookApp notebookapp:1924] The
Jupyter Notebook is running at:
[I 2020-02-18 14:27:53.697 SingleUserNotebookApp notebookapp:1924] http:
//6ea2c6d31f52:8888/user/zozo6015/
[I 2020-02-18 14:27:53.697 SingleUserNotebookApp notebookapp:1925] Use
Control-C to stop this server and shut down all kernels (twice to skip
confirmation).
[C 2020-02-18 14:28:20.462 SingleUserNotebookApp notebookapp:1615] received
signal 15, stopping
[I 2020-02-18 14:28:20.472 SingleUserNotebookApp notebookapp:1784] Shutting
down 0 kernels
Regards,
Peter
On Tuesday, 18 February 2020 20:56:50 UTC+2, Jason Anderson wrote:
>
> Hi Peter, those are the logs for JupyterHub itself, but do not contain
> logs for the individual container (jupyter-zozo6015) - you will probably
> have to use `docker logs` to get those. Maybe that will contain some
> additional information. It looks like JupyterHub is doing its job ok, but
> the user container is immediately exiting (ExitCode: 0 in your posted logs.)
>
> /Jason
>
> On 2/18/20 11:55 AM, Peter Zoltan Keresztes wrote:
>
> Already created the overlay network, I get the services up but for a very
> short time when the services ends with the following error:
>
>
> [W 2020-02-18 09:51:34.551 JupyterHub utils:192] Failed to connect to http
> ://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service not
> known)
> [W 2020-02-18 09:51:39.566 JupyterHub utils:192] Failed to connect to http
> ://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service not
> known)
> [W 2020-02-18 09:51:41.553 JupyterHub utils:192] Failed to connect to http
> ://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service not
> known)
> [W 2020-02-18 09:51:41.555 JupyterHub user:510] zozo6015's server never
> showed up at http://(:8888/user/zozo6015/ after 30 seconds. Giving up
> [D 2020-02-18 09:51:41.557 JupyterHub swarmspawner:144] Getting task of
> service 'jupyter-zozo6015'
> [D 2020-02-18 09:51:41.558 JupyterHub dockerspawner:777] Getting container
> 'jupyter-zozo6015'
> [D 2020-02-18 09:51:41.593 JupyterHub swarmspawner:133] Service 9ni1q9z
> status: {'ContainerStatus': {'ContainerID': '
> 9480724f2d31d9fe8a784c99a17441f024ceba683951ad9be4e71766cf8499b7',
> 'ExitCode': 0,
> 'PID': 18817},
> 'Message': 'started',
> 'PortStatus': {},
> 'State': 'running',
> 'Timestamp': '2020-02-18T14:51:10.859533481Z'}
> [I 2020-02-18 09:51:41.596 JupyterHub dockerspawner:1107] Stopping service
> jupyter-zozo6015 (id: 9ni1q9z)
> [I 2020-02-18 09:51:41.597 JupyterHub swarmspawner:232] Removing service
> 9ni1q9zsp99khlvpbnl3x4qyx
> [D 2020-02-18 09:51:41.653 JupyterHub user:578] Deleting oauth client
> jupyterhub-user-zozo6015
> [E 2020-02-18 09:51:41.674 JupyterHub gen:974] Exception in Future <Task
> finished coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn()
> done, defined at
> /srv/.jhub/lib/python3.5/site-packages/jupyterhub/handlers/base.py:629>
> exception=TimeoutError("Server at
> http://jupyter-zozo6015:8888/user/zozo6015/ didn't respond in 30 seconds",)>
> after timeout
> Traceback (most recent call last):
> File "/srv/.jhub/lib/python3.5/site-packages/tornado/gen.py", line
> 970, in error_callback
> future.result()
> File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
> raise self._exception
> File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
> result = coro.send(None)
> File "/srv/.jhub/lib/python3.5/site-packages/jupyterhub/handlers/
> base.py", line 636, in finish_user_spawn
> await spawn_future
> File "/srv/.jhub/lib/python3.5/site-packages/jupyterhub/user.py",
> line 528, in spawn
> raise e
> File "/srv/.jhub/lib/python3.5/site-packages/jupyterhub/user.py",
> line 502, in spawn
> resp = await server.wait_up(http=True,
> timeout=spawner.http_timeout)
> File "/srv/.jhub/lib/python3.5/site-packages/jupyterhub/utils.py",
> line 197, in wait_for_http_server
> timeout=timeout
> File "/srv/.jhub/lib/python3.5/site-packages/jupyterhub/utils.py",
> line 155, in exponential_backoff
> raise TimeoutError(fail_message)
> TimeoutError: Server at http://jupyter-zozo6015:8888/user/zozo6015/
> didn't respond in 30 seconds
>
>
> Regards,
> Peter
>
> On Tuesday, 18 February 2020 18:30:38 UTC+2, Jason Anderson wrote:
>>
>> Hi Peter,
>>
>> I don't have much experience with Docker Swarm, but this line jumped out
>> to me:
>>
>> root@jupyterhub:~# docker service ls
>>
>> ID NAME MODE REPLICAS
>> IMAGE PORTS
>>
>> wtq7n8hvymwf jupyter-zozo6015 replicated 0/1
>> jupyterhub/singleuser:0.9
>>
>> Replicas still show that 0 are up. I would try to check out the
>> containers that are supposed to be running (`docker container ls -a`) to
>> see what is happening. I did a quick StackOverflow search, you might find
>> some help/ideas here[1].
>>
>> Long story short, I am not sure it is a problem specifically with the
>> JupyterHub setup. Though, I'm not sure what parts need to be configured for
>> Swarm to work. Another thing the docs[2] mention is that the network needs
>> to exist. Is it created already?
>>
>> docker network create --driver overlay jupyterhub_network
>>
>>
>> Hope that helps,
>> /Jason
>>
>> [1]:
>> https://stackoverflow.com/questions/39284233/docker-service-replicas-remain-0-1
>> [2]:
>> https://github.com/jupyterhub/dockerspawner/blob/master/README.md#swarmspawner
>>
>> On 2/18/20 4:29 AM, Peter Zoltan Keresztes wrote:
>>
>> Is there anyone in here to help me with this issue I am struggling for
>> more than a week with?
>>
>>
>>
>> On Thursday, 13 February 2020 23:19:52 UTC+2, Peter Zoltan Keresztes
>> wrote:
>>>
>>> hello,
>>>
>>> I am trying to convert the dockerspawner setup into swarmspawner. All I
>>> am getting is timeout.
>>>
>>> The configuration I am using looking like this:
>>>
>>> # Configuration file for JupyterHub.*
>>>
>>> c.Application.log_level = 30
>>>
>>> c.JupyterHub.ssl_key = '/etc/letsencrypt/live/jhub.z0z0.tk/privkey.pem'
>>>
>>> c.JupyterHub.ssl_cert = '/etc/letsencrypt/live/
>>> jhub.z0z0.tk/fullchain.pem'
>>>
>>> c.JupyterHub.port = 443
>>>
>>>
>>> #from oauthenticator.github import LocalGitHubOAuthenticator
>>>
>>> from oauthenticator.github import GitHubOAuthenticator
>>>
>>> c.JupyterHub.authenticator_class = GitHubOAuthenticator
>>>
>>> #c.LocalGitHubOAuthenticator.create_system_users = True
>>>
>>>
>>> import os
>>>
>>> c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'
>>>
>>> c.SwarmSpawner.network_name = 'jupyterhub_network'
>>>
>>> from jupyter_client.localinterfaces import public_ips
>>>
>>> c.JupyterHub.hub_ip = '0.0.0.0'
>>>
>>> c.JupyterHub.proxy_api_ip = '0.0.0.0'
>>>
>>> #c.SwarmSpawner.extra_placement_spec = { 'constraints' :
>>> ['node.role==worker'] }
>>>
>>>
>>>
>>> while I can see the service starting:
>>>
>>>
>>> root@jupyterhub:~# docker service ls
>>>
>>> ID NAME MODE REPLICAS
>>> IMAGE PORTS
>>>
>>> wtq7n8hvymwf jupyter-zozo6015 replicated 0/1
>>> jupyterhub/singleuser:0.9
>>>
>>>
>>> This is what I get in the logs:
>>>
>>>
>>> [W 2020-02-13 16:15:12.977 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:13.035 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:13.214 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:13.461 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:13.781 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:14.293 JupyterHub base:744] User zozo6015 is slow to
>>> become responsive (timeout=10)
>>>
>>> [W 2020-02-13 16:15:15.134 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:18.796 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:21.634 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:26.651 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:31.664 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:36.683 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:41.701 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:44.048 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:45.070 JupyterHub utils:192] Failed to connect to
>>> http://jupyter-zozo6015:8888/user/zozo6015/ ([Errno -2] Name or service
>>> not known)
>>>
>>> [W 2020-02-13 16:15:45.071 JupyterHub user:510] zozo6015's server never
>>> showed up at http://jupyter-zozo6015:8888/user/zozo6015/ after 30
>>> seconds. Giving up
>>>
>>> [E 2020-02-13 16:15:45.229 JupyterHub gen:974] Exception in Future <Task
>>> finished coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn()
>>> done, defined at
>>> /srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/handlers/base.py:629>
>>>
>>> exception=TimeoutError("Server at
>>> http://jupyter-zozo6015:8888/user/zozo6015/ didn't respond in 30
>>> seconds",)> after timeout
>>>
>>> Traceback (most recent call last):
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/tornado/gen.py", line 970, in
>>> error_callback
>>>
>>> future.result()
>>>
>>> File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result
>>>
>>> raise self._exception
>>>
>>> File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
>>>
>>> result = coro.send(None)
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/handlers/base.py",
>>> line 636, in finish_user_spawn
>>>
>>> await spawn_future
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/user.py", line
>>> 528, in spawn
>>>
>>> raise e
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/user.py", line
>>> 502, in spawn
>>>
>>> resp = await server.wait_up(http=True,
>>> timeout=spawner.http_timeout)
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/utils.py", line
>>> 197, in wait_for_http_server
>>>
>>> timeout=timeout
>>>
>>> File
>>> "/srv/.jupyterhub/lib/python3.5/site-packages/jupyterhub/utils.py", line
>>> 155, in exponential_backoff
>>>
>>> raise TimeoutError(fail_message)
>>>
>>> TimeoutError: Server at http://jupyter-zozo6015:8888/user/zozo6015/
>>> didn't respond in 30 seconds
>>>
>>>
>>>
>>> 16:17:39.648 [ConfigProxy] info: 200 GET /api/routes
>>>
>>>
>>> Can I get any help to make this up and running?
>>>
>>>
>>> Regards,
>>>
>>> Peter
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Project Jupyter" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jupyter/57a3ad96-a24f-4925-964c-3b911ed596a5%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jupyter/57a3ad96-a24f-4925-964c-3b911ed596a5%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/3c901a13-6b0e-4d8b-93f7-6d2e45c95a13%40googlegroups.com
>
> <https://groups.google.com/d/msgid/jupyter/3c901a13-6b0e-4d8b-93f7-6d2e45c95a13%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jupyter/8d6df650-600a-460d-92bd-7bd17f98f57f%40googlegroups.com.
