Hi Dev,
I found some strange scenario on Notebook. I'm running jupyter notebook 4.1.0 version on Python 3.5.2 :: Anaconda 4.1.1 (64-bit) platform. Issue: If I enable N kernels, when I refresh Jupyter home page, then it's getting kernel spec N times and hence page refresh is too slow. Say for example: If I enable 10 custom kernels, when I refresh jupyter home page, it's taking 10+ seconds. When I see log, most of the time is taking in refreshing kernels again and again for 10 times as "Found kernel <> in /usr/local/share/jupyter/kernels". In logs, it show 100 times that kernel as found. Version: [22:48]:[root@myhost:~]# python -V Python 3.5.2 :: Anaconda 4.1.1 (64-bit) [22:48]:[root@myhost:~]# jupyter --version 4.1.0 Starting Jupyter with default Python Kernel: [22:48]:[root@myhost:~]# jupyter notebook --no-browser --ip 0.0.0.0 --port 9000 --debug Log: With only one Kernel by default: [D 22:40:45.377 NotebookApp] 200 GET /api/sessions?_=1513320006484 (10.196.177.146) 0.85ms [D 22:40:45.378 NotebookApp] 200 GET /api/terminals?_=1513320006485 (10.196.177.146) 0.42ms [D 22:40:45.459 NotebookApp] 200 GET /api/contents?type=directory&_=1513320006486 (10.196.177.146) 1.18ms [D 22:40:49.684 NotebookApp] Using contents: services/contents [D 22:40:49.685 NotebookApp] 304 GET /tree (10.196.177.146) 1.46ms [D 22:40:49.789 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.52ms [D 22:40:50.111 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.84ms [D 22:40:50.112 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.64ms [D 22:40:50.221 NotebookApp] 200 GET /api/config/common?_=1513320049961 (10.196.177.146) 2.28ms [D 22:40:50.223 NotebookApp] 200 GET /api/config/tree?_=1513320049960 (10.196.177.146) 1.93ms [D 22:40:50.225 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:40:50.745 NotebookApp] 304 GET /api/kernelspecs (10.196.177.146) 520.39ms [D 22:40:50.746 NotebookApp] 200 GET /api/terminals?_=1513320049962 (10.196.177.146) 0.71ms [D 22:40:50.747 NotebookApp] 200 GET /api/sessions?_=1513320049963 (10.196.177.146) 0.62ms [D 22:40:50.748 NotebookApp] 200 GET /api/terminals?_=1513320049964 (10.196.177.146) 0.42ms [D 22:40:50.993 NotebookApp] 200 GET /api/contents?type=directory&_=1513320049965 (10.196.177.146) 1.23ms Enabling 1st custom kernel [22:41]:[root@myhost:~]# jupyter-kernelspec install $SPARKMAGIC_PATH/sparkmagic/sparkmagic/kernels/sparkkernel [InstallKernelSpec] Installed kernelspec sparkkernel in /usr/local/share/jupyter/kernels/sparkkernel [22:41]:[root@myhost:~]# Log: After enabling 1st custom kernel, when I refresh home page [D 22:43:48.936 NotebookApp] 200 GET /api/sessions?_=1513320206761 (10.196.177.146) 0.88ms [D 22:43:48.938 NotebookApp] 200 GET /api/terminals?_=1513320206762 (10.196.177.146) 0.43ms [D 22:43:49.295 NotebookApp] 200 GET /api/contents?type=directory&_=1513320206763 (10.196.177.146) 1.17ms [D 22:44:05.654 NotebookApp] Using contents: services/contents [D 22:44:05.655 NotebookApp] 304 GET /tree (10.196.177.146) 1.56ms [D 22:44:05.769 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.55ms [D 22:44:06.154 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.50ms [D 22:44:06.160 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.61ms [D 22:44:06.252 NotebookApp] 200 GET /api/config/tree?_=1513320245977 (10.196.177.146) 2.08ms [D 22:44:06.255 NotebookApp] 200 GET /api/config/common?_=1513320245978 (10.196.177.146) 2.57ms [D 22:44:06.256 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:44:06.256 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:44:06.768 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:44:06.768 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:44:07.277 NotebookApp] 304 GET /api/kernelspecs (10.196.177.146) 1021.39ms [D 22:44:07.279 NotebookApp] 200 GET /api/terminals?_=1513320245979 (10.196.177.146) 0.71ms [D 22:44:07.280 NotebookApp] 200 GET /api/terminals?_=1513320245981 (10.196.177.146) 0.42ms [D 22:44:07.281 NotebookApp] 200 GET /api/sessions?_=1513320245980 (10.196.177.146) 0.60ms [D 22:44:07.602 NotebookApp] 200 GET /api/contents?type=directory&_=1513320245982 (10.196.177.146) 1.32ms Enabling 2nd custom kernel [22:41]:[root@myhost:~]# jupyter-kernelspec install $SPARKMAGIC_PATH/sparkmagic/sparkmagic/kernels/pysparkkernel [InstallKernelSpec] Installed kernelspec pysparkkernel in /usr/local/share/jupyter/kernels/pysparkkernel [22:45]:[root@myhost:~]# Log: After enabling 2nd custom kernel, when I refresh home page [D 22:45:37.584 NotebookApp] 200 GET /api/sessions?_=1513320245983 (10.196.177.146) 0.90ms [D 22:45:37.585 NotebookApp] 200 GET /api/terminals?_=1513320245984 (10.196.177.146) 0.42ms [D 22:45:37.668 NotebookApp] 200 GET /api/contents?type=directory&_=1513320245985 (10.196.177.146) 1.18ms [D 22:45:38.691 NotebookApp] Using contents: services/contents [D 22:45:38.692 NotebookApp] 304 GET /tree (10.196.177.146) 1.51ms [D 22:45:38.812 NotebookApp] 304 GET /custom/custom.css (10.196.177.146) 0.55ms [D 22:45:39.134 NotebookApp] 304 GET /custom/custom.js?v=20171214222020 (10.196.177.146) 0.50ms [D 22:45:39.150 NotebookApp] 200 GET /static/base/images/favicon.ico?v=30780f272ab4aac64aa073a841546240 (10.196.177.146) 0.61ms [D 22:45:39.231 NotebookApp] 200 GET /api/config/tree?_=1513320338963 (10.196.177.146) 2.07ms [D 22:45:39.233 NotebookApp] 200 GET /api/config/common?_=1513320338964 (10.196.177.146) 1.73ms [D 22:45:39.235 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.235 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.235 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:45:39.752 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.752 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.752 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:45:40.271 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:40.271 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:40.271 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:45:40.786 NotebookApp] 200 GET /api/kernelspecs (10.196.177.146) 1551.01ms [D 22:45:40.787 NotebookApp] 200 GET /api/terminals?_=1513320338965 (10.196.177.146) 0.73ms [D 22:45:40.788 NotebookApp] 200 GET /api/terminals?_=1513320338967 (10.196.177.146) 0.53ms [D 22:45:40.789 NotebookApp] 200 GET /api/sessions?_=1513320338966 (10.196.177.146) 0.65ms [D 22:45:41.125 NotebookApp] 200 GET /api/contents?type=directory&_=1513320338968 (10.196.177.146) 1.21ms In above logs, you can see I'm seeing Found Kernel 9 times, if I have 3 kernels enabled. [D 22:45:39.235 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.235 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.235 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:45:39.752 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.752 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:39.752 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources [D 22:45:40.271 NotebookApp] Found kernel pysparkkernel in /usr/local/share/jupyter/kernels [D 22:45:40.271 NotebookApp] Found kernel sparkkernel in /usr/local/share/jupyter/kernels [D 22:45:40.271 NotebookApp] Native kernel (python3) available from /home/anaconda3/lib/python3.5/site-packages/ipykernel/resources Please let me know your comments to fix this issue. It's really slowing down our Production instances performance when we enable 10+ kernels. Issue: https://github.com/jupyter/notebook/issues/3135 Thanks! -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/f3f3eb17-685b-47cd-9ff1-a6d1fe3ff432%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
