Re: [openstack-dev] [nova][ironic] do we really need websockify with numpy speedups?
On 11/26/2015 06:44 AM, Pavlo Shchelokovskyy wrote: > Hi all, > > I was long puzzled why devstack is installing numpy. Being a fantastic > package itself, it has the drawback of taking about 4 minutes to compile > its C extensions when installing on our gates (e.g. [0]). I finally took > time to research and here is what I've found: > > it is used only by websockify package (installed by AFAIK ironic and > nova only), and there it is used to speed up the HyBi protocol. Although > the code itself has a path to work without numpy installed [1], the > setup.py of websockify declares numpy as a hard dependency [2]. > > My question is do we really need those speedups? Do we test any feature > requiring fast HyBi support on gates? Not installing numpy would shave 4 > minutes off any gate job that is installing Nova or Ironic, which seems > like a good deal to me. > > If we decide to save this time, I have prepared a pull request for > websockify that moves numpy requirement to "extras" [3]. As a > consequence numpy will not be installed by default as dependency, but > still possible to install with e.g. "pip install websockify[fastHyBi]", > and package builders can also specify numpy as hard dependency for > websockify package in package specs. I went down this same path before. That masking is mandatory in the spec - https://github.com/kanaka/websockify/pull/163 The right answer here is to get the wheel mirrors working in infra. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ironic] do we really need websockify with numpy speedups?
Hi Roman, those wheels still have to be built and maintained by someone. As there are no wheels for Linux in upstream PyPI, those would have to be built and maintained by openstack-infra, and I'm not sure how big variety of platforms has to be supported. Just taking corresponding deb/rpm package from upstream seems better option in such case. Cheers, On Thu, Nov 26, 2015 at 3:57 PM Roman Podoliaka wrote: > Hi Pavlo, > > Can we just use a wheel package for numpy instead? > > Thanks, > Roman > > On Thu, Nov 26, 2015 at 3:00 PM, Pavlo Shchelokovskyy > wrote: > > Hi again, > > > > I've went on and created a proper pull request to websockify [0], comment > > there if you think we need it :) > > > > I also realized that there is another option, which is to include > > python-numpy to files/debs/ironic and files/debs/nova (strangely it is > > already present in rpms/ for nova, noVNC and spice services). > > This should install a pre-compiled version from distro repos, and should > > also speed things up. > > > > Any comments welcome. > > > > [0] https://github.com/kanaka/websockify/pull/212 > > > > Best regards, > > > > On Thu, Nov 26, 2015 at 1:44 PM Pavlo Shchelokovskyy > > wrote: > >> > >> Hi all, > >> > >> I was long puzzled why devstack is installing numpy. Being a fantastic > >> package itself, it has the drawback of taking about 4 minutes to > compile its > >> C extensions when installing on our gates (e.g. [0]). I finally took > time to > >> research and here is what I've found: > >> > >> it is used only by websockify package (installed by AFAIK ironic and > nova > >> only), and there it is used to speed up the HyBi protocol. Although the > code > >> itself has a path to work without numpy installed [1], the setup.py of > >> websockify declares numpy as a hard dependency [2]. > >> > >> My question is do we really need those speedups? Do we test any feature > >> requiring fast HyBi support on gates? Not installing numpy would shave 4 > >> minutes off any gate job that is installing Nova or Ironic, which seems > like > >> a good deal to me. > >> > >> If we decide to save this time, I have prepared a pull request for > >> websockify that moves numpy requirement to "extras" [3]. As a > consequence > >> numpy will not be installed by default as dependency, but still > possible to > >> install with e.g. "pip install websockify[fastHyBi]", and package > builders > >> can also specify numpy as hard dependency for websockify package in > package > >> specs. > >> > >> What do you think? > >> > >> [0] > >> > http://logs.openstack.org/82/236982/6/check/gate-tempest-dsvm-ironic-agent_ssh/1141960/logs/devstacklog.txt.gz#_2015-11-11_19_51_40_784 > >> [1] > >> > https://github.com/kanaka/websockify/blob/master/websockify/websocket.py#L143 > >> [2] https://github.com/kanaka/websockify/blob/master/setup.py#L37 > >> [3] > >> > https://github.com/pshchelo/websockify/commit/0b1655e73ea13b4fba9c6fb4122adb1435d5ce1a > >> > >> Best regards, > >> -- > >> Dr. Pavlo Shchelokovskyy > >> Senior Software Engineer > >> Mirantis Inc > >> www.mirantis.com > > > > -- > > Dr. Pavlo Shchelokovskyy > > Senior Software Engineer > > Mirantis Inc > > www.mirantis.com > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Dr. Pavlo Shchelokovskyy Senior Software Engineer Mirantis Inc www.mirantis.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ironic] do we really need websockify with numpy speedups?
Hi Pavlo, Can we just use a wheel package for numpy instead? Thanks, Roman On Thu, Nov 26, 2015 at 3:00 PM, Pavlo Shchelokovskyy wrote: > Hi again, > > I've went on and created a proper pull request to websockify [0], comment > there if you think we need it :) > > I also realized that there is another option, which is to include > python-numpy to files/debs/ironic and files/debs/nova (strangely it is > already present in rpms/ for nova, noVNC and spice services). > This should install a pre-compiled version from distro repos, and should > also speed things up. > > Any comments welcome. > > [0] https://github.com/kanaka/websockify/pull/212 > > Best regards, > > On Thu, Nov 26, 2015 at 1:44 PM Pavlo Shchelokovskyy > wrote: >> >> Hi all, >> >> I was long puzzled why devstack is installing numpy. Being a fantastic >> package itself, it has the drawback of taking about 4 minutes to compile its >> C extensions when installing on our gates (e.g. [0]). I finally took time to >> research and here is what I've found: >> >> it is used only by websockify package (installed by AFAIK ironic and nova >> only), and there it is used to speed up the HyBi protocol. Although the code >> itself has a path to work without numpy installed [1], the setup.py of >> websockify declares numpy as a hard dependency [2]. >> >> My question is do we really need those speedups? Do we test any feature >> requiring fast HyBi support on gates? Not installing numpy would shave 4 >> minutes off any gate job that is installing Nova or Ironic, which seems like >> a good deal to me. >> >> If we decide to save this time, I have prepared a pull request for >> websockify that moves numpy requirement to "extras" [3]. As a consequence >> numpy will not be installed by default as dependency, but still possible to >> install with e.g. "pip install websockify[fastHyBi]", and package builders >> can also specify numpy as hard dependency for websockify package in package >> specs. >> >> What do you think? >> >> [0] >> http://logs.openstack.org/82/236982/6/check/gate-tempest-dsvm-ironic-agent_ssh/1141960/logs/devstacklog.txt.gz#_2015-11-11_19_51_40_784 >> [1] >> https://github.com/kanaka/websockify/blob/master/websockify/websocket.py#L143 >> [2] https://github.com/kanaka/websockify/blob/master/setup.py#L37 >> [3] >> https://github.com/pshchelo/websockify/commit/0b1655e73ea13b4fba9c6fb4122adb1435d5ce1a >> >> Best regards, >> -- >> Dr. Pavlo Shchelokovskyy >> Senior Software Engineer >> Mirantis Inc >> www.mirantis.com > > -- > Dr. Pavlo Shchelokovskyy > Senior Software Engineer > Mirantis Inc > www.mirantis.com > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ironic] do we really need websockify with numpy speedups?
Hi again, I've went on and created a proper pull request to websockify [0], comment there if you think we need it :) I also realized that there is another option, which is to include python-numpy to files/debs/ironic and files/debs/nova (strangely it is already present in rpms/ for nova, noVNC and spice services). This should install a pre-compiled version from distro repos, and should also speed things up. Any comments welcome. [0] https://github.com/kanaka/websockify/pull/212 Best regards, On Thu, Nov 26, 2015 at 1:44 PM Pavlo Shchelokovskyy < pshchelokovs...@mirantis.com> wrote: > Hi all, > > I was long puzzled why devstack is installing numpy. Being a fantastic > package itself, it has the drawback of taking about 4 minutes to compile > its C extensions when installing on our gates (e.g. [0]). I finally took > time to research and here is what I've found: > > it is used only by websockify package (installed by AFAIK ironic and nova > only), and there it is used to speed up the HyBi protocol. Although the > code itself has a path to work without numpy installed [1], the setup.py of > websockify declares numpy as a hard dependency [2]. > > My question is do we really need those speedups? Do we test any feature > requiring fast HyBi support on gates? Not installing numpy would shave 4 > minutes off any gate job that is installing Nova or Ironic, which seems > like a good deal to me. > > If we decide to save this time, I have prepared a pull request for > websockify that moves numpy requirement to "extras" [3]. As a consequence > numpy will not be installed by default as dependency, but still possible to > install with e.g. "pip install websockify[fastHyBi]", and package builders > can also specify numpy as hard dependency for websockify package in package > specs. > > What do you think? > > [0] > http://logs.openstack.org/82/236982/6/check/gate-tempest-dsvm-ironic-agent_ssh/1141960/logs/devstacklog.txt.gz#_2015-11-11_19_51_40_784 > [1] > https://github.com/kanaka/websockify/blob/master/websockify/websocket.py#L143 > [2] https://github.com/kanaka/websockify/blob/master/setup.py#L37 > [3] > https://github.com/pshchelo/websockify/commit/0b1655e73ea13b4fba9c6fb4122adb1435d5ce1a > > Best regards, > -- > Dr. Pavlo Shchelokovskyy > Senior Software Engineer > Mirantis Inc > www.mirantis.com > -- Dr. Pavlo Shchelokovskyy Senior Software Engineer Mirantis Inc www.mirantis.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][ironic] do we really need websockify with numpy speedups?
Hi all, I was long puzzled why devstack is installing numpy. Being a fantastic package itself, it has the drawback of taking about 4 minutes to compile its C extensions when installing on our gates (e.g. [0]). I finally took time to research and here is what I've found: it is used only by websockify package (installed by AFAIK ironic and nova only), and there it is used to speed up the HyBi protocol. Although the code itself has a path to work without numpy installed [1], the setup.py of websockify declares numpy as a hard dependency [2]. My question is do we really need those speedups? Do we test any feature requiring fast HyBi support on gates? Not installing numpy would shave 4 minutes off any gate job that is installing Nova or Ironic, which seems like a good deal to me. If we decide to save this time, I have prepared a pull request for websockify that moves numpy requirement to "extras" [3]. As a consequence numpy will not be installed by default as dependency, but still possible to install with e.g. "pip install websockify[fastHyBi]", and package builders can also specify numpy as hard dependency for websockify package in package specs. What do you think? [0] http://logs.openstack.org/82/236982/6/check/gate-tempest-dsvm-ironic-agent_ssh/1141960/logs/devstacklog.txt.gz#_2015-11-11_19_51_40_784 [1] https://github.com/kanaka/websockify/blob/master/websockify/websocket.py#L143 [2] https://github.com/kanaka/websockify/blob/master/setup.py#L37 [3] https://github.com/pshchelo/websockify/commit/0b1655e73ea13b4fba9c6fb4122adb1435d5ce1a Best regards, -- Dr. Pavlo Shchelokovskyy Senior Software Engineer Mirantis Inc www.mirantis.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
