[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Eric, this bug is about the linux-kvm kernel _in Ubuntu_. If you're seeing a cloud-init issue on Centos, then please file a separate bug. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Ok, I got a question for all the interested folks out there; What is we were to have a test case that run init-local and fails if random gets imported? I'm looking at how this could be done, but that test could link to this bug and have people realise that they need to be careful about the

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

In Openstack Newton (linux-kvm) with Centos 7 (latest cloud-image) with cloud-init 19.3, the delay is observed. I think this is due to the version that Centos ships with for python (2.7.5). The problem is at the init-local stage. https://github.com/canonical /cloud-init/pull/103 here is the

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: linux-kvm (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1727358 Title:

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Marking as wont-fix for cloud-init for now as that would be a workaround for the base problem that is not desired right now. Adding linux-kvm to evaluate why this difference exists between linux-generic and linux-kvm ** Changed in: cloud-init Status: Incomplete => Won't Fix ** Also

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

So far I've tested in GCE and Azure, and both platforms show no delays at all due to lack of entropy at boot. Unless someone identifies a platform where this actually causes boot delays other than linux-kvm, which is presumed due to a missing kernel option, there's no point in changing anything

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

With python3-dbg and python3.6-examples installed, here's a commandline for getting to the problematic getrandom() call. # PYTHONPATH=/usr/share/doc/python3.6/examples/gdb PYTHONHASHSEED=1 gdb -ex 'break getrandom' -ex run -ex 'cont 4' -ex "python import libpython" -ex py-bt --args

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

This looks like a chain of decisions that are all locally reasonable but when put together may not give a sensible result. cloud-init uses jinja2 as a sensible templating engine. jinja2 has async handling, which relies on asyncio. asyncio uses the multiprocessing module, which has both thread

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

I've tracked down the blocking use of randomness in cloud-init's python module initialization. There are several nonblocking calls to getrandom(), which I believe should be fine; then a blocking call in the multiprocessing module. # PYTHONHASHSEED=1 gdb --args /usr/bin/python3-dbg

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

I've taken your kernel (vmlinuz-4.4.0-1009-kvm) and booted a "normal" cloud image with that kernel. It hangs similarly (but for noticibly shorter time). http://cloud-images.ubuntu.com/daily/server/artful/20171026.1/artful- server-cloudimg-amd64.img I did have to disable the UEFI partition

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

this does also reproduce: qemu-system-x86_64 -enable-kvm \ -device virtio-scsi-pci,id=virtio-scsi-xkvm \ -device virtio-net-pci,netdev=net00 -netdev type=user,id=net00 \ -drive file=my.img,id=disk00,if=none,format=qcow2,index=0 \ -device virtio-blk,drive=disk00,serial=my.img

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

It really is *defaults* though, because you don't want to have any extra devices included that could throw off the boot (such as a RNG, which should not be added by default). In any case, I'll attach the xml I use as well, but I reproduce the build on a typical artful install. The only important

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Libvirt XML for the VM. ** Attachment added: "vm.xml" https://bugs.launchpad.net/cloud-init/+bug/1727358/+attachment/4996984/+files/vm.xml -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1727358

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

On Thu, Oct 26, 2017 at 06:07:57PM -, Mathieu Trudel-Lapierre wrote: > You can easily reproduce this by running the following image in virt- > manager with a default setup (and optionally add a RNG device if > necessary): The defaults in both virt-manager and libvirt change over time. For

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

You'll want to add an additional disk to provide the metadata for cloud- init too. The exact iso file I have used is attached. ** Attachment added: "seed.iso" https://bugs.launchpad.net/cloud-init/+bug/1727358/+attachment/4996928/+files/seed.iso -- You received this bug notification because

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

You can easily reproduce this by running the following image in virt- manager with a default setup (and optionally add a RNG device if necessary): https://launchpad.net/~cyphermox/+livefs/ubuntu/artful/ubuntu- cpc/+build/113428/+files/livecd.ubuntu-cpc.img -- You received this bug notification

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

On Thu, Oct 26, 2017 at 03:56:24PM -, Scott Moser wrote: > You can grep through, the onlyi real function taht uses it is > 'rand_str'. and i'm not sure really why that would be called at the > point that you're seeing the slow down. > I think it more valuable for you to figure out why your

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

You can grep through, the onlyi real function taht uses it is 'rand_str'. and i'm not sure really why that would be called at the point that you're seeing the slow down. I think it more valuable for you to figure out why your environment is causing this issue and fixing that rather than fixing

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

You can grep through, the onlyi real function taht uses it is 'rand_str'. and i'm not sure really why that would be called at the point that you're seeing the slow down. I think it more valuable for you to figure out why your environment is causing this issue and fixing that rather than fixing

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

On Thu, Oct 26, 2017 at 10:37:37AM -, Scott Moser wrote: > Only via 'import random' and light usage. Even "light" usage is going to block at this point in the boot; can you give me a pointer into where this is in the code, so I can try to assess if this should be avoided? -- You received

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Only via 'import random' and light usage. On Oct 25, 2017 11:21 PM, "Steve Langasek" wrote: > On Thu, Oct 26, 2017 at 01:43:12AM -, Scott Moser wrote: > > possibly related, but was fixed: > > https://bugs.python.org/issue25420 > > Does cloud-init itself

Re: [Bug 1727358] Re: cloud-init is slow to complete init on minimized images

On Thu, Oct 26, 2017 at 01:43:12AM -, Scott Moser wrote: > possibly related, but was fixed: > https://bugs.python.org/issue25420 Does cloud-init itself consume entropy when it runs? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Please explain how you did this. What did you download, what qemu commands did you run. Thanks. ** Bug watch added: Python Roundup #25420 http://bugs.python.org/issue25420 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

possibly related, but was fixed: https://bugs.python.org/issue25420 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1727358 Title: cloud-init is slow to complete init on minimized images To manage

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

** Attachment added: "cloud-init.tar.gz" https://bugs.launchpad.net/cloud-init/+bug/1727358/+attachment/4995724/+files/cloud-init.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1727358

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

This is running on *minimized* images, with linux-kvm kernel; so there might well not be CONFIG_DMI. This is however not the issue (or most likely not). Tests are done in qemu, to simulate running the images in openstack for now (but they are meant to start in openstack without issues). It's

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

Could you run cloud-init collect-logs And then attach the cloud-init.tar.gz Also, Can you provide some information on what you were running? " cloud-init is slow to complete init on minimized images " How can I recreate this? One curious thing there is: 2017-10-25 13:22:07,157 -

[Bug 1727358] Re: cloud-init is slow to complete init on minimized images

** Attachment added: "cloud-init.log" https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1727358/+attachment/4994544/+files/cloud-init.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.