** Description changed: + ==== Begin SRU Template [cloud-init] ==== + [Impact] + One of cloud-init's features is to upgrade the system during first boot so that it is fully up to date when the user code starts running. + + [Test Case] + launch an old instance of 16.04 that will need an update to snapd with + user-data that indicates a package upgrade should be done. + + $ lxc image show ubuntu:74a491804877 + autoupdate: false + properties: + aliases: 16.04,default,lts,x,xenial + architecture: amd64 + description: ubuntu 16.04 LTS amd64 (release) (20160830) + label: release + os: ubuntu + release: xenial + serial: "20160830" + version: "16.04" + public: true + + $ printf "#%s\n%s\n" cloud-config "packages: [snapd]" > user-data + + $ lxc launch ubuntu:74a491804877 xrecreate "--config=user.user-data=$(cat user-data)" + $ lxc exec xrecreate -- tail -f /var/log/cloud-init-output.log + + # you will see the output log hang at: + # Setting up snapd (2.14.2~16.04) ... + + + ## Now get new container and patch in cloud-init + $ lxc launch ubuntu:74a491804877 xpatched + # let it boot, with no user-data saying to update. + $ sleep 10 + + # update the container to new cloud-init, then clean it to make + # it look like first boot again. + $ lxc file push - xpatched/etc/cloud/cloud.cfg.d/update.cfg < user-data + $ lxc exec xpatched -- sh -c ' + p=/etc/apt/sources.list.d/proposed.list + echo deb http://archive.ubuntu.com/ubuntu xenial-proposed main > "$p" && + apt-get update -q && apt-get -qy install cloud-init' + $ lxc exec xpatched -- sh -c ' + cd /var/lib/cloud && for d in *; do [ "$d" = "seed" ] || rm -Rf "$d"; done + rm -Rf /var/log/cloud-init*' + + $ lxc exec xpatched reboot + $ lxc exec xpatched -- tail -f /var/log/cloud-init-output.log + + # snapd installed and a 'Cloud-init finished' message. + + [Regression Potential] + The change to running package installation later in boot will likely affect some things. However, previously a larger set of things were unreliable. This will make things over all more reliable. + ==== End SRU Template [cloud-init] ==== + I reproducibly run into an eternal hang when deploying services with Juju, when it prepares a new xenial testbed. The current xenial cloud image does not have the latest snapd, so snapd gets dist-upgraded: Preparing to unpack .../snapd_2.14.2~16.04_amd64.deb ... Warning: Stopping snapd.service, but it can still be activated by: - snapd.socket + snapd.socket Unpacking snapd (2.14.2~16.04) over (2.13) ... Setting up snapd (2.14.2~16.04) ... [...] hangs The postinst tries to start snapd.boot-ok.service on upgrade: - |-cloud-init(311)-+-apt-get(577)---dpkg(845)---snapd.postinst(846)---perl(919)---systemctl(922) - | `-sh(354)---tee(355) + |-cloud-init(311)-+-apt-get(577)---dpkg(845)---snapd.postinst(846)---perl(919)---systemctl(922) + | `-sh(354)---tee(355) root 922 0.0 0.0 25316 1412 pts/0 S+ 06:09 0:00 /bin/systemctl start snapd.boot-ok.service This hangs eternally because: - - cloud-init's dist-upgrade runs *during* the boot process, so that the + - cloud-init's dist-upgrade runs *during* the boot process, so that the system is not fully booted yet when this happens (see bug 1576692); thus multi-user.target is *not* yet active - - snapd.boot-ok.service is After=multi-user.target + - snapd.boot-ok.service is After=multi-user.target - - "systemctl start" is synchronous by default, i. e. it waits until the + - "systemctl start" is synchronous by default, i. e. it waits until the service is started unless you use --no-block. Thus snapd.postinst waits on snapd.boot-ok.service waits on multi- user.target waits on cloud-init to finish waits on snapd.postinst to finish. I think conceptually you shouldn't start snapd.boot-ok.service in the postinst; if the system is already booted (manual dist-upgrade) it should already be running, and if it does get upgraded during boot (with cloud-init) then you shouldn't pretend that booting is already finished. So I suggest to use dh_installinit with --no-scripts for snapd.boot- ok.service.
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1621336 Title: snapd.boot-ok.service hangs eternally on cloud image upgrades To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1621336/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
