[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
This bug was fixed in the package cloud-init - 0.5.10-0ubuntu1.2 --- cloud-init (0.5.10-0ubuntu1.2) lucid-proposed; urgency=low * add support for setting debconf selections (LP: #582667) * force runcmd scripts to run after cloud-init/cloud-config scripts are finished (LP: #613309) -- Scott Moser smo...@ubuntu.com Tue, 17 Aug 2010 08:45:39 -0400 ** Changed in: cloud-init (Ubuntu Lucid) Status: Fix Committed = Fix Released -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
I've verified the update is functioning as it should. Here is how: - booted ami-1eea0077 (us-east-1 ubuntu-lucid-daily-i386-server-20100817) with the following user data: #cloud-config debconf_selections: | debconf debconf/priority select low debconf debconf/frontend select readline apt_update: true apt_upgrade: true sm_misc: - user_setup | set -x; exec ~/user_setup.log 21 echo starting at $(date -R) echo ps grep apt ps axw | grep apt echo ps grep cloud-init-cfg ps axw | grep cloud-init-cfg runcmd: - [ sudo, -Hu, ubuntu, sh, -c, 'read up sleep /proc/uptime; echo $(date): runcmd up at $up | tee -a ~/runcmd.log' ] - [ sudo, -Hu, ubuntu, sh, -c, *user_setup ] ### end user data ### - on initial boot, the instance has old cloud-init from lucid, so old debconf values will be in place. $ sudo apt-get install debconf-utils $ sudo debconf-get-selections | grep debconf/ debconf debconf/frontend select Dialog debconf debconf/priority select high - save off old logs $ mkdir old-logs mv *.log old-logs - add proposed and get cloud-init $ echo deb http://us.archive.ubuntu.com/ubuntu/ \ $(lsb_release -sc)-proposed restricted main multiverse universe | sudo tee -a /etc/apt/sources.list.d/proposed.list $ sudo apt-get install cloud-init $ dpkg-query --show cloud-init cloud-init 0.5.10-0ubuntu1.2 - remove /var/lib/cloud so cloud-init will run again on next boot $ sudo rm -Rf /var/lib/cloud $ sudo reboot - come back to instance, this time * debconf set selelections will have taken affect ( bug 582667 is fixed) $ sudo debconf-get-selections | grep debconf/ debconf debconf/frontend select readline debconf debconf/priority select low * the logs will not show apt or cloud-init-cfg processes ( bug 613309 is fixed) because the scripts will have run after those have finished (even though 'apt-get upgrade' will have pulled all the -proposed changes) the second time, runcmd.log showed it was running at 92.46 seconds of uptime, compared to 12.63 the first time. The second time neither 'ps' finds cloud-init-cfg processes or apt processes. ** Tags added: ec2-images uec-images verification-done ** Tags removed: ec2 uec userdata verification-needed -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Accepted cloud-init into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance! ** Changed in: cloud-init (Ubuntu Lucid) Status: In Progress = Fix Committed ** Tags added: verification-needed -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Scott, in lucid-proposed there are now two cloud-init packages. Please merge them into one upload (0.5.10-0ubuntu1.2); if that's impractical, then please build ubuntu1.3 with -v0.5.10-0ubuntu1.1, so that the source.changes contains both versions. Thanks! -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
** Branch linked: lp:ubuntu/lucid-proposed/cloud-init -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Martin, Sorry for the confusion. I've just uploaded 0.5.10-0ubuntu1.2 that has fixes for this bug and bug 613309. -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
above should have said bug 613309 (this one) and bug 582667. -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
** Changed in: cloud-init (Ubuntu Lucid) Importance: Undecided = High ** Changed in: cloud-init (Ubuntu Lucid) Status: New = In Progress ** Changed in: cloud-init (Ubuntu Lucid) Milestone: None = lucid-updates -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
** Branch linked: lp:~smoser/ubuntu/lucid/cloud-init/bug613309 -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
For maverick, this is fix-released. In newer versions of cloud-init there are only upstart jobs: cloud-init cloud-config cloud-run-user-script cloud-run-user-script runs on stopped rc RUNLEVEL=[2345] and stopped cloud-config ** Also affects: cloud-init (Ubuntu Lucid) Importance: Undecided Status: New ** Changed in: cloud-init (Ubuntu) Importance: Undecided = Medium ** Changed in: cloud-init (Ubuntu) Status: New = Fix Released -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Pete, A build of the linked branch above will be available shortly in my ppa for lucid. https://launchpad.net/~smoser/+archive/ppa/+packages . I'd appreciate some test on that. My initial test is that it functions correctly. To test: - launch an instance with some user data and runcmd - ssh to instance: sudo apt-add-repository ppa:smoser/ppa sudo apt-get install cloud-init sudo rm -Rf /var/lib/cloud sudo reboot - ssh to instance - you should see changed ssh keys - verify that your user scripts run I don't have a suggestion on a perfect way to prove that the dependencies are correct, but the fact that it runs generally indicates that it is good. Ie, I'm fairly sure, that given the new 'start on' for cloud-run-user-script, it is not going to run *before* any of the other scripts. | start on (stopped rc RUNLEVEL=[2345] \ |and stopped cloud-apt-update-upgrade \ |and stopped cloud-config-misc \ |and stopped cloud-config-mounts \ |and stopped cloud-config-puppet \ |and stopped cloud-config-ssh \ |and stopped cloud-disable-ec2-metadata ) -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Sure will do, I am working on this stuff right now so once the build is done I will deploy. -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Looks good Scott. Everything seem to start and execute in the correct order. * I purged all packages and installed via user-data and executed custom user run script. Which included the following: - Custom apt repo - Custom key for repo - packages: - curl - pwgen - pastebinit - python-software-properties - chef - rubygems - Custom script to configure chef - validation.pem role list and node id (instance-id) from meta-data service -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
** Description changed: Binary package hint: cloud-init All lucid/maverick source packages, The following upstart scripts need to wait for the 'cloud-config' status in addition to what they currently wait for, otherwise userdata might not be loaded yet. cloud-apt-update-upgrade.conf cloud-config-misc.conf cloud-config-mounts.conf cloud-config-ssh.conf cloud-disable-ec2-metadata.conf cloud-ec2-ebs-mounts.conf cloud-raid.conf cloud-runurl.conf ~~ most of these need to just add 'and cloud-config' but one or two need to just 'start on cloud-config' ~~~ cloud-run-user-script.conf needs 'start on (stopped rc RUNLEVEL=[2345] and stopped cloud-config- misc)' We need this package to work in ec2 as well as UEC otherwise it make provisioning of a new instance much harder and defeats the purpose of this package. Since Lucid is LTS I would like to see a SRU for this issue as well since it prevent out of the box/repo use of this feature. + + Begin SRU Justification + Impact: The impact of this bug is that a popular portion of cloud-config syntax [1] is not easily used in the 10.04 images. If the user specifies commands to run, they cannot rely on other portions of cloud-init having finished before those scripts run. The big examples is that if the user installs a package via 'pkgs', they cannot rely on it being present in their 'runcmd'. The ordering is simply not guaranteed. + Solution: The solution is to make the upstart script that executes the user's commands depend on 'stopped' of each of the other upstart jobs. In this manner, it will not execute until the other jobs are finished. + Patch: Available in branch attached to this bug [2] + Regression Potential: There should be low potential for regression and low realistic change of user expectations. Previously, the order was non-deterministic, this will guarantee that jobs run after packages are installed. + + TEST Case: + * launch ec2 instance (ubuntu-lucid-10.04-i386-server-20100427.1, such as ami-fd4aa494) with user data having 'packages' and 'runcmd' section. Such as: + | #cloud-config + | packages: [ bzr, ubuntu-dev-tools, ccache, vim-nox, git-core, lftp ] + | runcmd: + | - [ sudo, -Hu, ubuntu, sh, -c 'bzr branch lp:ubuntu/lucid-proposed/cloud-init 21 | tee cmd.log' ] + * without a fix for this bug, the ordering is indeterminable, but most likely, the 'bzr branch' command will run before bzr is installed. With the fix, it is guaranteed to run afterwards. + + -- + [1] http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/lucid/cloud-init/lucid-proposed/annotate/head%3A/doc/examples/cloud-config.txt + [2] http://bazaar.launchpad.net/~smoser/ubuntu/lucid/cloud-init/bug613309/revision/17 + End SRU Justification = -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 613309] Re: cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes
Looking at 'cloud-config-misc.conf' script this only waits for the filesystem. What prevents this from running prior to the cloud-init upstart script? 'cloud-config-misc.conf' also is what the 'cloud-run- user-script.conf' waits for to stop to allow the user scripts to run after it. So I think there is still an issue with ordering and making sure these upstart scripts all complete prior to the execution of 'cloud-run-user-script.conf' -- cloud-run-user-script.conf upstart script needs to run after all other cloud-init processes https://bugs.launchpad.net/bugs/613309 You received this bug notification because you are a member of Ubuntu Server Team, which is a direct subscriber. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
