Re: [ubuntu-cloud] Can I reboot from cloud-init?

2012-09-26 Thread Scott Moser 
On Tue, 25 Sep 2012, Dave Hein wrote:

 On Mon, Sep 24, 2012 at 7:50 AM, Scott Moser smo...@ubuntu.com wrote:

  The config snippet, will do what you'd want in 12.10 and later.  It
  reboots immediately after 'apt-get upgrade' by cloud-init if there is a
  reboot required.  This also allows you to install new packages and make
  sure that by the time your user-data scripts run, you're in the right
  kernel and modules for that kernel too.
 
#cloud-config
apt_upgrade: True
apt_reboot_if_required: True
 
  cloud-init is still a bit annoying in the logs to the console when this
  runs (as it's logging is going to rsyslog, and both cloud-init and rsyslog
  get killed by 'reboot'), but it generally works, and anything that was
  going to run after the 'apt_update_upgrade' config module will just run
  after reboot.
 
 
 This is really nice and seems to work almost exactly as I would want.
 However, the lack of persistent logs is a big roadblock.

I'm not really sure what you mean here.
All data will be logged.  And everything before '/sbin/reboot' will get to
syslog even.  The rest, after failure to write to /dev/log gets to the
images' console only.

I've not tried it, but you should actually be able to craft cloud-config
that sepecifies a python-logging configuration that would go to a remote
syslog.  Then, after user-data is consumed subsequent logs would go there.
You could even add a boothook that wrote this data to
/etc/cloud/cloud.cfg.d/90_custom_logging.conf, then the next boot would
pick up that configuration earlier.

See the /etc/cloud/cloud.cfg.d/05_logging.cfg file for example of what
that config looks like.  The end result is a string that is consumable by
http://docs.python.org/library/logging.config.html

Again, when I did this, the intent was to support remote logging, but I
have not tested it extensively.

 I'm trying to think of a way to attach a very small EBS volume and direct
 or copy the logs there (before the reboot). I have a vague notion of how to
 make that work when launching a single instance, but it seems difficult to
 do when launching multiple instances in one EC2 request.

 Or perhaps it would be possible to e-mail the logs before the reboot, or
 write them to S3, or post them to a web service?

email would seem a reasonable POST mechanism for non-sensitve data.
Dustin Kirkland wrote a package named 'bootmail' that can sign data from
the instance.  I'm not sure about encryption of that data using a
host-only key.

 --
 Dave


-- 
Ubuntu-cloud mailing list
Ubuntu-cloud@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-cloud

Re: [ubuntu-cloud] Can I reboot from cloud-init?

2012-09-24 Thread Scott Moser 
On Mon, 24 Sep 2012, Nick Barcet wrote:

 On 09/23/2012 04:50 PM, Dave Hein wrote:
  I'm trying to setup a cloud-config file to invoke a script that will run
  apt-get to update packages and install a few additional packages.
  Because the kernel can be updated by the 'apt-get upgrade', I want to
  reboot at the end to activate the new kernel.

This isn't *really* well tested, but one of our kernel developers
basically wanted to do exactly what you were requesting.
Under bug 1038108 (http://pad.lv/1038108) I added a cloud-config syntax
for doing explicitly this.

The config snippet, will do what you'd want in 12.10 and later.  It
reboots immediately after 'apt-get upgrade' by cloud-init if there is a
reboot required.  This also allows you to install new packages and make
sure that by the time your user-data scripts run, you're in the right
kernel and modules for that kernel too.

  #cloud-config
  apt_upgrade: True
  apt_reboot_if_required: True

cloud-init is still a bit annoying in the logs to the console when this
runs (as it's logging is going to rsyslog, and both cloud-init and rsyslog
get killed by 'reboot'), but it generally works, and anything that was
going to run after the 'apt_update_upgrade' config module will just run
after reboot.


 Hello Dave,

 Our best practice, in your case, is to modify the script that launches a
 new instance to always make sure you boot from the latest image in the
 suite you are running.  On an officially maintained cloud, our images

I agree here, this is definitely the best practice.  If you're launching
images in automation, you should hit /query to find what ami to launch.

 are updated as soon as a security update that mandates it is published,
 or at least every two months.  This means that by launching the latest
 image, you are quite unlikely to have a kernel update to be done.  This

Unfortunately, we've not done as good of a job of keeping up with updates
as we'd like.  The plan is to get that to being completely automated, and
ideally pretty much in lock-step with kernel upgrades.  Now, however, it
still requires someone to do manual work, and as a result we've not done
new image releases as much as we'd like.

 also limits the number of updates that you will have to pull, over time,
 for a given suite.

 To do so on EC2, I would recommend using the query interface on
 cloud-images.ubuntu.com [1].  Look at the file and URL, I think it is
 self explanatory, but more explanations are available at [2].

 [1] http://cloud-images.ubuntu.com/query/precise/server/released.current.txt
 [2]https://help.ubuntu.com/community/UEC/Images#Machine_Consumable_Ubuntu_Cloud_Guest_images_Availability_Data

 Hope this helps,
 Nick



-- 
Ubuntu-cloud mailing list
Ubuntu-cloud@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-cloud