(The statements and opinions in this e-mail are mine, not Google's.) On Tue, Jan 01, 2019 at 07:24:19AM +0100, Thomas Goirand wrote: > > Why didn't we use cloud-init? For pretty much the same reason the other > > cloud vendors haven't, and there was fair bit of discussion about it at > > the cloud-init summit back in August. > > I'm not sure what "other cloud vendors" you're referring to, but AWS use > it, and so are the vast majority of OpenStack based services (according > to Jeremy, that's about 72 cloud providers).
Azure installs waagent in its Debian image; AWS installs awscli in its Debian image ; Google installs a number of GCE-specific packages in its Debian Image. It may not be the "official Debian image", but guess which one gets recommended for use by customers in their official cloud documentation? > > The Redhat based distributions are running comparatively ancient > > versions of cloud-init, and we can't inject newer versions in the images > > used on our platforms, or we fall afoul of their branding rules and > > wouldn't be able to call it "CentOS", for example. > > You will have the very same problem with Debian (except maybe that > cloud-init isn't that old? I'm not sure what you call old, so I can't > tell about this specific point...) So that's going to be the problem. The cloud space is moving super fast, with new features being added as Microsoft, Amazon, Google, Oracle, et.al., compete with teach other. These new features will often require the support in the guest OS, whether it's in the kernel, or userspace (or, most often, both). If the official guest OS image won't allow those enhancements, then the cloud companies will either need to encourage their customers to use distros that *do* allow those enhancements (read: Ubuntu), or they will provide "value-added" variants of the distros that they decide they want to support --- because that's what all of their competitors will be doing, in order to provide the best possible customer exprience. > So, if there's a way to have an image work with cloud-init, then there > is a chance that we can produce a Stretch official image at some point > for the Oracle cloud. Otherwise, your only hope is to have your agent > packaged in Debian, or simply not hope for having the word *official* > stamped (ie: it's going to be your own version of Debian). I suspect what we will find is that for many cloud companies, being able to allow customers to use all of the advanced features of their cloud environment is going to be way more important than having the word "official" stamped on their distro image. On Tue, Jan 01, 2019 at 07:27:16AM +0100, Thomas Goirand wrote: > > "Run-and-done" also doesn't support disk snapshots and disk resizing > > properly. > > You didn't react to my suggestion about having some kind of signal sent > by Qemu to the kernel. Isn't this a better idea than having a userland > agent doing it? You need *both*. There will be some way of sending the signal from the Cloud environment to the kernel --- for example, via a virtio-scsi extension. But then, you need to have a way of having the guest kernel send a signal to userspace so that a database such as MySQL or Postgress to freeze their Database journal, and then ask the guest kernel to freeze their journalling file systems, and then tell the guest kernel to send a signal back to the Cloud environment saying, "it's OK to proceed with the block device snapshot". Then when the snapshot is done, the Cloud environment will need to send an "OK to unfreeze" message which has to get propagated to userspace so the database can resume their journal. In the ideal world, this would be standardized and be the same across all hosting / cloud environments, and the userspace hooks would also be standardized. The problem is that if you wait for the OpenStack standardization process to grind away slowly, and then wait for years(!) for the stable/enterprise distros to get it shipped into a cloud-init (that today is fundamentally architecturally deficient for this sort of feature; so ix-nay on using a cloud-init plugin) --- the companies that just throw support into their agent will be able to let their customers take advantage of this cloud feature years before their competitors. So at least for the short-term, the need for cloud-specific agents is going to be huge, and good luck trying to get cloud companies to agree to degrade customer functionality just for the sake of "Debian official" getting stamped on the image. Or worse, telling customers they should use "Debian Official" images and then use "curl | bash" to get the functionality back. For both the cloud company and the customer, using the "Debian as enhanced by {AWS|Azure|Google|Oracle}" is going to be more secure and more easy for the customer to use. Cheers, - Ted