On 22 January 2014 10:43, olivier sallou <[email protected]> wrote:
> > > > 2014/1/22 Anders Ingemann <[email protected]> > >> On 22 January 2014 08:23, olivier sallou <[email protected]>wrote: >> >>> >>> >>> >>> 2014/1/22 Jimmy Kaplowitz <[email protected]> >>> >>>> Neat. Yeah, a GCE image is simply a raw bootable disk named disk.raw, >>>> compressed in a gzipped GNU-format tarball with a filename ending in >>>> .tar.gz. We also encourage creating disk.raw as a sparse file and using GNU >>>> tar's S flag to minimize image size and add time. The tarball is necessary >>>> but hopefully the code is general enough to handle that. Certain bits of >>>> our image snapshotting tool gcimagebundle expect everything aside from the >>>> bootloader to be in a single MSDOS partition number 1. >>>> >>> I think this is the case for virtualbox provider. >>> >>>> We make various tweaks, such as installing the various integration >>>> software I've mentioned before, pointing to our Debian mirror, and (with >>>> Tomasz's pull request into the google GitHub fork) the host machine's NTP >>>> server, etc. Nothing that changes the essence. >>>> >>> Some of those (setting up a mirror, installing packages) can be done: >>> 1) with plugins (but as it is a requirement for you this is not the best >>> choice) >>> 2) in your plugin task, use the library part that manage package >>> install etc.. if you look at cloud-init plugin task for example, you will >>> see how to add a source (here debian backports) and packages to the image >>> (here "sudo"). >>> >>> The rest of the plugin task can focus on copying some files in the >>> image, setting ntp etc... >>> >>>> The strategy you suggest sounds worth trying indeed. >>>> >>>> - Jimmy >>>> On Jan 21, 2014 10:32 PM, "olivier sallou" <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> >>>>> 2014/1/21 Tomasz Rybak <[email protected]> >>>>> >>>>>> As Jimmy wrote in his email from 2014-01-14, I started >>>>>> looking at GCE-related parts of build-debian-cloud. >>>>>> >>>>>> I assume that the course for now is changing the scripts >>>>>> to work in Python, similar to what's being done with EC2 >>>>>> and VirtualBox parts. Should we branch from andsens/python >>>>>> and work on it, or do something else? Also, who'll create >>>>>> the main branch (GCE-python-WIP?), into which we would >>>>>> pull proposed changes? I think the best solution would be >>>>>> to create such branch in repository >>>>>> https://github.com/google/build-debian-cloud >>>>>> >>>>>> As for the work to do, I think we'll need to: >>>>>> 1. change gce file to proper manifest >>>>>> 2. move tasks from tasks/gce to providers/gce and >>>>>> rewrite them in Python >>>>>> 3. integrate cloud-init when appropriate >>>>>> >>>>> >>>>> If the base image requirement is a raw image file and GCE only adds >>>>> startup/management scripts for boot etc... you may only develop a plugin >>>>> and use VirtualBox provider which is in fact a quite generic one (not only >>>>> virtualbox). >>>>> >>>>> I personnaly use VirtualBox provider for my KVM machines and use the >>>>> opennebula plugin for the OpenNebula contextualization (will be modified >>>>> soon to use cloud-init too). >>>>> >>>>> Then, for GCE, it would be, for the user, only a matter of user >>>>> VirtualBox provider (raw format) and activating the GCE plugin. >>>>> >>>>> Olivier >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> This is rough idea, and I have not touched >>>>>> packaging of >>>>>> https://github.com/GoogleCloudPlatform/compute-image-packages >>>>>> >>>>>> Have I missed something? I assume we need to have >>>>>> more detailed plan of moving to Python so anyone >>>>>> can see what is to be done and volunteer to some >>>>>> tasks ;-) For now I just want to start discussion >>>>>> to see what I forgot about. >>>>>> >>>>>> Best regards. >>>>>> >>>>>> -- >>>>>> Tomasz Rybak GPG/PGP key ID: 2AD5 9860 >>>>>> Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860 >>>>>> http://member.acm.org/~tomaszrybak >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> gpg key id: 4096R/326D8438 (keyring.debian.org) >>>>> >>>>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 >>>>> >>>>> >>> >>> >>> -- >>> >>> gpg key id: 4096R/326D8438 (keyring.debian.org) >>> >>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 >>> >>> Heh, it's great to see others are able to figure out my framework >> already, without documentation. I should really get on with writing it >> though... :-) >> >> > If the base image requirement is a raw image file and GCE only adds >> startup/management scripts for boot etc... you may only develop a plugin >> and use VirtualBox provider which is in fact a quite generic one (not only >> virtualbox). >> >> I would strongly suggest to refrain from doing that. If the VirtualBox >> provider is indeed very generic, things should be abstracted into task >> sets. I have not abstracted much of it until now, since I wasn't aware of >> the commonalities between providers (given that there are only to - or >> three now with kvm). If You add GCE as a separate provider, I can take a >> look at it and create some new tasksets that should make the task resolving >> a bit easier. >> > > So, do you think we should create a KVM provider ? (which would be 99% > equivalent to VirtualBox for the moment but would preserve from virtualbox > modifications) > > Olivier > >> >> The advantages of having a provider rather than a plugin are manifold. >> >> 1. Plugins tasks will be resolved *after* the provider tasks, meaning >> they will be able to remove some provider tasks if they do something more >> specific. >> 2. You can enforce plugin compatibility in the manifest schemas by >> looking at the provider string, having a plugin look at what other plugins >> are loaded is just messy. >> >> Disadvantages of creating a provider as a plugin: >> >> 1. Any changes you make because the virtualbox provider doesn't quite >> fit will become hard to understand - one provider adding a task and the >> plugin removing that same task... you might end up with a tasklist that is >> completely different from virtualbox (once you are done). >> 2. Changes to the virtualbox provider will now have to be carefully >> made because other providers suddenly rely on it >> 3. You will have a hard time adding special things to the manifest >> since the vbox provider applies its own manifest schema >> >> > In your plugin task, use the library part that manage package install >> etc.. if you look at cloud-init plugin task for example, you will see how >> to add a source (here debian backports) and packages to the image (here >> "sudo"). >> What he said! The package API can save you a lot of trouble. Btw, you can >> add trusted keyrings to apt through the manifest as well. >> >> Anders >> >> > > > -- > > gpg key id: 4096R/326D8438 (keyring.debian.org) > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438 > > > So, do you think we should create a KVM provider? I do. Surely things like guest additions/guest tools are different. It would also allow use to make a proper vagrant box for it. The big challenge is making some proper taskset abstractions. It should be possible to create a minimal resolve_tasks() function if we code the tasksets properly (seeing how vbox and kvm are very similar), this would be very useful for future providers as well (e.g. gce or hp-cloud).
