[This is a Foundations/CI project; I haven't posted a status update on it before, but a few people have asked about it in private mail, so it might be helpful to have a public update to refer to.]
We've been working on reorganising the infrastructure for the live filesystem parts of our image builds. At the moment there's a single dedicated builder for each architecture that the central image build system (lp:ubuntu-cdimage, running on nusakan.canonical.com) can ask to produce various live filesystems; ubuntu-cdimage then fetches the results from it, performs some final polishing steps (for ISO images, this involves building a pool of .debs among other things, so is a fairly complex procedure requiring debian-cd and a local mirror; for phone images it's relatively simple), and publishes the output on cdimage.ubuntu.com. In the case of phone images, system-image then steps in to turn these into the system-image format and compute deltas etc. This is serviceable for what we're doing with it right now, but there are a few motivations for improving things which have been on the wishlist for some time: * It's not at all horizontally scalable. We notice this most around milestone releases when we try to do lots of image builds at once, and we can only do one at a time per architecture. This is basically why respin cycles for milestones take eight hours or whatever ridiculous amount it is. * We can't sensibly extend this to experimental builds. * To a first approximation, ubuntu-cdimage has only been deployed once (although I rewrote it a while back, and one of the goals of the rewrite was to make deployment easier). We discussed this at the release engineering sprint last year, and the general consensus was that we should move live filesystem building into Launchpad. This has a number of advantages: both build farms are normally well within capacity except for occasional bursts that tend to be at complementary times; we would be able to run many more image builds at once and thus drastically reduce the time required for a full respin; it would be much easier to monitor builds in progress; builds will be cancellable without sysadmin intervention; and we'll get one or two extra builders on most architectures by merging the build farms, at least where the hardware in use is new enough to make that worthwhile. I've been working on this on and off for a while, and essentially full-time since 14.04 released. The current status is as follows: * The launchpad-buildd package in production already has most of the code required to build live filesystems; this was done towards the end of last year. More recent testing uncovered a few small but critical problems with this code with various real-world image types (buggy handling of symlinks and of duplicate files in build output); one fix is in trunk and the other is awaiting review. * The bulk of the required changes to Launchpad itself have been through one review pass and are awaiting re-review. The required extensions to the web UI are still only on my local disk, as I have a few more tests to write first, but I expect to finish that in the first half of the coming week. * I have a local installation that is capable of building full live filesystem images, tested with both Ubuntu Desktop and Ubuntu Touch (i386). * I have a local ubuntu-cdimage branch that can request live filesystem builds from a local Launchpad instance and wait for them to complete. It cannot yet download the results. There's still a fair bit to do, but my aim is to have this landed and in production in time for the core sprint in Malta in the last week of May. The engineers developing the CI Airline need to be able to build live filesystems against PPAs as a component of their work, and so I've been in discussions with them to make sure that we will be able to use the same software for this, and where possible the same infrastructure, rather than duplicating work. In anticipation of this requirement, the database modelling for live filesystems in Launchpad includes the facility to associate a build with a PPA, although the pieces required to pass that through to the sources.list used to construct the live filesystem aren't quite in place yet. The CI Airline will need its own cut-down ubuntu-cdimage and system-image instances, and I'll be working with Paul Larson early this week to get the ubuntu-cdimage part of that up and running; phone image builds are simple enough from this point of view that I don't expect that to be difficult. Although I can't speak for system-image, I think we are well on course to have all the rest of this either done before Malta or easy to finish off there. Cheers, -- Colin Watson [[email protected]] -- Mailing list: https://launchpad.net/~ubuntu-phone Post to : [email protected] Unsubscribe : https://launchpad.net/~ubuntu-phone More help : https://help.launchpad.net/ListHelp
