Hello.
Over the past two weeks I've been prototyping a new tool for managing
releases of Linaro validation deliverables. Some of the problems are
unique (we target older releases, we are upstream) some are shared
(building lots of packages together, making sure they all work,
reproducing builds).
TLDR; Skip to the bottom of the message where I list the features.
I had some prior experience in this task and I wanted to see how the old
solutions would map to the new environment. As we target Ubuntu Lucid
aka latest LTS and most of us develop on the latest Ubuntu I wanted to
ensure that all of our releases can be installed and would work
correctly on Lucid. Earlier in the cycle we decided that the delivery
method would be a PPA with multiple .debs, some of which may be
backports required for Lucid. This is unlike using pip to pull in all
the things we are interested in from various places (possibly using a
requirements file).
So having those requirements my goals were to build a tool that would be
useful for those two tasks:
1) Daily development helper as a sort of CI tool.
2) Monthly release helper
Both tasks have different requirements but involve similar actions.
Those actions are:
*) Figuring out what packages to work on (aka project definition)
*) Getting the appropriate source/revision/tag from launchpad
*) Building a source packages for the target distribution
*) Building a binary package in pbuilder
- this is where we also run all the unit tests that tells us if
something is broken.
The tricky part is where we need to build test and release *multiple*
source packages from *multiple* branches and target *multiple* target
distributions.
For some concrete example. LAVA today is composed of the following core
packages:
- lava-server
- lava-dashboard
- lava-scheduler
- lava-dispatcher
- lava-tool
- lava-test (-tool)
- lava-dashboard-tool
- lava-scheduler-tool (upcoming)
We also have a set of support libraries:
- versiontools
- linaro-django-xmlrpc
- linaro-django-pagination
- linaro-dashboard-bundle
- django-restricted-resource
- django-debian
- linaro-json
We also have some libraries that are required for testing:
- python-django-testproject
- python-django-testscenarios
- python-django-testinvariants (upcoming)
We also have some backports (for both production and testing):
- python-simplejson
- python-django 1.2 (soon 1.3)
- python-django-south
- python-testtools
- python-testscenarios
- python-fixtures
Now that's a lot of stuff to build and release manually. Granted not
everything is changing but at the very least the first group (lava-*)
will see a lot of activity each month.
Now the way I see it, each month, this list needs to be released.
Possibly some of them will be unchanged. In that case there is no need
to actually upload new packages to a PPA. Still we need to ensure that
all of them build and work on all Lucid, Maverick and so on, all up to
the latest version of Ubuntu.
Since we want to build Debian packages I decided to use bzr builder to
create source packages from a recipe files. Recipes are a simple (2-3
lines at most) text files that say how to assemble source branches to
get Debian packaging. The key feature of builder is it's ability to
create derivative packages for a particular distribution. Recipe files
are also used by launchpad for building source packages. In the future
this tool could actually push/pull the recipes to launchpad directly.
To build binary packages I used pbuilder. The configuration is a little
more complex than simple raw pbuilder or pbuilder-dist. The
configuration I made turns it into a ppa-like builder that can feed from
it's own packages. Each time you build a package it will land in a small
apt repository and can be used to feed another build. This is a critical
feature as all of our packages depend on something not available in the
main archive.
The trick is to know the right sequence of builds that would satisfy
build-dependencies. I did not find any existing tool to do this and
after a brief discussion with doko it seems there are no such tools
available. In general the problem can be simplified to a subset of real
dependencies (without conflicts, virtual packages and stuff like that)
and resolved automatically. I did not want implement that as our package
list can be managed manually. A special sequence file contains a list of
"things" to build, in order, to satisfy dependencies.
So getting everything together, lava-ci does the following:
* FEATURES *
$ ./lava-ci project help
Usage: lava-ci project [COMMAND]
Summary: Manipulate and release your project
Available commands:
- init - Prepare lava-ci to work with a particular project
- help - This message (default)
$ ./lava-ci distro help
Usage: lava-ci distro [COMMAND]
Summary: Manipulate pbuilder environments
Available commands:
- help - This message
- show - Display list of distributions and their status (default)
- enable - Enable one or more distributions
- disable - Disable one or more distributions
- update - Update one or more distributions
$ ./lava-ci package help
Usage: lava-ci package [COMMAND]
Summary: Manipulate source and binary packages
Available commands:
- srcpkg - Create a source package from a recipe
- help - This message
- show - Display list of packages and their status (default)
- wipe - Remove all source and binary packages
- sequence - Build a sequence of packages following
- binpkg - Create binary packages from a source package
A subset of the README file:
lava-ci - CI and release tools for the LAVA project (and others).
The purpose of this tool is to facilitate teamwork and monthly releases by
automating the act of creating and testing a release from source
repositories
all the way to the binary packages for various platforms.
Workflow example:
*) Prepare recipe files (bzr help builder) for each maintained source
package and put them in a bzr repository. Each recipe file *MUST* be
named like the source package it builds.
*) Use `lava-ci project init lp:BRANCH-WITH-RECIPES`. This will create
.lava-ci directory and a local checkout of you recipes in the
current directory.
*) Use `lava-ci distro` to list the distributions you wish to target.
For each one you are interested in do `lava-ci distro enable
$distro`. For example to enable lucid and maverick you could use
`lava-ci distro enable lucid maverick`.
*) Use `lava-ci package` to list available "packages" (based on
recipes). You can add/edit more just by creating additional
*.recipe files.
*) Use `lava-ci package src $package` to build source packages for
the selected recipe. There will be one source package per
distribution. You can check `lava-ci package [show]` to see what
source packages are already available.
*) Use `lava-ci package bin $package` to build binary packages for
the selected recipe. Again there will be multiple builds, one for
each distribution. Each build will result with a number of
additional debian packages being produced. You can use those
packages as build-dependencies of other packages you maintain,
similar to a Lauchpad PPA.
*) Write down a sequence of build operations to perform in and save it
as `recipe/sequence`. This file should contain one word per line
- the name of the source package to build. The order will depend on
the build-dependencies among your packages. To test that building
the whole sequence works do `lava-ci package wipe` followed by
`lava-ci package sequence`. If the build succeeds you can tag your
recipes as a working configuration and make a release.
Best regards
Zygmunt Krynicki
_______________________________________________
linaro-dev mailing list
linaro-dev@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-dev
