Monty Taylor: Hudson Parameterized Matrix Builds
I've been making some improvements to our use of Hudson recently that have been really helpful.
The first was starting to use a set of parameterized builds. These use the Parameterized Trigger plugin, which allows you to pass parameters to triggered additional jobs when a job finishes. So using these we make a job which checks out the latest source (which should just about always succeed) and then fire off a whole host of jobs running on all of our build hosts. It has a single "build now" submit form which takes a bzr branch location as the branch to build. With this all of our developers can check their tree against the build farm before submitting the branch for merge.
That's great, but as we got more and more build hosts, we had to set up a job on each of them - and then having 4 machines which were all amd64 ru! nning Ubuntu 9.10 didn't help - it ran on all of them.
Enter PlatformLabeler
Robert Collins wrote a plugin for Hudson which checks a build slave for OS details when it connects (architecture, OS and OS Release) and adds Labels to the slave based on the values it finds. With that - instead of targetting a job to say, hades.inaugust.com, I can target that job to x64_64-Mac-10.4.1. I can easily see both platform coverage, and I can take advantage of having multiple machines with the same config, since it'll run the job on only one of them sharing that label.
The next problem with that setup is that there is not a tight binding between the parent and child jobs. You can see that the parent job triggered the child, but you can't really get a single snapshot of "how did this job work everywhere?&q! uot; If you're doing this pre-merge request, it might be nice to get a URL you can refer to saying "hey! check it out, it worked"
Enter Matrix Builds.
Matrix Builds are a single job. They allow you to define how you run the job, and then you can select which hosts to run it on. You can also set up arbitrary sets of flags and hudson will build a grid. So in our case, I set up a job (shortened for brevity)
./config/autorun.sh
./configure
make distcheck
Then I told hudson to build it both with and without the --with-debug flag to configure, and on a set of our build hosts. Now when I kick it off, I get a page with an overview of how that particular build went. Also, having one build description means I don't accidentally do something different on one host.
Matrix Builds also have a nice feature for a sentinel or canary build, which let you build first on a host where you're pretty darned sure it's going to build, and then build everywhere else. In our case, if it do! esn't build on 64-bit Ubuntu 9.10, it's very unlikely to build anywhere, so we don't bother trying.
I still use the Parameterized Trigger from the Matrix build to kick of jobs for things like Valgrind and Sysbench when the build completes cleanly. Those jobs themselves can be launched independently (if you only wanted to check how valgrind is treating your new branch)
Moving forward, I would love to migrate our main set of build branches to Matrix builds, but Matrix currently lacks support for launching an individual sub-build. If it doesn't show up soon, perhaps I'll add it... ah the joy of open source!
See the Hudson link at http://hudson.drizzle.org/view/Drizzle-param/job/drizzle-param/ for more details and examples.
URL: http://inaugust.com/post/76
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
