Hello Ubuntu developers, the second monthly cycle of our new Stable+1 maintenance team [1] rotation is almost over. In December, the team consisted of James Page, Allison Randal, and myself.
James and Allison are already on vacation, and yesterday was my last day, so time to summarize our stable+1 activity for December and describe the current status. Daily ISO --------- https://jenkins.qa.ubuntu.com/view/Precise%20ISO%20Testing%20Dashboard/view/Daily/? This was our topmost priority, Martin was responsible for watching and fixing these, with help from James and the server team for non-trivial server specific issues. We succeeded in getting working daily ISOs for desktop, alternate, and server every workday, and on most weekends. There was a weekend following a large swamp of syncs/Haskell transition/etc. uploads which caused a huge buildd lag and some uninstallability, but at that point we could not do anything about that any more except for making sure that the packages on the ISOs build first. The situation on ports images has been a bit worse. We often experienced large armel backlogs because a lot of the armel builders were borrowed for the armhf bootstrap. But as this is over now, we should have enough armel builders again, so I don't think this is an ongoing problem. powerpc is worse, though, the buildds almost always have a backlog of 15 hours or more, so we often saw ISO build failures there. But as nobody really cares much about these any more, we did not waste any time on those and just let it sort itself out eventually. We retired the manual tracking of the ISO status [2] and are exclusively using Jenkins now. Testing ------- https://jenkins.qa.ubuntu.com/view/Precise/ The QA team has done tremendous work on keeping the automatic ISO and upgrade testing working and made a lot of improvements there. From the stable+1 team side this was James' work area in December. James added a number of interesting test cases: * LVM multi-partition mode with separate /usr/, /var/, /tmp/ etc. * RAID-1 tests * EC2 tests: the "precise-server-ec2" one is known-broken right now, but -adhoc and -daily work * started working on Openstack tests Now all the tests pass, except: * precise-server-ec2 (see above) * precise-server-*-virtual: it's "yellow", i. e. install succeeds, but post-install tests fail. That's because they are currently oversized by ~ 50 MB, the documented limit is 500 MB. Either the server team needs to downsize the footprint, or raise the documented limit. * upgrades (see below) The daily tests were tremendously helpful for discovering regressions which we otherwise wouldn't have noticed until the next (i. e. alpha-2) milestone testing. Now we could fix them right away, which is both a lot easier (as the offending change was very recent) and also takes the pressure out of the alpha-2 release engineering. Uninstallable packages ---------------------- http://people.canonical.com/~ubuntu-archive/testing/precise_probs.html At the start of December there still were a number of long-standing uninstallable packages. Martin took over the responsibility for this area. All uninstallable packages were fixed in the first days, and since then our #1 priority was to keep this at zero. The only prolonged time period where it wasn't zero for about three days was around that huge buildd lag weekend, where armel was uninstallable. Martin also worked on improving the report to actually say _why_ a package is not installable. This was a snapshot when the script was improved: http://people.canonical.com/~pitti/tmp/precise_probs.html On the ports side [3] there are still some problems. Most of it right now is just buildd lag again (seriously, we either need to kill powerpc or give it more buildds..), but the only "real" issue here is that libreoffice fails on armhf. This is allegedly being worked on between Matthias, Bjoern, and upstream, but I'm not sure of the current status. Upgrades -------- https://jenkins.qa.ubuntu.com/view/Precise/job/precise-upgrade/ This was mostly Martin's area during that stable+1 cycle. We now have a fairly large coverage of upgrade scenarios, including an upgrade of the entire lucid main to precise (which is certainly the hardest one to get right, but at the same time is also the most useful one to discover problems). As you can see from the history we never managed to get this fully working yet, but Martin spent about half of his time on upgrade bugs (e. g. #900004, #897880, #901572, #902077, and lots more). Just today all the upgrade scenarios actually succeeded without failures in apt and during the upgrade, we now "just" fail some post-upgrade tests. Apparently some Python modules stop working after the upgrade, there was a dpkg conffile prompt in mailman (we used to have lots more, but most of them got fixed), and powernap crashed [4]. However, they currently cheat a bit: The main-all test currently blacklists the "tdsodbc" package, which is LP #901638. We already uploaded a fix for this, but had to revert it because it breaks K/Edubuntu, so we need to convert soprano-daemon from the old unmaintained iodbc2 to unixodbc. Martin has a first attempt in a branch/PPA, but it doesn't work yet. So this is definitively an area where the January team should continue on, to iron out the remaining issues and then keep them working. NBS --- http://people.canonical.com/~ubuntu-archive/nbs.html This was the second-biggest chunk of work for us. Both James and I worked on that pretty much every day. The most important and time-consuming one so far was the poppler transition, which included some nontrivial API changes/drops, but is fully finished now. But these days new issues come up about as fast as we are able to fix them (mostly through autosyncs, but some also through Ubuntu uploads), so this is an area which needs constant attention and a huge amount of work. Outstanding issues: * libcegui-mk2-1 - still two packages outstanding with non-trivial fixes - bugs raised in LP/Debian: https://launchpad.net/bugs/903121 https://launchpad.net/bugs/903126 * libhpdf-2.1.0 - One outstanding (witty) - needs work upstream; other two are just because of waiting for Godot^Wpowerpc buildds. * simgear2.0.0 - non-trivial fix required for transition - bug reported in debian and upstream working on new release: https://launchpad.net/bugs/903225 * libahven17.0 - this has crept through from Debian pre-transition completion of gnat-4.4 to gnat-4.6 into testing as it does not comply with the Debian Ada Library Policy in that it only depends on gnat-4.6 (not gnat and gnat-4.6). Ubuntu already builds gnat from gnat-4.6 so we can't actually re-build any of the Ada libraries which are policy compliant ATM. * gnutls-bin - This is now built by gnutls28 in Debian, which we don't have in Ubuntu yet. Suggested plan of action: Verify that we can build all of main against gnutls28 with reasonable porting effort, sync gnutls28 into main, do the transition, drop gnutls26. Debian is undergoing this transition now, I think, so we can cooperate with them and send patches. If porting to 28 involves too big hurdles, the fallback is to build gnutls-bin from gnutls26 again (i. e. partial revert). * kalgebra-dev, libanalitza4, libanalitza4-dev: New kalgebra was uploaded to Ubuntu instead of the PPA. Kubuntu team asked us to leave it that way for the time being, though, they promised to sort it out. * libkdb5-5: New krb5 came through autosyncs with new build dependencies libverto and libev. libverto needs a MIR (LP#906281) and be changed to build against libevent instead of libev. * vtk: This is another bigger one which came in through autosyncs. We could not work on this yesterday yet, because it still needed building on some architectures, but hopefully will just be mostly no-change rebuilds. priority/architecture mismatches -------------------------------- http://people.canonical.com/~ubuntu-archive/priority-mismatches.txt http://people.canonical.com/~ubuntu-archive/architecture-mismatches.txt This was a rather easy topic. We fixed all outstanding issues, blacklisted the one that we cannot technically fix (hfsutils on powerpc optional → standard, Launchpad cannot do per-architecture priority overrides), and since then kept it at zero. As these reports are not usually looked at very often, we integrated them into Jenkins, so that we see the red dot popping up when problems come up. component-mismatches -------------------- http://people.canonical.com/~ubuntu-archive/component-mismatches.txt http://people.canonical.com/~ubuntu-archive/component-mismatches.svg Martin worked on improving the textual report to point to MIR bugs and also mark packages with (MAIN) which are already in main, i. e. the root of the chain. He also added an SVG report to give a more easily readable way of the source/binary universe → main promotions with clickable nodes (-> MIR bugs). We resolved most of the issues and kep the list as small as possible Outstanding issues: * openstack seed wants "keystone", but this still needs a MIR completion. This does not break any image builds (they will just ignore unavailable seeded packages), the server team asked us to leave it seeded for now. * krb5 -- see above in NBS * koffice-l10n demotion: Jonathan asked to leave it in main until he tested that updates correctly move over to the new "calligra" which replaces KOffice. FTBFS ----- http://qa.ubuntuwire.org/ftbfs/ Allison started looking into this and fixed a couple of long-standing ones like magic++. James fixed ~25 bugs relating to source file encoding errors with Java 7 and one relating to public API changes for JDBC4.1. Still lots more work to be done: https://wiki.ubuntu.com/JavaTeam/Java7Default. Otherwise we did not explicitly work on FTBFS a lot, mostly because our time got fully eaten up by fixing installability, ISO failures, upgrade bugs, NBS, and component-mismatches. This is a very active time of development, so apparently two full time engineers are still not enough with keeping up with all these and fix FTBFS. So this remains to be a major area of work. Other infrastructure -------------------- http://conflictchecker.ubuntu.com/possible-conflicts/ http://harvest.ubuntu.com/ As already described above, both Jenkins and the archive consistency reports got some improvements. We did not start fixing the possible-conflicts reports to ignore all the false positives, or look into improving Harvest, so these also remain outstanding. Summary ------- I am very happy with how this new a stable+1 team works. It it a huge amount of work, but the benefits (timely feedback, keeping everything installable, quickly fixing regressions) are well worth it. I have heard quite a lot of positive feedback on IRC or elsewhere, in the tone of "how do you guys make precise so stable at that early time?", and I think we also have a lot more Canonical folks running precise now than we used to have in earlier development releases. I certainly enjoyed playing firefighter for that month, keeping precise working, and also working on some report improvements. Thanks, and enjoy the end-of-year holidays! Martin [1] https://wiki.ubuntu.com/PlusOneMaintenanceTeam [2] https://wiki.ubuntu.com/PlusOneMaintenanceTeam/Status [3] http://people.canonical.com/~ubuntu-archive/testing-ports/precise_probs.html [4] https://jenkins.qa.ubuntu.com/view/Precise/job/precise-upgrade/PROFILE=main-all,label=upgrade-test/80/ -- Martin Pitt | http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
signature.asc
Description: Digital signature
-- ubuntu-devel mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
