So vUDS is behind us and it's time to solidify the cadence testing
schedule for Saucy. I've update the cadence page with actual dates now,
starting June 15th. See the schedule here:
https://wiki.ubuntu.com/QATeam/Cadence/Saucy
Now in addition to that, as part of the
https://blueprints.launchpad.net/ubuntu/+spec/community-s-quality-coverage
blueprint we discussed the idea brought up by crhrabal and smartboyhw
(thanks guys!). The outcome was an idea to change the way we do cadence
testing. The iead was to track all the packages that we care about for
the entire cycle -- things like our list of default applications
firefox, thunderbird, nautilus, etc. As a new build of the package is
published to the archive a new build is entered into the tracker and all
subscribers to that package are notified. I promised to mock up the
idea, and that's what I'm including below for discussion :-)
Let's step back quickly for a moment though. For those not familiar with
last cycle's cadence testing, let me describe it quickly. Every cadence
week we created a milestone and chose packages to test. In addition we
always tested the daily images during that week, as well as sometimes
including a bit of hardware testing against the milestone. The cadence
milestone was only open for the cadence week, after which the results
would be frozen.
Onto the mockups for the new idea! I've laid out two examples of how we
could implement the new idea.
The first shows the idea of lumping all packages into one milestone;
http://packages.qa.dev.stgraber.org/qatracker/milestones/252/builds. If
you then view the history
http://packages.qa.dev.stgraber.org/qatracker/milestones/252/history you
can see every package we're tracking, test results, and bugs. Clicking
on any old build let's you see the details as well.
The second shows the idea of giving each package a milestone;
http://packages.qa.dev.stgraber.org/qatracker/milestones/253/builds. If
you then view the history
http://packages.qa.dev.stgraber.org/qatracker/milestones/253/history you
can see only that package, test results, and bugs. Clicking on any old
build let's you see the details as well.
So what does this new idea do for us?
-- Let's us follow a package for the entire cycle, and provides bugs
linked to versions, and allows you to 'track' the status of the package
in ubuntu
-- Provides a summary report of bugs specific to that package that we've
opened
-- Allows you to subscribe to a package you like/care about and make
sure it's tested
-- Allows you to filter test results / versions / bugs by time
What I'm looking to gather now is if we should switch how we test our
packages as part of our cadence testing to the new system. Let me
describe how it would work.
Each cadence week we would:
-- Test the daily images
-- (Optionally, when requested) Perform laptop/hardware tests against
specific image
-- Test the packages we're tracking and ensure results are entered for
the current builds
The difference is that the milestones would be availible outside of the
'designated' cadence weeks and thus you are free to test the packages at
any time, as always, but you can also now report your results! The
cadence weeks stay a rallying cry towards us committing to test
regularly to ensure the archive, images and packages are in good shape
all throughout the cycle.
So, in summary, let's hear your feedback on:
1) Switching to the new idea for tracking packages all cycle
2) Lumping the packages together or making a milestone for each one
If we do decide to switch, we'll need to create a list of "packages we
care about" :-)
Nicholas
--
Ubuntu-quality mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-quality
