Hi, I was told that QA Team's work's visibility needs to be improved, so as a team member I'll try to give you some sort of status report.
QA team currently mainly works on Mageia 1 updates testing and validation. We managed to gather various volunteers, and while we probably need still more volunteers, at least there's already a lot of work being done and updates are being pushed. *** Updates pushed *** Since the updates media opened, 65 updates have been pushed (in terms of packages, a lot more since an update can contain several packages). - June: 9, including nvidia drivers and system-config-printer - July: 46, including kernel, sun's java, iptables, flash-player-plugin, kernel, logrotate... - August: 25 as of today, including KDE SC 4.6.5, libreoffice, flash-player- plugin The number of real bugfix or security updates is approximatively 3/5 of those, because we also pushed new packages that were present in Mandriva 2010.2 but absent from Mageia 1, as the policy permits (exception for Mageia 1): There is a mailing list where updates are announced, which is what I used to count the updates for this report. See https://ml.mageia.org/wwsympa-wrapper.fcgi/arc/updates-announce *** Pending updates *** Right now there are 16 pending update candidates (in fact 18, but I excluded one that is not ready to be tested, and one which is in fact a backport request waiting for the backports media to open). See https://bugs.mageia.org/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=waiting%20for%20QA%20test&sharer_id=22 5 are recent (assigned to QA for less than one week) and being taken care of The others are less recent (more than one week, sometimes even more than one month) : - 7 are waiting for packager input, QA testing having uncovered bugs and/or raised questions - 2 have been in need for testing for several weeks, here QA team must wake up :) - 1 has been validated by QA one month ago, but was assigned to security team following updates policy for security fixes, and got not answer. We have to improve either the policy or the security team here (or both). - 1 is ready to go, we just have to tag it as such and pass it to the sysadmins - the last one is firefox: this is probably the longer pending update (started as firefox 5, now is firefox 6), and that with the higher visibility for users. I would like to let dmorgan and those involved in its testing comment about it : what caused such a long delay ? I'm not trying to blame anybody, especially dmorgan who did most (it not all) of the hard work: I'm genuinely curious about the causes of this delay so that we can explain it to our users, to ourselves and above all learn from it. *** Short analysis (my opinion) *** Most updates passed to QA went fine, quite well tested. There's still a lot to improve in the QA team's work, but right now we kind of manage to handle the update candidates. Updates are probably not as thoroughly tested as we would like to and often we have to trust that the bugs the upstream project claims to have fixed are fixed. But we always try to check that there are no regressions. What can delay an update is mostly: - bugs or other problems found during testing. An exchange starts between the packager and the QA team and it can become very long. The above report shows that currently there are more reports waiting for the packager's return than waiting for QA feedback. - testers not knowing what to test, thus giving their time to other bug reports. We can't know every package as well as the packagers know them. When the packager provides good test cases, it can speed up the process drastically. - lack of testers (not the majority for the current pending updates). It happens mostly when we need testers with specific configurations or hardware. Here stops my own analysis, let the rest of the packagers and testers give their views :) *** Need a broader view *** This report would be more complete with information about current and past open bugs on Mageia 1 (and pending security issues), but this would go farther than the purpose of this QA team report. I'll let the triage team give us its report about pending bugs, and someone (stewb ?) tell us about pending security issues. Best regards Samuel Verschelde
