On 07/18/2012 01:58 PM, James Holland wrote: >> Would love to see this testing process/code documented somewhere and >> what the implications are. So we might be able to better and improve the >> process in CentOS as well. > > Is there a place where you document the CentOS processes? >
we are starting to, the QA team has been in place for about 4 years now and we try to cover as many roles/deployment types as can. There is quite a bit of documentation on the wiki, additions and contributions to that are welcome. I've personally spoken at Fosdem earlier this year on the Continuous Integration process we are working on ( http://ci.dev.centos.org/ is the 'public dev' side for this, with the t_functional suite triggers in place now ). The Distro/tree tests, which themselves number 150+, is something I'm working on to get into the same harness before we hit CentOS-5.9 / 6.4. Problem with that is as the other test suites before, its been built in place to cater to a very specific setup working in a very specific state - and because they consider the entire 'distro tree' ( ie. the mirror.centos.org layout ), its not exactly portable. I hope to have that fixed and onto the gitorious setup soon. the actual test harness around it does dozens of installs trying to cover every possible install role and type ( eg. we test xen, kvm, virtualbox, vmware, real machines for http installs, iso installs, ftp installs, nfs installs. And were using local as well as remote storage in each of those setups ). If you now write up a matrix, just the installer is getting well exercised. We're hoping to expand this into the next release by adding more install environments and roles per deployment (eg. most of the testing now is done for a base setup and the test suites being run inside each base setup per environment ). The plan is to expand that into addressing different roles ( please contribute kickstarts! ) as well. Its worth noting here that this is not something we do on a per-release basis, the entire test suite is something we aim to run on every individual package build + release. For release time work, the QA team guys do manual tests over and above everything else. To some extent, I feel the 'QA team' term is no longer really relevant, these guys have mostly transitioned into being the Release Team now. As before, contributions and additions to the test harness, the test suites, the automation and even ideas for testing etc, are welcome. -- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax GnuPG Key : http://www.karan.org/publickey.asc
