On Tue, 15 Nov 2011 12:34:48 -0500, Juan Luis Baptiste <[email protected]> wrote:
I wrote about this some weeks ago but it went unnoticed[1], about a cross-platform tool developed by OpenSuSE that is for doing all kind of automated QA tests[2]. That tool looks very promising and helpful. [1] http://thread.gmane.org/gmane.linux.mageia.devel/8661 [2] http://news.opensuse.org/2011/10/11/opensuse-announces-first-public-release-of-openqa/
There was brief discussion about it on the qa discuss mailing list. I have experience with automated testing going back to the early 80's. This was on ibm mainframes, starting with Batch Terminal Simulator, and many others packages, often developed in-house. As soon as I saw that the testing scripts are using hashes of screen shots to determine if a test was successful, I cringed. You better not have a clock displayed, during the test. When you know *exactly* what the expected output will be, the tests can be automated. Otherwise, you'll spend more time tweaking the test cases, rather then testing the updates. Where the qa team does have problems, or encounter delays, at present, there are two main reasons. Lack of hardware. For example, mtd-utils is currently on hold pending testing by the person who requested the package, as no one on the qa team has an mtd flash device. Time and knowledge required to set up a test environment. For example, in order to test gssftp from the krb5-appl-servers package, the qa tester has to set up and configure a kerberos server, with an appropriate backend such as ldap. There are only a few cases where repetitive testing is involved, where an automated procedure would be useful. Regards, Dave Hodgins
