On Sat, 2002-04-27 at 23:15, Larry Kollar wrote: > > When I do QA+RM for a customer for Red Hat/eCos it costs me at a minimum > > a full days work to follow a QA sheet by hand and go through the > > motions, do the README, etc.. We have pretty much a turn-key release > > system, and before a release we have run 12k+ tests in our test farm. > > It's a chore to do - but I do it because it's necessary and because I'm > > paid to do it. > > A QA "sheet"? As in singular?
Yeah. Well, the "sheet" is a web page. A long one :) It would print to multiple sheets, no doubt. But what we're testing is a bootstrap monitor - not a big thing to test. The automated testing does most of our coverage (luckily!) > - Unless someone can provide us with a ready-made form > for creating test cases, that needs to be done. I'll > "take that action" if nothing pre-made is forthcoming. > I'd *like* to see a web form that allows over-the-Net > entry of test case results; or users could email them > to abi-dev or perhaps a mailing list created for the > purpose. (The web form could also do the mailing.) Web form would be best; allowing a pool of people to help: Number Test title Assigned State ... 101. Lists 0% done, 0% PASS, 0% FAIL 101.1 List creation 1 unassigned N/A 101.2 List creation 2 unassigned N/A ... Total 76% done, 98% PASS, 2% FAIL Clicking test title should pop up a window with a description of the test that needs to be made, and if the test has been assigned, that page should also have PASS/FAIL/SKIP buttons. The assignment part would be nice to be without - but my thought is that someone deciding to test an entire section (such as "Lists") could lock those subtests so someone else didn't step on his toes during the process. Maybe we could start without it, and just point people to join #abiword-qa to coordinate before doing any QA. Btw, I have started to play a little with the AbiCommand plugin to make an automatic testing infrastructure. There are limits to how much can be tested automatically though - the user interface, which is a big part of a WP, must be tested by hand, at least. Cheers, Jesper
