Bill:
One strategy that improves testing results is to correctly define what a
successful test is. A successful test is one that *finds* errors in the
code. If no errors are found, then either there aren't any (and that's
highly unlikely,) or the tests aren't good enough to trigger the
problems.
Ideally, you get a hold of someone who knows the application very well,
and has a demonic streak. Have that person follow the strategy Ken Hall
suggested in the earlier post. If they are immediately successful (i.e.
finding errors,) track the rate of error discovery. When the rate falls
below, say one per day, you might be at a cutoff point. Get a second
opinion from another destructive expert if you can, particularly if the
first fails (i.e. finds no errors.) Basically, nothing beats an
application expert bent on breaking the software for finding bugs the
fastest.
What your boss is proposing to do is to kill several hundred bacteria by
boiling the ocean. Managing test case coverage for a typical application
is a huge task. And when you'll get a positive ROI is unclear: At what point
will the test system have enough critical mass (test coverage) to actually
demonstrably improve reliability and user satisfaction?
Lee
On Tue, 22 Sep 2009, Brutzman, Bill wrote:
My boss has a new grand scheme for software testing. To test each
program revision, he wants scripts to inject sample data into user apps
and then another (big) program looks at EVERYTHING that comes out,
comparing it to a baseline.
To me, the scope of this project is huge, perhaps bigger that our ERP
system. So far, I have been unable to talk him out of it.
Thus, I am wondering what other people are doing in the way of software
test. Virtually all of the source code that we have is in UniBasic.
I do not know what jargon to use to describe the test that I do now. On
a spreadsheet, I layout the various possiblities and then try each
scenario... One-by-one... Fixing bugs as I find them and then starting
the batch of tests from scratch.
--Bill
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
------------------------------------------------------
Lee J. Leitner, Ph.D. [email protected]
http://www.leitner.org/~leitnerl
The world can only be grasped by action, not by
contemplation. The hand is the cutting edge of the mind.
-- Jacob Bronowski V.13.0
-------------------------------------------------------
_______________________________________________
U2-Users mailing list
[email protected]
http://listserver.u2ug.org/mailman/listinfo/u2-users
