--- Tony Bowden <[EMAIL PROTECTED]> wrote:
> We insist on a full regression test on checkin, so having the test suite
> take a long time to run puts people off making small changes. :(
>
> On one project a full regression test takes about 8 minutes, which is at
> least an order of magnitude longer than I'd like. I couldn't dream of
> having it take an hour!
Unfortunately, that's something we seem to be stuck with here. At my previous job, we
had one
test suit with a similar number of tests, but it took -- at most -- about 3 minutes to
run. The
problem is that test driven development started here before the current testing
craze. As a
result, my fellow programmers have some curious notions about testing. Amongst other
things, each
file is a package that has several TEST subroutines in it. These tend to have the
following
characteristics:
sub TEST__WE_CAN_ADD_FOO_TO_BAR
{
make_up_test_data();
add_test_data_to_database();
my $foo = function_to_test();
assert($foo, $some_made_up_val);
}
Now, before that function is run, we automatically have a "setup_test" function that
runs. After
that function runs, a "teardown_test" function runs (which typically does a rollback).
So we have to make up a bunch of dummy data, insert this data, test this data, and
roll back this
data FOR EVERY TEST. Further, because of cut-n-paste testing, much of the dummy data
that is
inserted has nothing to do with a particular test and therefore is useless overhead.
Not only
does this slow the test suite to a crawl (and this is with Oracle, no less), I've
asked some of my
fellow programmers if they see a problem with testing made-up data and all I get are
blank stares.
It's similar to why programmers should not do their own acceptance testing: two
minutes later
they send an email saying "yup, it does what I programmed it to do."
To make matters worse, I've also asked "have you ever considered testing the data that
you have in
production?"
The answer, from every programmer I've talked to, has been "how can we test the data
if we don't
know what it is?" The programmes working here haven't worked with any other way of
testing so it
never occurs to them that there might be another way.
This company is doing some great work, but sometimes I want to scream.
Cheers,
Ovid
=====
Silence is Evil http://users.easystreet.com/ovid/philosophy/indexdecency.htm
Ovid http://www.perlmonks.org/index.pl?node_id=17000
Web Programming with Perl http://users.easystreet.com/ovid/cgi_course/
__________________________________
Do you Yahoo!?
The New Yahoo! Shopping - with improved product search
http://shopping.yahoo.com
