On Fri, May 03, 2013 at 11:03:28AM -0400, Mark Stosberg wrote:
> > No, you can't have your tests clean up after themselves. For two
> > reasons. First, that means you have to scatter house-keeping crap all
> > over your tests. Second, if you have a test failure you want the
> > results to be sitting there in the database to help with debugging.
> There is another way to have tests clean up after themselves, which
> addresses both the shortcoming you address.
>
> First, I have functions like "insert_test_user()". At the end of these
> functions, there is another call like this:
>
> schedule_test_user_for_deletion(...)
>
> That inserts a row into a table "test_ids_to_delete" which includes
> column for the table name and primary key of the entity to delete.
> Another column has the insertion timestamp.
>
> So, there's no "clean-up" code in all of our test scripts, and we have
> the data there for debugging when the tests are done.
OK, but you still have to clean out your database before you start each
independent chunk of your test suite, otherwise you start from an
unknown state. You might, for example, start with broken data still
hanging around from a previous test failure, which then causes your
correct code to fail, which is a massive pain in the arse if you are, as
you should be, using your tests as a debugging aid.
Consider, for example, your test for "retrieve a list of all customers
from the database and inflate them to objects". If some previous test
left a broken customer record in the database, then your code for
inflating customers to objects will (correctly but unexpectedly) fail.
If you want to paralellise tests that use a shared resource, such as a
database, then you will normally need to mock that shared resource.
> In Jenkins, after the test suite runs, a job to "Delete Test Data" is
> kicked off, which deletes all test data older than hour. (Longer than it
> takes the test suite to run).
What about when you're not running under Jenkins. Like when you're
writing and testing your code. You still need to start testing from a
known state each time, which means you must clean out the database at
startup.
--
David Cantrell | Cake Smuggler Extraordinaire
Nuke a disabled unborn gay baby whale for JESUS!
