Test framework for end-user applications

[Russell Keith-Magee]

Hi all,

I've got a project at work that requires a formal test harness, so I thought I would kill two birds with one stone and spend some time adding a formal test framework for all Django applications.

This came up a few weeks ago [1], and although I didn't comment at the time, it struck me as a good idea. Plus, as Simon pointed out, it is a feature that Rails has but Django does not, and we can't let those dirty Rails-using savages have something that we don't :-) I have created ticket #2333 to track the feature.

As a first pass, I have been concentrating on adding the framework to find and run unittests and doctests within a Django application. This is essentially what tests/runtests.py already does for the Django tests, except that I am using the unittest framework to find and execute tests.

In my sandbox, I have made modifications such that:
- Any classes in models.py that extend unittest.TestCase will be identified as a test case.
- Any doctests in models.py will be extracted as test cases (using normal doctest extraction strategies - any module, class, or method level doctests, plus anything listed in a __test__ attribute)
- If a mysite.myapp.tests module exists, it will be searched for unittests and doctests in the same way as models.py

I have also added two targets to django-admin:
- 'test' which runs all the unit tests for all installed applications.
- 'testapp' which runs the unit tests for all applications named on the command line

With this new framework in place, it is possible to refactor the django tests; to this end, I have moved 'othertests' into the 'regressiontests' package, so that it looks like an 'application' within the regressiontests 'site'.

When a test targets is executed, django-admin:
1) creates a test database named 'test_' + settings.DATABASE_NAME
2) Installs all the applications in INSTALLED_APPS
3) Constructs a test suite from the test cases found in the requested apps
4) Runs the suite
5) Destroys the test database

Note that at the moment, I'm just looking at getting a basic testing framework in place - after that, I will be looking at other problems such as:
- installation of 'fixtures' (YAML/JSON/Python data for testing purposes).
- integration with selenium type tests that require a running server

I figure that once we have a generic testing framework, we can wrap just about anything else into it (for example, using purpose-specific subclasses of unittest.TestCase for different test setup/teardown sequences).

I'm almost ready to submit some patches (I still need to do some code cleanup and write some documentation), but before I do:
1) Are there any comments on the approach that I have suggested?
2) Is this a big enough change that people want to see the patch before I commit it? It shouldn't actually break anything already in place, just expose some new functionality that might be useful.

Russ Magee %-)

[1] 'Site Testing how-to', http://groups.google.com/group/django-users/browse_thread/thread/24a50006c43f43b5/eee75d48505d1af5


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/django-developers
-~----------~----~----~----~------~----~------~--~---