Hi! I'm opening this topic to discuss some TestCase refactoring I have been working on, as it might have a non-negligible impact.
The basic idea — which is not mine, see #20392 and that related discussion https://groups.google.com/forum/#!topic/django-developers/N0HEAD1ht8k — is to allow the creation of initial data once for a TestCase, within setUpClass. Currently, data created this way will persist between different test cases, thus breaking tests isolation. To maintain that isolation, the idea is to wrap the whole TestCase execution within an *atomic* block. Note that the isolation between individual test functions within a TestCase will be preserved as those functions are themselves called within an atomic block. I implemented that change in https://github.com/django/django/pull/3464, not without some difficulties. As a demonstration of how that change could speed up tests, I refactored the *queries* and *select_related* tests within django test suite to create initial data in setUpClass instead of setUp. The result is an approximate -70% execution time — indeed, I chose those tests because a lot of data is created and the result would probably not be the same with other tests: Using setUp(): $ ./tests/runtests.py select_related queries --settings=test_postgres ... Ran 253 tests in 10.637s Using setUpClass(): $ ./tests/runtests.py select_related queries --settings=test_postgres ... Ran 253 tests in 2.291s In order to make the whole Django test suite pass with that change, I had to adapt a bunch of tests. I described the technical issues in https://code.djangoproject.com/ticket/20392#comment:22 but the main problem is due to some tests in Django test suite relying on the fact that a new database connection is generated after each test, which is a behaviour that should change if we want to maintain a transaction at TestCase level. I don't know if a lot of TestCase out of Django test suite rely on that behavior (for instance, Django relies on that in a test which closes the connection explicitely) and those tests should work if turned into TransactionTestCase. Another issue I can think of is that if people start refactoring their tests to create initial data in setUpClass, the refactored tests will generate data that will leak between TestCase when run on a database which does not support transactions (e.g. MySQL with MyISAM engine). So, do you think it's worth implementing that despite the possible incompatibilities ? Do you see any other issue ? -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/eabd76ea-29c6-498b-b639-d94cf0a91d5c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
