On Sat, Apr 10, 2010 at 2:25 PM, Alex Gaynor <[email protected]> wrote: > On Sat, Apr 10, 2010 at 2:21 AM, Russell Keith-Magee > <[email protected]> wrote: >> On Sat, Apr 10, 2010 at 8:29 AM, Gabriel Hurley <[email protected]> wrote: >>> Maybe it's an overly simplistic question, but: what makes the tests >>> slow currently? It's not simply the volume of them. It's more than >>> possible for Python to race through hundreds of tests per second under >>> the right conditions. >>> >>> Some of Django's test modules obviously zip along, but others seem to >>> take a rather long time to complete individual tests. Has anyone >>> identified which tests are the slowest and what is making them so >>> slow? Is it the length of the tests? The parsing? The fixtures that >>> are loaded? The data generated in the test itself? >> >> Yes. :-) >> >> There are multiple factors at play here. The first is obvious if you >> run the test suite right now using different backends: SQLite is much >> faster than Postgres, which is much faster than MySQL/MyISAM. This is >> because SQLite uses an in-memory store, so it isn't disk bound; >> Postgres is disk bound, but is able to use transactions to optimize >> test setup and teardown; MySQL is also disk bound, but doesn't support >> transactions, so there are a lot of "CREATE DATABASE; do one thing; >> DESTROY DATABASE" calls in the lifepan of the test suite. >> >> The move to using transaction based tests in Django 1.1 sped things up >> a lot (for Postgres and SQLite, anyway); but there is still room to >> improve. The biggest single problem that I am aware of is that even >> though tests are composed in TestCases that have common setup/teardown >> routines, each test is handled separately. As a result, if you have a >> test case with 10 tests and a complex fixture setup, the fixtures are >> read from disk 10 times, loaded into the database 10 times, and rolled >> back out of the database 10 times. Ideally, this should only happen >> once, but Python's unittest doesn't (currently) provide easy hooks to >> do TestCase-wide setup. >> > > Spoke too late ;) Python 2.7 SVN now has setUpClass and tearDownClass.
Yes, it does, but we still need to support Python 2.4, so we can't just start using those features -- we need to have a migration strategy. However, if we determine that unittest2 will give us a major performance boost, I have no particular objection to shipping unittest2 with Django as a support mechanism for Python 2.4-6 (in the same way that we ship a copy of doctest.py). Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
