On Sat, Apr 10, 2010 at 8:29 AM, Gabriel Hurley <[email protected]> wrote: > Maybe it's an overly simplistic question, but: what makes the tests > slow currently? It's not simply the volume of them. It's more than > possible for Python to race through hundreds of tests per second under > the right conditions. > > Some of Django's test modules obviously zip along, but others seem to > take a rather long time to complete individual tests. Has anyone > identified which tests are the slowest and what is making them so > slow? Is it the length of the tests? The parsing? The fixtures that > are loaded? The data generated in the test itself?
Yes. :-) There are multiple factors at play here. The first is obvious if you run the test suite right now using different backends: SQLite is much faster than Postgres, which is much faster than MySQL/MyISAM. This is because SQLite uses an in-memory store, so it isn't disk bound; Postgres is disk bound, but is able to use transactions to optimize test setup and teardown; MySQL is also disk bound, but doesn't support transactions, so there are a lot of "CREATE DATABASE; do one thing; DESTROY DATABASE" calls in the lifepan of the test suite. The move to using transaction based tests in Django 1.1 sped things up a lot (for Postgres and SQLite, anyway); but there is still room to improve. The biggest single problem that I am aware of is that even though tests are composed in TestCases that have common setup/teardown routines, each test is handled separately. As a result, if you have a test case with 10 tests and a complex fixture setup, the fixtures are read from disk 10 times, loaded into the database 10 times, and rolled back out of the database 10 times. Ideally, this should only happen once, but Python's unittest doesn't (currently) provide easy hooks to do TestCase-wide setup. So - a couple of options that are worth looking at: * Reduce the amount of parsing that is needed. Keep the pre-parsed results of loading fixtures in memory, so that fixtures don't need to be repeatedly parsed. This is logged as #9449. * Reduce the amount of loading/unloading that is needed. One way to do this is to look at integrating support for unittest2 [1]. This has been logged as #12991. In addition to adding lots of nifty features like test skipping and better assertion diffs, unittest2 adds support for TestCase-wide (and module wide) setups. [1] http://www.voidspace.org.uk/python/articles/unittest2.shtml > Without knowing what gives the current test suite its performance > characteristics aren't we all just guessing at how a rewrite of it > will compare? > > Perhaps a good test would be to take one test module and convert it > from doc tests to unit tests and profile the before and after versions > to see how they compare. While the micro-scale may not ultimately > compare to the macro, it would at least be a starting benchmark. Also a good idea. Some quality time with the profiler would almost certainly reveal extra room for optimization. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
