On Sat, Apr 10, 2010 at 2:21 AM, Russell Keith-Magee <[email protected]> wrote: > On Sat, Apr 10, 2010 at 8:29 AM, Gabriel Hurley <[email protected]> wrote: >> Maybe it's an overly simplistic question, but: what makes the tests >> slow currently? It's not simply the volume of them. It's more than >> possible for Python to race through hundreds of tests per second under >> the right conditions. >> >> Some of Django's test modules obviously zip along, but others seem to >> take a rather long time to complete individual tests. Has anyone >> identified which tests are the slowest and what is making them so >> slow? Is it the length of the tests? The parsing? The fixtures that >> are loaded? The data generated in the test itself? > > Yes. :-) > > There are multiple factors at play here. The first is obvious if you > run the test suite right now using different backends: SQLite is much > faster than Postgres, which is much faster than MySQL/MyISAM. This is > because SQLite uses an in-memory store, so it isn't disk bound; > Postgres is disk bound, but is able to use transactions to optimize > test setup and teardown; MySQL is also disk bound, but doesn't support > transactions, so there are a lot of "CREATE DATABASE; do one thing; > DESTROY DATABASE" calls in the lifepan of the test suite. > > The move to using transaction based tests in Django 1.1 sped things up > a lot (for Postgres and SQLite, anyway); but there is still room to > improve. The biggest single problem that I am aware of is that even > though tests are composed in TestCases that have common setup/teardown > routines, each test is handled separately. As a result, if you have a > test case with 10 tests and a complex fixture setup, the fixtures are > read from disk 10 times, loaded into the database 10 times, and rolled > back out of the database 10 times. Ideally, this should only happen > once, but Python's unittest doesn't (currently) provide easy hooks to > do TestCase-wide setup. >
Spoke too late ;) Python 2.7 SVN now has setUpClass and tearDownClass. > So - a couple of options that are worth looking at: > > * Reduce the amount of parsing that is needed. Keep the pre-parsed > results of loading fixtures in memory, so that fixtures don't need to > be repeatedly parsed. This is logged as #9449. > > * Reduce the amount of loading/unloading that is needed. One way to > do this is to look at integrating support for unittest2 [1]. This has > been logged as #12991. In addition to adding lots of nifty features > like test skipping and better assertion diffs, unittest2 adds support > for TestCase-wide (and module wide) setups. > > [1] http://www.voidspace.org.uk/python/articles/unittest2.shtml > >> Without knowing what gives the current test suite its performance >> characteristics aren't we all just guessing at how a rewrite of it >> will compare? >> >> Perhaps a good test would be to take one test module and convert it >> from doc tests to unit tests and profile the before and after versions >> to see how they compare. While the micro-scale may not ultimately >> compare to the macro, it would at least be a starting benchmark. > > Also a good idea. Some quality time with the profiler would almost > certainly reveal extra room for optimization. > > Yours, > Russ Magee %-) > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
