On Wed, Jan 14, 2009 at 7:38 AM, Karen Tracey <kmtra...@gmail.com> wrote: > On Tue, Jan 13, 2009 at 12:06 PM, Jacob Kaplan-Moss > <jacob.kaplanm...@gmail.com> wrote: >> >> On Tue, Jan 13, 2009 at 10:00 AM, Karen Tracey <kmtra...@gmail.com> wrote: >> > OK, so that sounds like one vote for leaving things as they are for >> > doctests, that is with no rolled-back transaction cleaning up after >> > them. >> >> Yeah, I agree. doctests are the 80% testing tool; if you need more >> control, that's what formal test cases are for. >> >> > Of the three #2 sounds most attractive to me, but I have no idea if it >> > is >> > feasible. >> >> I also agree, and I *think* it ought to be possible by modifying the >> way the test runner is constructed during discovery. Failing that, I >> think keeping TestCase as it is in 1.0 and introducing a new >> FasterTestCase would be the better approach -- that way people have to >> opt-in to the new behavior, and can decide if the caveats are worth >> it. >> > > OK, I looked into this idea a little more. It's not too hard in > django.test.simple.run_tests to take the full suite once it has been > discovered and re-order it so that all of the unit tests that are using > rollback are put first. Then they are guaranteed a clean database since > they run first and they clean up after themselves. > > This can, however, still run into some problems. For example the > file_uploads test has trouble with this approach since it has two component > tests, one a django.test.TestCase and one a unittest.TestCase. Ordinarily > the unittest.TestCase DirectoryCreationTests runs first. Pulling the > django.test.TestCase FileUploadTests out and running it first results in > DirectoryCreationTests failing because it tries to create as a file a > directory that is created and not cleaned up by FileUploadTests. > > This is easily fixed but highlights that reordering tests from the way they > have been running to date may result in unexpected errors. This, like > assumptions about primary keys being assigned starting with one suddenly no > longer being true, may make for some nasty surprises for people with their > own set of tests if we just change django.test.TestCase to use the rollback > approach. It may be safer to force people to switch to using a new kind of > test case if they want to see performance improvements for their tests. But > then, of course, they don't just get the nice performance benefit for free. > Opinions?
Each test should be a unit, and should be able to be invoked independently. If one test is depending on another test being executed first, that's a problem with the test case that can be provoked right now by running $ manage test app.OrderDependentTestCase so the problem with our test suite needs to be fixed, regardless. As for making the whole thing opt-in, I'm of two minds. Introducing potential test failures (however legitimate) as part of an upgrade to the test system is something I'd rather avoid doing. On the other hand, this will only affect people that have pre-existing problems with their test suite (just problems they don't know about), and the performance improvement is a huge boost that it seems a pity to relegate it to sitting in the corner until it gets noticed. I suppose we can manage this with a release note highlighting that v1.1 will highlight any order dependent tests. Affected users can replace TestCase with TransactionTestCase to make the problem go away, or they can fix their tests. Russ %-) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
