On Tue, Jan 13, 2009 at 8:29 PM, Russell Keith-Magee <freakboy3...@gmail.com > wrote:
> > On Wed, Jan 14, 2009 at 7:38 AM, Karen Tracey <kmtra...@gmail.com> wrote: > > On Tue, Jan 13, 2009 at 12:06 PM, Jacob Kaplan-Moss > > <jacob.kaplanm...@gmail.com> wrote: > >> > >> On Tue, Jan 13, 2009 at 10:00 AM, Karen Tracey <kmtra...@gmail.com> > wrote: > >> > OK, so that sounds like one vote for leaving things as they are for > >> > doctests, that is with no rolled-back transaction cleaning up after > >> > them. > >> > >> Yeah, I agree. doctests are the 80% testing tool; if you need more > >> control, that's what formal test cases are for. > >> > >> > Of the three #2 sounds most attractive to me, but I have no idea if it > >> > is > >> > feasible. > >> > >> I also agree, and I *think* it ought to be possible by modifying the > >> way the test runner is constructed during discovery. Failing that, I > >> think keeping TestCase as it is in 1.0 and introducing a new > >> FasterTestCase would be the better approach -- that way people have to > >> opt-in to the new behavior, and can decide if the caveats are worth > >> it. > >> > > > > OK, I looked into this idea a little more. It's not too hard in > > django.test.simple.run_tests to take the full suite once it has been > > discovered and re-order it so that all of the unit tests that are using > > rollback are put first. Then they are guaranteed a clean database since > > they run first and they clean up after themselves. > > > > This can, however, still run into some problems. For example the > > file_uploads test has trouble with this approach since it has two > component > > tests, one a django.test.TestCase and one a unittest.TestCase. > Ordinarily > > the unittest.TestCase DirectoryCreationTests runs first. Pulling the > > django.test.TestCase FileUploadTests out and running it first results in > > DirectoryCreationTests failing because it tries to create as a file a > > directory that is created and not cleaned up by FileUploadTests. > > > > This is easily fixed but highlights that reordering tests from the way > they > > have been running to date may result in unexpected errors. This, like > > assumptions about primary keys being assigned starting with one suddenly > no > > longer being true, may make for some nasty surprises for people with > their > > own set of tests if we just change django.test.TestCase to use the > rollback > > approach. It may be safer to force people to switch to using a new kind > of > > test case if they want to see performance improvements for their tests. > But > > then, of course, they don't just get the nice performance benefit for > free. > > Opinions? > > Each test should be a unit, and should be able to be invoked > independently. If one test is depending on another test being executed > first, that's a problem with the test case that can be provoked right > now by running > > $ manage test app.OrderDependentTestCase > > so the problem with our test suite needs to be fixed, regardless. > > As for making the whole thing opt-in, I'm of two minds. Introducing > potential test failures (however legitimate) as part of an upgrade to > the test system is something I'd rather avoid doing. On the other > hand, this will only affect people that have pre-existing problems > with their test suite (just problems they don't know about), and the > performance improvement is a huge boost that it seems a pity to > relegate it to sitting in the corner until it gets noticed. > > I suppose we can manage this with a release note highlighting that > v1.1 will highlight any order dependent tests. Affected users can > replace TestCase with TransactionTestCase to make the problem go away, > or they can fix their tests. > > Russ %-) > > > > Personally I think anyone with broken tests(and that's wat they are) should just have them break, though testing is a stable API I personally don't think that covers stuff that is broken(as Russ demonstrated it's broken by running the test by itself). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire "The people's good is the highest law."--Cicero --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
