On Feb 15, 2010, at 3:50 PM, Michael Foord wrote: > On 15/02/2010 20:27, Glyph Lefkowitz wrote: >> >> >> On Feb 13, 2010, at 12:46 PM, Guido van Rossum wrote: >> >>> On Fri, Feb 12, 2010 at 8:01 PM, Glyph Lefkowitz >>> <gl...@twistedmatrix.com> wrote: >>>> I find setUpClass more hostile to *other* kinds of testing, because this >>>> convenience for simple integration tests makes more involved, >>>> performance-intensive integration tests harder to write and manage. >>> >>> That sounds odd, as if the presence of this convenience would prohibit >>> you from also implement other features. >> >> And I am pretty sure this is not just my over-reaction; Michael still >> appears to be wrestling with the problems I'm describing. > And I appreciate your input.
Thanks :). >> In a recent message he was talking about either breaking compatibility with >> TestSuite implementations that override run(), or test-reordering - both of >> which I consider important, core features of the unittest module. > > Well, by "breaking compatibility with custom TestSuite implementations that > override run" I mean that is one possible place to put the functionality. > Code that does override it will *not* stop working, it just won't support the > new features. Ah, I see. This doesn't sound *too* bad, but I'd personally prefer it if the distinction were a bit more clearly drawn. I'd like frameworks to be able to implement extension functionality without having to first stub out functionality. In other words, if I want a test suite without setUpClass, I'd prefer to avoid having an abstraction inversion. Practically speaking this could be implemented by having a very spare, basic TestSuite base class and ClassSuite/ModuleSuite subclasses which implement the setUpXXX functionality. > If we chose this implementation strategy there would be no compatibility > issues for existing tests / frameworks that don't use the new features. That's very good to hear. > If tests do want to use the new features then the framework authors will need > to ensure they are compatible with them. This seems like a reasonable > trade-off to me. We can ensure that it is easy to write custom TestSuite > objects that work with earlier versions of unittest but are also compatible > with setUpClass in 2.7 (and document the recipe - although I expect it will > just mean that TestSuite.run should call a single method if it exists). This is something that I hope Jonathan Lange or Robert Collins will chime in to comment on: expanding the protocol between suite and test is an area which is fraught with peril, but it seems like it's something that test framework authors always want to do. (Personally, *I* really want to do it because I want to be able to run things asynchronously, so the semantics of 'run()' need to change pretty dramatically to support that...) It might be good to eventually develop a general mechanism for this, rather than building up an ad-hoc list of test-feature compatibility recipes which involve a list of if hasattr(...): foo(); checks in every suite implementation. > Perhaps a better idea might be to also add startTest and stopTest methods to > TestSuite so that frameworks can build in features like timing tests (etc) > without having to override run itself. This is already possible in the > TestResult of course, which is a more common extensibility point in *my* > experience. I think timing and monitoring tests can mostly be done in the TestResult class; those were bad examples. There's stuff like synthesizing arguments for test methods, or deciding to repeat a potentially flaky test method before reporting a failure, which are not possible to do from the result. I'm not sure that startTest and stopTest hooks help with those features, the ones which really need suites; it would seem it mostly gives you a hook to do stuff that could already be done in TestResult anyway.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
