On Thu, Jun 26, 2008 at 7:13 AM, Justin Mazzola Paluska <[EMAIL PROTECTED]> wrote: > Hi, > ... > The idea behind the patch is that it's sometimes useful to disable > tests without removing them from the TestCase. For example, a > co-worker and I have a module with a few tests that will fail for the > forseeable future because we haven't had a chance to implement the > features the tests are testing. The tracebacks for these "not > implemented yet" tests obscure real failures in other tests. ... > The patch in issue3202 implements the decorator @disabled and a few > modifications to the classes in the unittest module. > > What does the Python hivemind think? I'm also open to enhancing it if > the list has any ideas.
I think it's really worth looking at the approach that bzrlib's tests take here (see bzrlib.tests.ExtendedTestResult and the out-of-date-but-helpful http://bazaar-vcs.org/BzrExtendTestSuite) Instead of disabling the tests, their practice is to run the tests but have them raise KnownFailure. When they write a test that they know will fail, they have it raise this exception. The extended TestResult object catches this exception in addFailure and reports it appropriately.[1] So, when there's a test that is expected to fail, they do something like this: def test_foo(self): x = get_some_value() self.expectFailure('get_some_value is bogus', self.assertEqual, x, 42) Using KnownFailure is better than disabling because you'll be able to tell if the test suddenly and mysteriously passes. I can see that disabling still has some value, but I'd rather have KnownFailure first. jml [1] They use a similar strategy for skipping tests based on platform. (You can't test Bazaar's symlink support on Windows, for example). _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
