jayvdb added a comment.
On https://gerrit.wikimedia.org/r/#/c/278456/@xqt said > expectedFailure does not what is sounds. It also passes for unexpected failures and you cannot decide wether > the failure is expected or due to an unknown reason and the test is finally worthless. `@expectedFailure` should only be used if the failure is 100% expected, and typically the failure is expected on the last line, and the test method is written so that other tests will fail if some earlier part of the test sequence has a problem. The test will become an "unexpected success" if the test passes. (And we can configure "unexpected success" to be a treated as an error, but that only helps if someone is regularly looking for & fixing build breakages.) > Every test should be expressed in a form that it may fail in expected manner e.g. checking for wrong > results or for raised exceptions etc. If the test is written so that it is the last line that fails (see above) the following are functionally equivalent @expectedFailure def test_foo(self): 1 / 0 def test_foo2(self): with self.assertRaises(Exception): 1 / 0 Except that the former has more semantic information, and code checkers explicitly fail `self.assertRaises(Exception)` as being too broad. Of course we can work around that by using `self.assertRaises(ZeroDivisionError):` , but that means our tests are then explicitly linked to the underlying implementation of why it fails, which often isnt desirable because we dont have any control over that. expectedFailure is used to prove something doesnt work, irrespective of why, and often it exists to help explain a `# TODO` item by showing what 'should' work. TASK DETAIL https://phabricator.wikimedia.org/T129368 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: jayvdb Cc: Mpaa, gerritbot, valhallasw, jayvdb, Aklapper, Xqt, pywikibot-bugs-list _______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
