On 16 February 2014 09:20, Ned Deily <n...@acm.org> wrote: > In article > <1392492250.26338.83831085.39a5e...@webmail.messagingengine.com>, > Benjamin Peterson <benja...@python.org> wrote: >> On Sat, Feb 15, 2014, at 10:12 AM, Serhiy Storchaka wrote: >> > Although Raymond approved a patch for test_bigmem [2], his expressed the >> > insistent recommendation not to do this. So I stop committing new >> > reviewed patches. Terry recommended to discuss this in Python-Dev. What >> > are your thoughts? >> I tend to agree with Raymond. I think such changes are very welcome when >> the module or tests are otherwise being changed, but on their on >> constitute unnecessary churn. > > +1 > > Integrity of the test suite and minimizing code churn top any benefits > of more specific messages on failures. The expectation is that most > tests will never fail so their changed messages will never be seen. For > the rare cases when a test does fail, quite often the test was written > in a way that will require examination of the code to understand exactly > what the test case was intending to test and why it failed. Having a > more specific exception message wouldn't help for many tests without > further modifications; the key point is to know that the test failed.
Right, there are a few key problems with large scale style changes to the test suite: 1. The worst case scenario is where we subtly change a test so that it is no longer testing what it is supposed to be testing, allowing the future introduction of an undetected regression. This isn't particularly *likely*, but a serious problem if it happens. 2. If there are pending patches for that module that include new tests, then the style change may cause the patches to no longer apply cleanly, require rework of bug fix and new feature patches to accommodate the style change. 3. Merges between branches may become more complicated (for reasons similar to 2), unless the style change is also applied to the maintenance branches (which is problematic due to 1). The practical benefits of this kind of change in the test suite are also highly dubious, because they *only help if the test fails at some point in the future*. At that point, whoever caused the test to fail will switch into debugging mode, and a couple of relevant points apply: * the cause of the problem may be immediately obvious anyway, since it's likely in whatever code they just changed * if they need more information, then they can refactor the failing test to use richer (and perhaps additional) assertions as part of the debugging exercise Now, debugging a failing test isn't a skill most programming courses teach, so it may be worth our while to actually provide some pointers on doing so effectively in the devguide (including a note about checking that the failing test is using rich assertions and potentially updating it if it isn't), but the general policy against large scale changes to take advantage of new features still applies to the test suite just as much as it does to the rest of the standard library. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
