On Wed, 06 Apr 2011 18:05:57 -0400, Tres Seaver <tsea...@palladion.com> wrote: > On 04/06/2011 04:37 PM, Antoine Pitrou wrote: > > On Wed, 6 Apr 2011 13:22:09 -0700 Brett Cannon <br...@python.org> wrote: > >> How about the test suite needs to have 100% test coverage (or as close as > >> possible) on the pure Python version? > > > > Let's say "as good coverage as the C code has", instead ;) > > The point is to require that the *Python* version be the "reference > implementation", which means that the tests should be fully covering it > (especially for any non-grandfathered module).
There are two slightly different requirements covered by these two suggested rules. The Python one says "any test the Python package passes the C version should also pass, and let's make sure we test all of the Python code". The C one says "any test that the C code passes the Python code should also pass". These are *almost* the same rule, but not quite. Brett's point in asking for 100% coverage of the Python code is to make sure the C implementation covers the same ground as the Python code. Antoine's point in asking that the Python tests be at least as good as the C tests is to make sure that the Python code covers the same ground as the C code. The former is most important for modules that are getting new accelerator code, the latter for existing modules that already have accelerators or are newly acquiring Python versions. The PEP actually already contains the combined rule: both the C and the Python version must pass the *same* test suite (unless there are virtual machine issues that simply can't be worked around). I think the thing that we are talking about adding to the PEP is that there should be no untested features in *either* the Python or the C version, insofar as we can make that happen (that is, we are testing also that the feature sets are the same). And that passing that comprehensive test suite is the definition of compliance with the PEP, not abstract arguments about semantics. (We can argue about the semantics when we discuss individual tests :) 100% branch coverage as measured by coverage.py is one good place to start for building such a comprehensive test suite. Existing tests for C versions getting (or newly testing) Python code is another. Bug reports from alternate VMs will presumably fill out the remainder. -- R. David Murray http://www.bitdance.com PS: testing that Python code handles subclasses and duck typing is by no means wasted effort; I've some bugs in the email package using such tests, and it is pure Python. _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
