Re: [Python-Dev] Test the test suite?
On 29 Aug 2013 02:34, "Serhiy Storchaka" wrote: > > 28.08.13 14:37, Victor Stinner написав(ла): > >> No, my question is: how can we detect that a test is never run? Do we >> need test covertage on the test suite? Or inject faults in the code to >> test the test suite? Any other idea? > > > Currently a lot of tests are skipped silently. See issue18702 [1]. Perhaps we need a tool which collects skipped and runned tests, compare these sets with sets from a previous run on the same buildbot and reports if they are different. > > [1] http://bugs.python.org/issue18702 Figuring out a way to collect and merge coverage data would likely be more useful, since that could be applied to the standard library as well. Ned Batchelder's coverage.py supports aggregating data from multiple runs. Cheers, Nick. > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test the test suite?
28.08.13 14:37, Victor Stinner написав(ла): No, my question is: how can we detect that a test is never run? Do we need test covertage on the test suite? Or inject faults in the code to test the test suite? Any other idea? Currently a lot of tests are skipped silently. See issue18702 [1]. Perhaps we need a tool which collects skipped and runned tests, compare these sets with sets from a previous run on the same buildbot and reports if they are different. [1] http://bugs.python.org/issue18702 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test the test suite?
It happens that few tests are also never run because of name conflicts. See issue 16056. Xavier On Wed, Aug 28, 2013 at 1:37 PM, Victor Stinner wrote: > Hi, > > I just noticed that tests using @requires_freebsd_version and > @requires_linux_version decorator from test.support are never run > since this commit (almost 2 years ago): > > changeset: 72618:3b1859f80e6d > user:Charles-François Natali > date:Mon Oct 03 19:40:37 2011 +0200 > files: Lib/test/support.py > description: > Introduce support.requires_freebsd_version decorator. > > ... > > raise unittest.SkipTest( > -"Linux kernel %s or higher required, not %s" > -% (min_version_txt, version_txt)) > -return func(*args, **kw) > -wrapper.min_version = min_version > +"%s version %s or higher required, not %s" > +% (sysname, min_version_txt, version_txt)) > > > I don't want to blame Charles-François, nobody saw the issue during 2 years! > > No, my question is: how can we detect that a test is never run? Do we > need test covertage on the test suite? Or inject faults in the code to > test the test suite? Any other idea? > > I fixed the decorators in Python 3.3 (84debb4abd50) and 3.4 (f98fd5712b0e). > > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/xdegaye%40gmail.com -- Xavier Les Chemins de Lokoti: http://lokoti.alwaysdata.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Test the test suite?
Hi, I just noticed that tests using @requires_freebsd_version and @requires_linux_version decorator from test.support are never run since this commit (almost 2 years ago): changeset: 72618:3b1859f80e6d user:Charles-François Natali date:Mon Oct 03 19:40:37 2011 +0200 files: Lib/test/support.py description: Introduce support.requires_freebsd_version decorator. ... raise unittest.SkipTest( -"Linux kernel %s or higher required, not %s" -% (min_version_txt, version_txt)) -return func(*args, **kw) -wrapper.min_version = min_version +"%s version %s or higher required, not %s" +% (sysname, min_version_txt, version_txt)) I don't want to blame Charles-François, nobody saw the issue during 2 years! No, my question is: how can we detect that a test is never run? Do we need test covertage on the test suite? Or inject faults in the code to test the test suite? Any other idea? I fixed the decorators in Python 3.3 (84debb4abd50) and 3.4 (f98fd5712b0e). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
