Hello community,
here is the log from the commit of package python-zope.testrunner for
openSUSE:Factory checked in at 2019-11-22 10:20:12
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-zope.testrunner (Old)
and /work/SRC/openSUSE:Factory/.python-zope.testrunner.new.26869 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-zope.testrunner"
Fri Nov 22 10:20:12 2019 rev:12 rq:745976 version:5.1
Changes:
--------
---
/work/SRC/openSUSE:Factory/python-zope.testrunner/python-zope.testrunner.changes
2019-04-10 23:10:55.303955212 +0200
+++
/work/SRC/openSUSE:Factory/.python-zope.testrunner.new.26869/python-zope.testrunner.changes
2019-11-22 10:20:13.817348467 +0100
@@ -1,0 +2,9 @@
+Wed Oct 30 12:13:54 UTC 2019 - Tomáš Chvátal <[email protected]>
+
+- Update to 5.1:
+ * Recover more gracefully when layer setUp or tearDown fails, producing
useful subunit output.
+ * Prevent a spurious warning from the --require-unique option if the
--module option was not used.
+ * Add optional buffering of standard output and standard error during tests,
requested via the --buffer option or enabled by default for subunit.
+ * Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.
+
+-------------------------------------------------------------------
Old:
----
zope.testrunner-5.0.tar.gz
New:
----
zope.testrunner-5.1.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-zope.testrunner.spec ++++++
--- /var/tmp/diff_new_pack.8wPrkq/_old 2019-11-22 10:20:14.269348350 +0100
+++ /var/tmp/diff_new_pack.8wPrkq/_new 2019-11-22 10:20:14.273348349 +0100
@@ -26,29 +26,27 @@
%bcond_with test
%endif
Name: python-zope.testrunner
-Version: 5.0
+Version: 5.1
Release: 0
Summary: Zope testrunner script
License: ZPL-2.1
-Group: Development/Languages/Python
-URL: https://pypi.python.org/pypi/zope.testrunner
+URL: https://github.com/zopefoundation/zope.testrunner
Source:
https://files.pythonhosted.org/packages/source/z/zope.testrunner/zope.testrunner-%{version}.tar.gz
BuildRequires: %{python_module setuptools}
-# runtime requirements:
BuildRequires: %{python_module six}
BuildRequires: %{python_module zope.exceptions}
BuildRequires: %{python_module zope.interface}
-# Test requirements:
-%if %{with test}
-BuildRequires: %{python_module zope.testing}
-BuildRequires: %{python_module zope.testrunner = %{version}}
-%endif
BuildRequires: fdupes
BuildRequires: python-rpm-macros
Requires: python-six
Requires: python-zope.exceptions
Requires: python-zope.interface
BuildArch: noarch
+%if %{with test}
+BuildRequires: %{python_module zope.testing}
+BuildRequires: %{python_module zope.testrunner = %{version}}
+BuildRequires: %{pythons}
+%endif
%python_subpackages
%description
@@ -82,7 +80,7 @@
%python_uninstall_alternative zope-testrunner
%files %{python_files}
-%license LICENSE.rst
+%license LICENSE.md
%doc README.rst
%python_alternative %{_bindir}/zope-testrunner
%{python_sitelib}/*
++++++ zope.testrunner-5.0.tar.gz -> zope.testrunner-5.1.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/.gitignore
new/zope.testrunner-5.1/.gitignore
--- old/zope.testrunner-5.0/.gitignore 2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/.gitignore 1970-01-01 01:00:00.000000000 +0100
@@ -1,15 +0,0 @@
-*.pyc
-__pycache__
-
-.installed.cfg
-bin/
-eggs/
-develop-eggs/
-*.egg-info/
-tags
-.tox
-parts/
-.coverage
-.coverage.*
-htmlcov/
-docs/_build/
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/CHANGES.rst
new/zope.testrunner-5.1/CHANGES.rst
--- old/zope.testrunner-5.0/CHANGES.rst 2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/CHANGES.rst 2019-10-19 11:47:21.000000000 +0200
@@ -2,6 +2,21 @@
zope.testrunner Changelog
===========================
+5.1 (2019-10-19)
+================
+
+- Recover more gracefully when layer setUp or tearDown fails, producing
+ useful subunit output.
+
+- Prevent a spurious warning from the ``--require-unique`` option if the
+ ``--module`` option was not used.
+
+- Add optional buffering of standard output and standard error during tests,
+ requested via the ``--buffer`` option or enabled by default for subunit.
+
+- Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.
+
+
5.0 (2019-03-19)
================
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/LICENSE.md
new/zope.testrunner-5.1/LICENSE.md
--- old/zope.testrunner-5.0/LICENSE.md 1970-01-01 01:00:00.000000000 +0100
+++ new/zope.testrunner-5.1/LICENSE.md 2019-10-19 11:47:21.000000000 +0200
@@ -0,0 +1,44 @@
+Zope Public License (ZPL) Version 2.1
+
+A copyright notice accompanies this license document that identifies the
+copyright holders.
+
+This license has been certified as open source. It has also been designated as
+GPL compatible by the Free Software Foundation (FSF).
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions in source code must retain the accompanying copyright
+notice, this list of conditions, and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the accompanying copyright
+notice, this list of conditions, and the following disclaimer in the
+documentation and/or other materials provided with the distribution.
+
+3. Names of the copyright holders must not be used to endorse or promote
+products derived from this software without prior written permission from the
+copyright holders.
+
+4. The right to distribute this software or to use it for any purpose does not
+give you the right to use Servicemarks (sm) or Trademarks (tm) of the
+copyright
+holders. Use of them is covered by separate agreement with the copyright
+holders.
+
+5. If any files are modified, you must cause the modified files to carry
+prominent notices stating that you changed the files and the date of any
+change.
+
+Disclaimer
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESSED
+OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
+EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/LICENSE.rst
new/zope.testrunner-5.1/LICENSE.rst
--- old/zope.testrunner-5.0/LICENSE.rst 2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/LICENSE.rst 1970-01-01 01:00:00.000000000 +0100
@@ -1,44 +0,0 @@
-Zope Public License (ZPL) Version 2.1
-
-A copyright notice accompanies this license document that identifies the
-copyright holders.
-
-This license has been certified as open source. It has also been designated as
-GPL compatible by the Free Software Foundation (FSF).
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-
-1. Redistributions in source code must retain the accompanying copyright
-notice, this list of conditions, and the following disclaimer.
-
-2. Redistributions in binary form must reproduce the accompanying copyright
-notice, this list of conditions, and the following disclaimer in the
-documentation and/or other materials provided with the distribution.
-
-3. Names of the copyright holders must not be used to endorse or promote
-products derived from this software without prior written permission from the
-copyright holders.
-
-4. The right to distribute this software or to use it for any purpose does not
-give you the right to use Servicemarks (sm) or Trademarks (tm) of the
-copyright
-holders. Use of them is covered by separate agreement with the copyright
-holders.
-
-5. If any files are modified, you must cause the modified files to carry
-prominent notices stating that you changed the files and the date of any
-change.
-
-Disclaimer
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY EXPRESSED
-OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
-OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
-EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT,
-INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
-EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/MANIFEST.in
new/zope.testrunner-5.1/MANIFEST.in
--- old/zope.testrunner-5.0/MANIFEST.in 2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/MANIFEST.in 2019-10-19 11:47:21.000000000 +0200
@@ -11,3 +11,6 @@
recursive-include docs *.rst
recursive-include docs *.py
+
+# added by check_manifest.py
+include *.md
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/PKG-INFO
new/zope.testrunner-5.1/PKG-INFO
--- old/zope.testrunner-5.0/PKG-INFO 2019-03-19 08:17:20.000000000 +0100
+++ new/zope.testrunner-5.1/PKG-INFO 2019-10-19 11:47:21.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: zope.testrunner
-Version: 5.0
+Version: 5.1
Summary: Zope testrunner script.
Home-page: https://github.com/zopefoundation/zope.testrunner
Author: Zope Foundation and Contributors
@@ -186,6 +186,21 @@
zope.testrunner Changelog
===========================
+ 5.1 (2019-10-19)
+ ================
+
+ - Recover more gracefully when layer setUp or tearDown fails, producing
+ useful subunit output.
+
+ - Prevent a spurious warning from the ``--require-unique`` option if
the
+ ``--module`` option was not used.
+
+ - Add optional buffering of standard output and standard error during
tests,
+ requested via the ``--buffer`` option or enabled by default for
subunit.
+
+ - Fix incorrect failure counts in per-layer summary output, broken in
4.0.1.
+
+
5.0 (2019-03-19)
================
@@ -599,5 +614,5 @@
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*
Provides-Extra: test
-Provides-Extra: docs
Provides-Extra: subunit
+Provides-Extra: docs
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/appveyor.yml
new/zope.testrunner-5.1/appveyor.yml
--- old/zope.testrunner-5.0/appveyor.yml 2019-03-19 08:17:18.000000000
+0100
+++ new/zope.testrunner-5.1/appveyor.yml 2019-10-19 11:47:21.000000000
+0200
@@ -5,8 +5,6 @@
matrix:
- python: 27
- python: 27-x64
- - python: 34
- - python: 34-x64
- python: 35
- python: 35-x64
- python: 36
@@ -15,8 +13,6 @@
- python: 37-x64
- { python: 27, SUBUNIT: 1 }
- { python: 27-x64, SUBUNIT: 1 }
- - { python: 34, SUBUNIT: 1 }
- - { python: 34-x64, SUBUNIT: 1 }
- { python: 35, SUBUNIT: 1 }
- { python: 35-x64, SUBUNIT: 1 }
- { python: 36, SUBUNIT: 1 }
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/setup.py
new/zope.testrunner-5.1/setup.py
--- old/zope.testrunner-5.0/setup.py 2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/setup.py 2019-10-19 11:47:21.000000000 +0200
@@ -21,7 +21,7 @@
from setuptools import setup
from setuptools.command.test import test
-version = '5.0'
+version = '5.1'
INSTALL_REQUIRES = [
'setuptools',
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/src/zope/testrunner/formatter.py
new/zope.testrunner-5.1/src/zope/testrunner/formatter.py
--- old/zope.testrunner-5.0/src/zope/testrunner/formatter.py 2019-03-19
08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/formatter.py 2019-10-19
11:47:21.000000000 +0200
@@ -15,6 +15,10 @@
"""
from __future__ import print_function
+try:
+ from collections.abc import MutableMapping
+except ImportError:
+ from collections import MutableMapping
from contextlib import contextmanager
import doctest
import os
@@ -323,7 +327,7 @@
sys.stdout.write(s)
self.test_width += len(s) + 1
- def test_error(self, test, seconds, exc_info):
+ def test_error(self, test, seconds, exc_info, stdout=None, stderr=None):
"""Report that an error occurred while running a test.
Should be called right after start_test().
@@ -334,9 +338,10 @@
print(" (%s)" % self.format_seconds_short(seconds))
print()
self.print_traceback("Error in test %s" % test, exc_info)
+ self.print_std_streams(stdout, stderr)
self.test_width = self.last_width = 0
- def test_failure(self, test, seconds, exc_info):
+ def test_failure(self, test, seconds, exc_info, stdout=None, stderr=None):
"""Report that a test failed.
Should be called right after start_test().
@@ -347,6 +352,7 @@
print(" (%s)" % self.format_seconds_short(seconds))
print()
self.print_traceback("Failure in test %s" % test, exc_info)
+ self.print_std_streams(stdout, stderr)
self.test_width = self.last_width = 0
def print_traceback(self, msg, exc_info):
@@ -355,6 +361,21 @@
print(msg)
print(self.format_traceback(exc_info))
+ def print_std_streams(self, stdout, stderr):
+ """Emit contents of buffered standard streams."""
+ if stdout:
+ sys.stdout.write("Stdout:\n")
+ sys.stdout.write(stdout)
+ if not stdout.endswith("\n"):
+ sys.stdout.write("\n")
+ sys.stdout.write("\n")
+ if stderr:
+ sys.stderr.write("Stderr:\n")
+ sys.stderr.write(stderr)
+ if not stderr.endswith("\n"):
+ sys.stderr.write("\n")
+ sys.stderr.write("\n")
+
def format_traceback(self, exc_info):
"""Format the traceback."""
v = exc_info[1]
@@ -792,6 +813,37 @@
self.decorated.status(**kwargs)
+class _SortedDict(MutableMapping, object):
+ """A dict that always returns items in sorted order.
+
+ This differs from collections.OrderedDict in that it returns items in
+ *sorted* order, not in insertion order.
+
+ We use this as a workaround for the fact that
+ testtools.ExtendedToStreamDecorator doesn't sort the details dict when
+ encoding it, which makes it difficult to write stable doctests for
+ subunit v2 output.
+ """
+
+ def __init__(self, items):
+ self._dict = dict(items)
+
+ def __getitem__(self, key):
+ return self._dict[key]
+
+ def __setitem__(self, key, value):
+ self._dict[key] = value
+
+ def __delitem__(self, key):
+ del self._dict[key]
+
+ def __iter__(self):
+ return iter(sorted(self._dict))
+
+ def __len__(self):
+ return len(self._dict)
+
+
class SubunitOutputFormatter(object):
"""A subunit output formatter.
@@ -885,6 +937,15 @@
self._subunit.addError(test, exc_info)
self._subunit.stopTest(test)
+ def _emit_failure(self, failure_id, tag, exc_info):
+ """Emit an failure to the subunit stream.
+
+ Use this to pass on information about failures that occur outside of
+ tests.
+ """
+ test = FakeTest(failure_id)
+ self._subunit.addFailure(test, exc_info)
+
def _enter_layer(self, layer_name):
"""Tell subunit that we are entering a layer."""
self._subunit.tags(['zope:layer:%s' % (layer_name,)], [])
@@ -1043,20 +1104,20 @@
def refcounts(self, rc, prev):
"""Report a change in reference counts."""
- details = {
+ details = _SortedDict({
'sys-refcounts': text_content(str(rc)),
'changes': text_content(str(rc - prev)),
- }
+ })
# XXX: Emit the details dict as JSON?
self._emit_fake_test(self.TAG_REFCOUNTS, self.TAG_REFCOUNTS, details)
def detailed_refcounts(self, track, rc, prev):
"""Report a change in reference counts, with extra detail."""
- details = {
+ details = _SortedDict({
'sys-refcounts': text_content(str(rc)),
'changes': text_content(str(rc - prev)),
'track': text_content(str(track.delta)),
- }
+ })
self._emit_fake_test(self.TAG_REFCOUNTS, self.TAG_REFCOUNTS, details)
def start_set_up(self, layer_name):
@@ -1089,6 +1150,11 @@
self._subunit.stopTest(test)
self._enter_layer(layer_name)
+ def layer_failure(self, failure_type, exc_info):
+ layer_name, start_time = self._last_layer
+ self._emit_failure(
+ '%s:%s' % (layer_name, failure_type), self.TAG_LAYER, exc_info)
+
def start_tear_down(self, layer_name):
"""Report that we're tearing down a layer.
@@ -1180,13 +1246,26 @@
else:
unicode_tb = traceback
- return {
+ return _SortedDict({
'traceback': Content(
self.TRACEBACK_CONTENT_TYPE,
lambda: [unicode_tb.encode('utf8')]),
- }
+ })
+
+ def _add_std_streams_to_details(self, details, stdout, stderr):
+ """Add buffered standard stream contents to a subunit details dict."""
+ if stdout:
+ if isinstance(stdout, bytes):
+ stdout = stdout.decode('utf-8', 'replace')
+ details['test-stdout'] = Content(
+ self.PLAIN_TEXT, lambda: [stdout.encode('utf-8')])
+ if stderr:
+ if isinstance(stderr, bytes):
+ stderr = stderr.decode('utf-8', 'replace')
+ details['test-stderr'] = Content(
+ self.PLAIN_TEXT, lambda: [stderr.encode('utf-8')])
- def test_error(self, test, seconds, exc_info):
+ def test_error(self, test, seconds, exc_info, stdout=None, stderr=None):
"""Report that an error occurred while running a test.
Should be called right after start_test().
@@ -1195,9 +1274,10 @@
"""
self._emit_timestamp()
details = self._exc_info_to_details(exc_info)
+ self._add_std_streams_to_details(details, stdout, stderr)
self._subunit.addError(test, details=details)
- def test_failure(self, test, seconds, exc_info):
+ def test_failure(self, test, seconds, exc_info, stdout=None, stderr=None):
"""Report that a test failed.
Should be called right after start_test().
@@ -1206,6 +1286,7 @@
"""
self._emit_timestamp()
details = self._exc_info_to_details(exc_info)
+ self._add_std_streams_to_details(details, stdout, stderr)
self._subunit.addFailure(test, details=details)
def stop_test(self, test):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/src/zope/testrunner/options.py
new/zope.testrunner-5.1/src/zope/testrunner/options.py
--- old/zope.testrunner-5.0/src/zope/testrunner/options.py 2019-03-19
08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/options.py 2019-10-19
11:47:21.000000000 +0200
@@ -285,6 +285,16 @@
it will be ignored.
""")
+reporting.add_argument(
+ '--buffer', action="store_true", dest="buffer",
+ help="""\
+Buffer the standard output and standard error streams during each test.
+Output during a passing test is discarded.
+Output during failing or erroring tests is echoed.
+This option is enabled by default if --subunit or --subunit-v2 is in use, to
+avoid corrupting the subunit stream.
+""")
+
######################################################################
# Analysis
@@ -591,6 +601,7 @@
""")
options.fail = True
return options
+ options.buffer = True
if options.subunit and options.subunit_v2:
print("""\
@@ -628,6 +639,7 @@
options.ignore_dir = set(options.ignore_dir)
options.test = options.test or ['.']
+ module_set = bool(options.module)
options.module = options.module or ['.']
options.path = options.path or []
@@ -694,7 +706,7 @@
options.fail = True
return options
- if options.module and options.require_unique_ids:
+ if module_set and options.require_unique_ids:
# We warn if --module and --require-unique are specified at the same
# time, though we don't exit.
print("""\
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/zope.testrunner-5.0/src/zope/testrunner/runner.py
new/zope.testrunner-5.1/src/zope/testrunner/runner.py
--- old/zope.testrunner-5.0/src/zope/testrunner/runner.py 2019-03-19
08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/runner.py 2019-10-19
11:47:21.000000000 +0200
@@ -317,11 +317,22 @@
if setup_layers:
if self.options.resume_layer is None:
self.options.output.info("Tearing down left over layers:")
- tear_down_unneeded(self.options, (), setup_layers, True)
+ tear_down_unneeded(
+ self.options, (), setup_layers, self.errors, optional=True)
self.failed = bool(self.import_errors or self.failures or self.errors)
+def handle_layer_failure(failure_type, output, errors):
+ if hasattr(output, 'layer_failure'):
+ output.layer_failure(failure_type.subunit_label, sys.exc_info())
+ else:
+ f = StringIO()
+ traceback.print_exc(file=f)
+ output.error(f.getvalue())
+ errors.append((failure_type, sys.exc_info()))
+
+
def run_tests(options, tests, name, failures, errors, skipped, import_errors):
repeat = options.repeat or 1
repeat_range = iter(range(repeat))
@@ -389,14 +400,16 @@
t = time.time() - t
output.stop_tests()
failures.extend(result.failures)
+ n_failures = len(result.failures)
if hasattr(result, 'unexpectedSuccesses'):
# Python versions prior to 2.7 do not have the concept of
# unexpectedSuccesses.
failures.extend(result.unexpectedSuccesses)
+ n_failures += len(result.unexpectedSuccesses)
skipped.extend(result.skipped)
errors.extend(result.errors)
output.summary(n_tests=result.testsRun,
- n_failures=len(failures),
+ n_failures=n_failures,
n_errors=len(result.errors) + len(import_errors),
n_seconds=t,
n_skipped=len(result.skipped))
@@ -442,7 +455,7 @@
needed = dict([(l, 1) for l in gathered])
if options.resume_number != 0:
output.info("Running %s tests:" % layer_name)
- tear_down_unneeded(options, needed, setup_layers)
+ tear_down_unneeded(options, needed, setup_layers, errors)
if options.resume_layer is not None:
output.info_suboptimal(" Running in a subprocess.")
@@ -451,11 +464,10 @@
setup_layer(options, layer, setup_layers)
except zope.testrunner.interfaces.EndRun:
raise
+ except MemoryError:
+ raise
except Exception:
- f = StringIO()
- traceback.print_exc(file=f)
- output.error(f.getvalue())
- errors.append((SetUpLayerFailure(layer), sys.exc_info()))
+ handle_layer_failure(SetUpLayerFailure(layer), output, errors)
return 0
else:
return run_tests(options, tests, layer_name, failures, errors, skipped,
@@ -464,6 +476,8 @@
class SetUpLayerFailure(unittest.TestCase):
+ subunit_label = 'setUp'
+
def __init__(self, layer):
super(SetUpLayerFailure, self).__init__()
self.layer = layer
@@ -472,8 +486,22 @@
pass
def __str__(self):
- return "Layer: %s" % (name_from_layer(self.layer))
+ return "Layer: %s.setUp" % (name_from_layer(self.layer))
+
+class TearDownLayerFailure(unittest.TestCase):
+
+ subunit_label = 'tearDown'
+
+ def __init__(self, layer):
+ super(TearDownLayerFailure, self).__init__()
+ self.layer = layer
+
+ def runTest(self):
+ pass
+
+ def __str__(self):
+ return "Layer: %s.tearDown" % (name_from_layer(self.layer))
def spawn_layer_in_subprocess(result, script_parts, options, features,
@@ -748,7 +776,7 @@
return sum(r.num_ran for r in results)
-def tear_down_unneeded(options, needed, setup_layers, optional=False):
+def tear_down_unneeded(options, needed, setup_layers, errors, optional=False):
# Tear down any layers not needed for these tests. The unneeded layers
# might interfere.
unneeded = [l for l in setup_layers if l not in needed]
@@ -766,6 +794,10 @@
output.tear_down_not_supported()
if not optional:
raise CanNotTearDown(l)
+ except MemoryError:
+ raise
+ except Exception:
+ handle_layer_failure(TearDownLayerFailure(l), output, errors)
else:
output.stop_tear_down(time.time() - t)
finally:
@@ -790,6 +822,8 @@
if hasattr(layer, 'setUp'):
try:
layer.setUp()
+ except MemoryError:
+ raise
except Exception:
if options.post_mortem:
if options.resume_layer:
@@ -821,6 +855,10 @@
for test in tests:
count += test.countTestCases()
self.count = count
+ self._stdout_buffer = None
+ self._stderr_buffer = None
+ self._original_stdout = sys.stdout
+ self._original_stderr = sys.stderr
def testSetUp(self):
"""A layer may define a setup method to be called before each
@@ -842,6 +880,31 @@
if hasattr(layer, 'testTearDown'):
layer.testTearDown()
+ def _setUpStdStreams(self):
+ """Set up buffered standard streams, if requested."""
+ if self.options.buffer:
+ if self._stdout_buffer is None:
+ self._stdout_buffer = StringIO()
+ if self._stderr_buffer is None:
+ self._stderr_buffer = StringIO()
+ sys.stdout = self._stdout_buffer
+ sys.stderr = self._stderr_buffer
+
+ def _restoreStdStreams(self):
+ """Restore the buffered standard streams and return any contents."""
+ if self.options.buffer:
+ stdout = sys.stdout.getvalue()
+ stderr = sys.stderr.getvalue()
+ sys.stdout = self._original_stdout
+ sys.stderr = self._original_stderr
+ self._stdout_buffer.seek(0)
+ self._stdout_buffer.truncate(0)
+ self._stderr_buffer.seek(0)
+ self._stderr_buffer.truncate(0)
+ return stdout, stderr
+ else:
+ return None, None
+
def startTest(self, test):
self.testSetUp()
unittest.TestResult.startTest(self, test)
@@ -854,17 +917,23 @@
self._threads = threading.enumerate()
self._start_time = time.time()
+ self._setUpStdStreams()
+
def addSuccess(self, test):
+ self._restoreStdStreams()
t = max(time.time() - self._start_time, 0.0)
self.options.output.test_success(test, t)
def addSkip(self, test, reason):
+ self._restoreStdStreams()
unittest.TestResult.addSkip(self, test, reason)
self.options.output.test_skipped(test, reason)
def addError(self, test, exc_info):
+ stdout, stderr = self._restoreStdStreams()
self.options.output.test_error(test, time.time() - self._start_time,
- exc_info)
+ exc_info,
+ stdout=stdout, stderr=stderr)
unittest.TestResult.addError(self, test, exc_info)
@@ -879,8 +948,10 @@
self.stop()
def addFailure(self, test, exc_info):
+ stdout, stderr = self._restoreStdStreams()
self.options.output.test_failure(test, time.time() - self._start_time,
- exc_info)
+ exc_info,
+ stdout=stdout, stderr=stderr)
unittest.TestResult.addFailure(self, test, exc_info)
@@ -892,15 +963,18 @@
self.stop()
def addExpectedFailure(self, test, exc_info):
+ self._restoreStdStreams()
t = max(time.time() - self._start_time, 0.0)
self.options.output.test_success(test, t)
unittest.TestResult.addExpectedFailure(self, test, exc_info)
def addUnexpectedSuccess(self, test):
+ stdout, stderr = self._restoreStdStreams()
self.options.output.test_error(
test, time.time() - self._start_time,
- (UnexpectedSuccess, UnexpectedSuccess(), None))
+ (UnexpectedSuccess, UnexpectedSuccess(), None),
+ stdout=stdout, stderr=stderr)
unittest.TestResult.addUnexpectedSuccess(self, test)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-debugging-layer-setup.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-debugging-layer-setup.rst
---
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-debugging-layer-setup.rst
2019-03-19 08:17:18.000000000 +0100
+++
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-debugging-layer-setup.rst
2019-10-19 11:47:21.000000000 +0200
@@ -131,7 +131,7 @@
<BLANKLINE>
<BLANKLINE>
Tests with errors:
- Layer: tests2.Layer2
+ Layer: tests2.Layer2.setUp
Total: 1 tests, 0 failures, 1 errors and 0 skipped in 0.210 seconds.
True
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-edge-cases.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-edge-cases.rst
--- old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-edge-cases.rst
2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-edge-cases.rst
2019-10-19 11:47:21.000000000 +0200
@@ -524,3 +524,30 @@
Tearing down left over layers:
Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
False
+
+
+Requiring unique test IDs
+-------------------------
+
+The --require-unique option causes the test runner to require that all test
+IDs be unique. Its behaviour is tested in zope.testrunner.tests.test_find;
+here we check its interaction with other options.
+
+The --require-unique option does not issue any warnings on its own.
+
+ >>> sys.argv = 'test --require-unique'.split()
+ >>> testrunner.run_internal(defaults)
+ ... # doctest: +ELLIPSIS
+ Running zope.testrunner.layer.UnitTests tests:
+ ...
+
+Attempting to use both --module and --require-unique issues a warning.
+
+ >>> sys.argv = 'test --module sample --require-unique'.split()
+ >>> testrunner.run_internal(defaults)
+ ... # doctest: +ELLIPSIS
+ You specified a module along with --require-unique;
+ --require-unique will not try to enforce test ID uniqueness when
+ working with a specific module.
+ Running zope.testrunner.layer.UnitTests tests:
+ ...
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-errors.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-errors.rst
--- old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-errors.rst
2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-errors.rst
2019-10-19 11:47:21.000000000 +0200
@@ -366,6 +366,87 @@
as progress output.
+Capturing output
+----------------
+
+We can ask for output on stdout and stderr to be buffered, and emitted only
+for failing and erroring tests.
+
+ >>> sys.argv = (
+ ... 'test -vv --buffer -ssample2'
+ ... ' --tests-pattern ^stdstreamstest$'.split())
+ >>> orig_stderr = sys.stderr
+ >>> sys.stderr = sys.stdout
+ >>> testrunner.run_internal(defaults) # doctest: +NORMALIZE_WHITESPACE
+ Running tests at level 1
+ Running zope.testrunner.layer.UnitTests tests:
+ Set up zope.testrunner.layer.UnitTests in N.NNN seconds.
+ Running:
+ test_stderr_error (sample2.stdstreamstest.Test)
+ <BLANKLINE>
+ Error in test test_stderr_error (sample2.stdstreamstest.Test)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_error
+ raise Exception("boom")
+ Exception: boom
+ <BLANKLINE>
+ Stderr:
+ stderr output on error
+ <BLANKLINE>
+ <BLANKLINE>
+ test_stderr_failure (sample2.stdstreamstest.Test)
+ <BLANKLINE>
+ Failure in test test_stderr_failure (sample2.stdstreamstest.Test)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ <BLANKLINE>
+ Stderr:
+ stderr output on failure
+ <BLANKLINE>
+ <BLANKLINE>
+ test_stderr_success (sample2.stdstreamstest.Test)
+ test_stdout_error (sample2.stdstreamstest.Test)
+ <BLANKLINE>
+ Error in test test_stdout_error (sample2.stdstreamstest.Test)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_error
+ raise Exception("boom")
+ Exception: boom
+ <BLANKLINE>
+ Stdout:
+ stdout output on error
+ <BLANKLINE>
+ <BLANKLINE>
+ test_stdout_failure (sample2.stdstreamstest.Test)
+ <BLANKLINE>
+ Failure in test test_stdout_failure (sample2.stdstreamstest.Test)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ <BLANKLINE>
+ Stdout:
+ stdout output on failure
+ <BLANKLINE>
+ <BLANKLINE>
+ test_stdout_success (sample2.stdstreamstest.Test)
+ Ran 6 tests with 2 failures, 2 errors and 0 skipped in N.NNN seconds.
+ Tearing down left over layers:
+ Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
+ <BLANKLINE>
+ Tests with errors:
+ test_stderr_error (sample2.stdstreamstest.Test)
+ test_stdout_error (sample2.stdstreamstest.Test)
+ <BLANKLINE>
+ Tests with failures:
+ test_stderr_failure (sample2.stdstreamstest.Test)
+ test_stdout_failure (sample2.stdstreamstest.Test)
+ True
+ >>> sys.stderr = orig_stderr
+
+
Suppressing multiple doctest errors
-----------------------------------
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-ex/brokenlayer.py
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-ex/brokenlayer.py
---
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-ex/brokenlayer.py
1970-01-01 01:00:00.000000000 +0100
+++
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-ex/brokenlayer.py
2019-10-19 11:47:21.000000000 +0200
@@ -0,0 +1,61 @@
+##############################################################################
+#
+# Copyright (c) 2012-2018 Zope Foundation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Sample tests with layers that have broken set up and tear down."""
+
+import unittest
+
+
+class BrokenSetUpLayer:
+
+ @classmethod
+ def setUp(cls):
+ raise ValueError('No value is good enough for me!')
+
+ @classmethod
+ def tearDown(cls):
+ pass
+
+
+class BrokenTearDownLayer:
+
+ @classmethod
+ def setUp(cls):
+ pass
+
+ @classmethod
+ def tearDown(cls):
+ raise TypeError('You are not my type. No-one is my type!')
+
+
+class TestSomething1(unittest.TestCase):
+
+ layer = BrokenSetUpLayer
+
+ def test_something(self):
+ pass
+
+
+class TestSomething2(unittest.TestCase):
+
+ layer = BrokenTearDownLayer
+
+ def test_something(self):
+ pass
+
+
+def test_suite():
+ suite = unittest.TestSuite()
+ suite.addTest(unittest.makeSuite(TestSomething1))
+ suite.addTest(unittest.makeSuite(TestSomething2))
+ return suite
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-ex/sample2/stdstreamstest.py
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-ex/sample2/stdstreamstest.py
---
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-ex/sample2/stdstreamstest.py
1970-01-01 01:00:00.000000000 +0100
+++
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-ex/sample2/stdstreamstest.py
2019-10-19 11:47:21.000000000 +0200
@@ -0,0 +1,47 @@
+##############################################################################
+#
+# Copyright (c) 2003 Zope Foundation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+
+"""Sample tests that produce output on stdout and stderr."""
+
+import sys
+import unittest
+
+
+class Test(unittest.TestCase):
+
+ def test_stdout_success(self):
+ sys.stdout.write("stdout output on success\n")
+
+ def test_stdout_failure(self):
+ sys.stdout.write("stdout output on failure\n")
+ self.assertTrue(False)
+
+ def test_stdout_error(self):
+ sys.stdout.write("stdout output on error\n")
+ raise Exception("boom")
+
+ def test_stderr_success(self):
+ sys.stderr.write("stderr output on success\n")
+
+ def test_stderr_failure(self):
+ sys.stderr.write("stderr output on failure\n")
+ self.assertTrue(False)
+
+ def test_stderr_error(self):
+ sys.stderr.write("stderr output on error\n")
+ raise Exception("boom")
+
+
+def test_suite():
+ return unittest.makeSuite(Test)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-stops-when-stop-on-error.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-stops-when-stop-on-error.rst
---
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-stops-when-stop-on-error.rst
2019-03-19 08:17:18.000000000 +0100
+++
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-stops-when-stop-on-error.rst
2019-10-19 11:47:21.000000000 +0200
@@ -35,7 +35,7 @@
testrunner-ex-37/stop_on_error.py", Line NNN, in test
self.assertTrue(False)
AssertionError: False is not true
- Ran 1 tests with 2 failures, 0 errors and 0 skipped in N.NNN seconds.
+ Ran 1 tests with 1 failures, 0 errors and 0 skipped in N.NNN seconds.
Tearing down left over layers:
Tear down layers.LayerB in N.NNN seconds.
Total: 2 tests, 2 failures, 0 errors and 0 skipped in N.NNN seconds.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-subunit-v2.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-subunit-v2.rst
--- old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-subunit-v2.rst
2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-subunit-v2.rst
2019-10-19 11:47:21.000000000 +0200
@@ -291,6 +291,86 @@
True
+Capturing output
+----------------
+
+To avoid corrupting subunit streams, any output on stdout and stderr is
+buffered; for failing and erroring tests, it is recorded in the subunit
+stream as MIME-encoded chunks of text.
+
+ >>> sys.argv = 'test -ssample2 --tests-pattern ^stdstreamstest$'.split()
+ >>> subunit_summarize(testrunner.run_internal, defaults)
+ id=zope.testrunner.layer.UnitTests:setUp status=inprogress !runnable
+ id=zope.testrunner.layer.UnitTests:setUp status=success tags=(zope:layer)
+ !runnable
+ id=sample2.stdstreamstest.Test.test_stderr_error status=inprogress
+ id=sample2.stdstreamstest.Test.test_stderr_error
+ test-stderr (text/plain; charset="utf8")
+ stderr output on error
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stderr_error
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_error
+ raise Exception("boom")
+ Exception: boom
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stderr_error status=fail
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=sample2.stdstreamstest.Test.test_stderr_failure status=inprogress
+ id=sample2.stdstreamstest.Test.test_stderr_failure
+ test-stderr (text/plain; charset="utf8")
+ stderr output on failure
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stderr_failure
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stderr_failure status=fail
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=sample2.stdstreamstest.Test.test_stderr_success status=inprogress
+ id=sample2.stdstreamstest.Test.test_stderr_success status=success
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=sample2.stdstreamstest.Test.test_stdout_error status=inprogress
+ id=sample2.stdstreamstest.Test.test_stdout_error
+ test-stdout (text/plain; charset="utf8")
+ stdout output on error
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stdout_error
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_error
+ raise Exception("boom")
+ Exception: boom
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stdout_error status=fail
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=sample2.stdstreamstest.Test.test_stdout_failure status=inprogress
+ id=sample2.stdstreamstest.Test.test_stdout_failure
+ test-stdout (text/plain; charset="utf8")
+ stdout output on failure
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stdout_failure
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ <BLANKLINE>
+ id=sample2.stdstreamstest.Test.test_stdout_failure status=fail
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=sample2.stdstreamstest.Test.test_stdout_success status=inprogress
+ id=sample2.stdstreamstest.Test.test_stdout_success status=success
+ tags=(zope:layer:zope.testrunner.layer.UnitTests)
+ id=zope.testrunner.layer.UnitTests:tearDown status=inprogress !runnable
+ id=zope.testrunner.layer.UnitTests:tearDown status=success
+ tags=(zope:layer) !runnable
+ True
+
+
Layers that can't be torn down
------------------------------
@@ -314,6 +394,34 @@
False
+Layer failures
+--------------
+
+If a layer's setUp or tearDown method fails in some other way, this is shown
+in the subunit stream.
+
+ >>> sys.argv = 'test --tests-pattern ^brokenlayer$'.split()
+ >>> subunit_summarize(testrunner.run_internal, defaults)
+ id=brokenlayer.BrokenSetUpLayer:setUp status=inprogress !runnable
+ id=brokenlayer.BrokenSetUpLayer:setUp
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ ...
+ ValueError: No value is good enough for me!
+ <BLANKLINE>
+ id=brokenlayer.BrokenSetUpLayer:setUp status=fail tags=(zope:layer)
+ ...
+ id=brokenlayer.BrokenTearDownLayer:tearDown status=inprogress !runnable
+ id=brokenlayer.BrokenTearDownLayer:tearDown
+ traceback (text/x-traceback...)
+ Traceback (most recent call last):
+ ...
+ TypeError: You are not my type. No-one is my type!
+ <BLANKLINE>
+ id=brokenlayer.BrokenTearDownLayer:tearDown status=fail tags=(zope:layer)
+ True
+
+
Module import errors
--------------------
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-subunit.rst
new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-subunit.rst
--- old/zope.testrunner-5.0/src/zope/testrunner/tests/testrunner-subunit.rst
2019-03-19 08:17:18.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope/testrunner/tests/testrunner-subunit.rst
2019-10-19 11:47:21.000000000 +0200
@@ -283,6 +283,122 @@
True
+Capturing output
+----------------
+
+To avoid corrupting subunit streams, any output on stdout and stderr is
+buffered; for failing and erroring tests, it is recorded in the subunit
+stream as MIME-encoded chunks of text.
+
+ >>> sys.argv = 'test -ssample2 --tests-pattern ^stdstreamstest$'.split()
+ >>> testrunner.run_internal(defaults)
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: zope.testrunner.layer.UnitTests:setUp
+ tags: zope:layer
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ successful: zope.testrunner.layer.UnitTests:setUp
+ tags: zope:layer:zope.testrunner.layer.UnitTests
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stderr_error
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ error: sample2.stdstreamstest.Test.test_stderr_error [ multipart
+ Content-Type: text/plain;charset=utf8
+ test-stderr
+ 17\r
+ stderr output on error
+ 0\r
+ <BLANKLINE>
+ Content-Type: text/x-traceback...
+ traceback
+ NNN\r
+ <BLANKLINE>
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_error
+ raise Exception("boom")
+ Exception: boom
+ 0\r
+ <BLANKLINE>
+ ]
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stderr_failure
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ failure: sample2.stdstreamstest.Test.test_stderr_failure [ multipart
+ Content-Type: text/plain;charset=utf8
+ test-stderr
+ 19\r
+ stderr output on failure
+ 0\r
+ <BLANKLINE>
+ Content-Type: text/x-traceback...
+ traceback
+ NNN\r
+ <BLANKLINE>
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stderr_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ 0\r
+ <BLANKLINE>
+ ]
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stderr_success
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ successful: sample2.stdstreamstest.Test.test_stderr_success
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stdout_error
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ error: sample2.stdstreamstest.Test.test_stdout_error [ multipart
+ Content-Type: text/plain;charset=utf8
+ test-stdout
+ 17\r
+ stdout output on error
+ 0\r
+ <BLANKLINE>
+ Content-Type: text/x-traceback...
+ traceback
+ NNN\r
+ <BLANKLINE>
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_error
+ raise Exception("boom")
+ Exception: boom
+ 0\r
+ <BLANKLINE>
+ ]
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stdout_failure
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ failure: sample2.stdstreamstest.Test.test_stdout_failure [ multipart
+ Content-Type: text/plain;charset=utf8
+ test-stdout
+ 19\r
+ stdout output on failure
+ 0\r
+ <BLANKLINE>
+ Content-Type: text/x-traceback...
+ traceback
+ NNN\r
+ <BLANKLINE>
+ Traceback (most recent call last):
+ testrunner-ex/sample2/stdstreamstest.py", Line NNN, in test_stdout_failure
+ self.assertTrue(False)
+ AssertionError: False is not true
+ 0\r
+ <BLANKLINE>
+ ]
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: sample2.stdstreamstest.Test.test_stdout_success
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ successful: sample2.stdstreamstest.Test.test_stdout_success
+ tags: -zope:layer:zope.testrunner.layer.UnitTests
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ test: zope.testrunner.layer.UnitTests:tearDown
+ tags: zope:layer
+ time: YYYY-MM-DD HH:MM:SS.mmmmmmZ
+ successful: zope.testrunner.layer.UnitTests:tearDown
+ True
+
+
Layers that can't be torn down
------------------------------
@@ -312,6 +428,33 @@
False
+Layer failures
+--------------
+
+If a layer's setUp or tearDown method fails in some other way, this is shown
+in the subunit stream.
+
+ >>> sys.argv = 'test --tests-pattern ^brokenlayer$'.split()
+ >>> testrunner.run_internal(defaults)
+ time: ...
+ test: brokenlayer.BrokenSetUpLayer:setUp
+ tags: zope:layer
+ failure: brokenlayer.BrokenSetUpLayer:setUp [
+ Traceback (most recent call last):
+ ...
+ ValueError: No value is good enough for me!
+ ]
+ time: ...
+ test: brokenlayer.BrokenTearDownLayer:tearDown
+ tags: zope:layer
+ failure: brokenlayer.BrokenTearDownLayer:tearDown [
+ Traceback (most recent call last):
+ ...
+ TypeError: You are not my type. No-one is my type!
+ ]
+ True
+
+
Module import errors
--------------------
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope.testrunner.egg-info/PKG-INFO
new/zope.testrunner-5.1/src/zope.testrunner.egg-info/PKG-INFO
--- old/zope.testrunner-5.0/src/zope.testrunner.egg-info/PKG-INFO
2019-03-19 08:17:19.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope.testrunner.egg-info/PKG-INFO
2019-10-19 11:47:21.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: zope.testrunner
-Version: 5.0
+Version: 5.1
Summary: Zope testrunner script.
Home-page: https://github.com/zopefoundation/zope.testrunner
Author: Zope Foundation and Contributors
@@ -186,6 +186,21 @@
zope.testrunner Changelog
===========================
+ 5.1 (2019-10-19)
+ ================
+
+ - Recover more gracefully when layer setUp or tearDown fails, producing
+ useful subunit output.
+
+ - Prevent a spurious warning from the ``--require-unique`` option if
the
+ ``--module`` option was not used.
+
+ - Add optional buffering of standard output and standard error during
tests,
+ requested via the ``--buffer`` option or enabled by default for
subunit.
+
+ - Fix incorrect failure counts in per-layer summary output, broken in
4.0.1.
+
+
5.0 (2019-03-19)
================
@@ -599,5 +614,5 @@
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*
Provides-Extra: test
-Provides-Extra: docs
Provides-Extra: subunit
+Provides-Extra: docs
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/zope.testrunner-5.0/src/zope.testrunner.egg-info/SOURCES.txt
new/zope.testrunner-5.1/src/zope.testrunner.egg-info/SOURCES.txt
--- old/zope.testrunner-5.0/src/zope.testrunner.egg-info/SOURCES.txt
2019-03-19 08:17:19.000000000 +0100
+++ new/zope.testrunner-5.1/src/zope.testrunner.egg-info/SOURCES.txt
2019-10-19 11:47:21.000000000 +0200
@@ -1,10 +1,9 @@
.coveragerc
-.gitignore
.travis.yml
CHANGES.rst
COPYRIGHT.rst
DEVELOPING.rst
-LICENSE.rst
+LICENSE.md
MANIFEST.in
README.rst
appveyor.yml
@@ -131,6 +130,7 @@
src/zope/testrunner/tests/testrunner-wo-source.rst
src/zope/testrunner/tests/testrunner.rst
src/zope/testrunner/tests/testrunner-ex/README.rst
+src/zope/testrunner/tests/testrunner-ex/brokenlayer.py
src/zope/testrunner/tests/testrunner-ex/gc0.py
src/zope/testrunner/tests/testrunner-ex/gc1.py
src/zope/testrunner/tests/testrunner-ex/gcset.py
@@ -191,6 +191,7 @@
src/zope/testrunner/tests/testrunner-ex/sample2/sampletests_ntd.py
src/zope/testrunner/tests/testrunner-ex/sample2/sampletests_ntds.py
src/zope/testrunner/tests/testrunner-ex/sample2/stderrtest.py
+src/zope/testrunner/tests/testrunner-ex/sample2/stdstreamstest.py
src/zope/testrunner/tests/testrunner-ex/sample2/do-not-enter/sampletests.py
src/zope/testrunner/tests/testrunner-ex/sample2/donotenter/sampletests.py
src/zope/testrunner/tests/testrunner-ex/sample2/sample21/__init__.py