On 8/13/2013 7:51 PM, Steven D'Aprano wrote:

http://bugs.python.org/issue18606

Tests at end of statistics.patch.

and I'm about to submit a patch containing my updated code and tests,
but I've run into a problem with testing. My existing tests use
unittest, and follow the basic boilerplate documented here:

http://docs.python.org/3/library/test.html

+def test_main():
+    # run_unittest()
+    unittest.main()
+
+
+if __name__ == "__main__":
+    test_main()

This is faulty, as explained below. It is not the boilerplate in
http://docs.python.org/3/library/test.html#writing-unit-tests-for-the-test-package

which is correct. Compress to just

if __name__ == "__main__":
   unittest_main()

To avoid all doubt, the tests pass if I call them like this:
./python Lib/test/test_statistics.py

That is a faulty test as explained below.

Patch applies cleanly on Win7, 32bit build. And indeed,
F:\Python\dev\py34\PCbuild>python_d ../Lib/test/test_statistics.py
works. (the progess dots will have to go before being applied ;-).

The above should be equivalent to
...> python -m test.test_statistics.
but it is not.

F:\Python\dev\py34\PCbuild>python_d -m test.test_statistics
Traceback (most recent call last):
  File "F:\Python\dev\py34\lib\runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "F:\Python\dev\py34\lib\runpy.py", line 73, in _run_code
    exec(code, run_globals)
File "F:\Python\dev\py34\lib\test\test_statistics.py", line 16, in <module>
    from test_statistics_approx import NumericTestCase
ImportError: No module named 'test_statistics_approx'

Your test *depends* on Lib/test being the current directory, at the beginning of the path. It should not. Your import must be
   from test.test_statistics_appox import NumericTestCase

With this fixed, the above command works.

Once the test runs from anywhere with unittest, run with regrtest,

...> python -m test test_statistics

Initially, this gives a traceback with the same last three lines. With the revision, I get

File "F:\Python\dev\py34\lib\unittest\loader.py", line 113, in loadTestsFromName
    parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_statistics'

This is related to the first fault I mentioned above. If regrtest finds the old style 'test_main', it uses the regrtest loader. If it does not, it uses the unittest loader. This is documented in the code ;-). With the proper 2-line boilerplate as a second change, the second line now works.

but raise errors when I call them like this:
./python -m test -j3

LOL. In one jump, you changed the current directory, the test runner, the number of test files run, and the mode of testing. Try just one change at a time.

When you run the suite, you run test_statistics_approx.py, including the test case imported into test_statistics. (Is it run twice?).

Tested as part of the suite gives a bizarre message I do not pretend to understand.
[290/379/3] test_statistics_approx
Usage: regrtest.py [options]

regrtest.py: error: no such option: --slaveargs
test test_statistics_approx crashed -- Traceback (most recent call last):
  File "F:\Python\dev\py34\lib\optparse.py", line 1391, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "F:\Python\dev\py34\lib\optparse.py", line 1431, in _process_args
    self._process_long_opt(rargs, values)
File "F:\Python\dev\py34\lib\optparse.py", line 1484, in _process_long_opt
    opt = self._match_long_opt(opt)
  File "F:\Python\dev\py34\lib\optparse.py", line 1469, in _match_long_opt
    return _match_abbrev(opt, self._long_opt)
  File "F:\Python\dev\py34\lib\optparse.py", line 1674, in _match_abbrev
    raise BadOptionError(s)
optparse.BadOptionError: no such option: --slaveargs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "F:\Python\dev\py34\lib\test\regrtest.py", line 1305, in runtest_inner
    test_runner()
File "F:\Python\dev\py34\lib\test\test_statistics_approx.py", line 597, in test_main
    unittest.main()
  File "F:\Python\dev\py34\lib\unittest\main.py", line 124, in __init__
    self.parseArgs(argv)
  File "F:\Python\dev\py34\lib\unittest\main.py", line 148, in parseArgs
    options, args = parser.parse_args(argv[1:])
  File "F:\Python\dev\py34\lib\optparse.py", line 1393, in parse_args
    self.error(str(err))
  File "F:\Python\dev\py34\lib\optparse.py", line 1573, in error
    self.exit(2, "%s: error: %s\n" % (self.get_prog_name(), msg))
  File "F:\Python\dev\py34\lib\optparse.py", line 1563, in exit
    sys.exit(status)
SystemExit: 2

So back up and test it by itself first. It passes under unittest, but needs the revised boilerplate for regrtest. The two together work fine. When I rerun the suite, poof!, the mysterious tracback is gone and both pass.

The following appears after the 2nd. '''
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK
'''
Perhaps this is from the doctest. If so, it probably should be surpressed.

One last thing: test order. Buildbots randomize the order of running files, so if your tests modify the environment in a way that affects another test, it might cause occasional failures.

--
Terry Jan Reedy

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Reply via email to