[Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Steven D'Aprano

Hi,

I have raise a tracker item and PEP for adding a statistics module to the 
standard library:

http://bugs.python.org/issue18606

http://www.python.org/dev/peps/pep-0450/


and I'm about to submit a patch containing my updated code and tests, but I've 
run into a problem with testing. My existing tests use unittest, and follow the 
basic boilerplate documented here:

http://docs.python.org/3/library/test.html

and when run directly they pass, but when I try to run them using Python 3.4a 
-m test -j3 they break. My question is, is it acceptable to post the code and 
tests to the tracker as-is, and ask for a pronouncement on the PEP first, and 
then fix the test breakage later? If not, can somebody mentor me in 
understanding what I need to do here?

To avoid all doubt, the tests pass if I call them like this:


./python Lib/test/test_statistics.py


but raise errors when I call them like this:

./python -m test -j3


Thanks in advance,




--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Ethan Furman

On 08/13/2013 04:51 PM, Steven D'Aprano wrote:


I have raise a tracker item and PEP for adding a statistics module to the 
standard library:

http://bugs.python.org/issue18606

http://www.python.org/dev/peps/pep-0450/


The bug-tracker doesn't think you've submitted a CLA yet.  If you haven't 
you'll need to get that done.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Ethan Furman

On 08/13/2013 04:51 PM, Steven D'Aprano wrote:


My question is, is it acceptable to post the code and tests to
 the tracker as-is, and ask for a pronouncement on the PEP first,
and then fix the test breakage later?


Certainly.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Victor Stinner
Send the patch somewhere (ex: attach it to an email, or to the bug
tracker, as you want), or give the error message, if you want some
help.

 Ask for a pronouncement on the PEP first, and then fix the test breakage 
 later?

Sometimes, it's possible to pronounce on a PEP without a working
implementation. But it's easier to pronounce with a working
implementation :-)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Nick Coghlan
On 13 Aug 2013 19:40, Victor Stinner victor.stin...@gmail.com wrote:

 Send the patch somewhere (ex: attach it to an email, or to the bug
 tracker, as you want), or give the error message, if you want some
 help.

  Ask for a pronouncement on the PEP first, and then fix the test
breakage later?

 Sometimes, it's possible to pronounce on a PEP without a working
 implementation. But it's easier to pronounce with a working
 implementation :-)

It's more typical for reference implementations to be proof of concept
quality code, though. It's very rare to have a full, ready to apply patch
(although we certainly don't complain if that happens!)

(To directly answer the original question: a test integration glitch isn't
a blocker for PEP acceptance, just for actually committing the
implementation)

Cheers,
Nick.


 Victor
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Guidance regarding tests for the standard lib

2013-08-13 Thread Terry Reedy

On 8/13/2013 7:51 PM, Steven D'Aprano wrote:


http://bugs.python.org/issue18606


Tests at end of statistics.patch.


and I'm about to submit a patch containing my updated code and tests,
but I've run into a problem with testing. My existing tests use
unittest, and follow the basic boilerplate documented here:

http://docs.python.org/3/library/test.html


+def test_main():
+# run_unittest()
+unittest.main()
+
+
+if __name__ == __main__:
+test_main()

This is faulty, as explained below. It is not the boilerplate in
http://docs.python.org/3/library/test.html#writing-unit-tests-for-the-test-package

which is correct. Compress to just

if __name__ == __main__:
   unittest_main()


To avoid all doubt, the tests pass if I call them like this:
./python Lib/test/test_statistics.py


That is a faulty test as explained below.

Patch applies cleanly on Win7, 32bit build. And indeed,
F:\Python\dev\py34\PCbuildpython_d ../Lib/test/test_statistics.py
works. (the progess dots will have to go before being applied ;-).

The above should be equivalent to
... python -m test.test_statistics.
but it is not.

F:\Python\dev\py34\PCbuildpython_d -m test.test_statistics
Traceback (most recent call last):
  File F:\Python\dev\py34\lib\runpy.py, line 160, in _run_module_as_main
__main__, fname, loader, pkg_name)
  File F:\Python\dev\py34\lib\runpy.py, line 73, in _run_code
exec(code, run_globals)
  File F:\Python\dev\py34\lib\test\test_statistics.py, line 16, in 
module

from test_statistics_approx import NumericTestCase
ImportError: No module named 'test_statistics_approx'

Your test *depends* on Lib/test being the current directory, at the 
beginning of the path. It should not. Your import must be

   from test.test_statistics_appox import NumericTestCase

With this fixed, the above command works.

Once the test runs from anywhere with unittest, run with regrtest,

... python -m test test_statistics

Initially, this gives a traceback with the same last three lines. With 
the revision, I get


  File F:\Python\dev\py34\lib\unittest\loader.py, line 113, in 
loadTestsFromName

parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_statistics'

This is related to the first fault I mentioned above. If regrtest finds 
the old style 'test_main', it uses the regrtest loader. If it does not, 
it uses the unittest loader. This is documented in the code ;-). With 
the proper 2-line boilerplate as a second change, the second line now works.



but raise errors when I call them like this:
./python -m test -j3


LOL. In one jump, you changed the current directory, the test runner, 
the number of test files run, and the mode of testing. Try just one 
change at a time.


When you run the suite, you run test_statistics_approx.py, including the 
test case imported into test_statistics. (Is it run twice?).


Tested as part of the suite gives a bizarre message I do not pretend to 
understand.

[290/379/3] test_statistics_approx
Usage: regrtest.py [options]

regrtest.py: error: no such option: --slaveargs
test test_statistics_approx crashed -- Traceback (most recent call last):
  File F:\Python\dev\py34\lib\optparse.py, line 1391, in parse_args
stop = self._process_args(largs, rargs, values)
  File F:\Python\dev\py34\lib\optparse.py, line 1431, in _process_args
self._process_long_opt(rargs, values)
  File F:\Python\dev\py34\lib\optparse.py, line 1484, in 
_process_long_opt

opt = self._match_long_opt(opt)
  File F:\Python\dev\py34\lib\optparse.py, line 1469, in _match_long_opt
return _match_abbrev(opt, self._long_opt)
  File F:\Python\dev\py34\lib\optparse.py, line 1674, in _match_abbrev
raise BadOptionError(s)
optparse.BadOptionError: no such option: --slaveargs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File F:\Python\dev\py34\lib\test\regrtest.py, line 1305, in 
runtest_inner

test_runner()
  File F:\Python\dev\py34\lib\test\test_statistics_approx.py, line 
597, in test_main

unittest.main()
  File F:\Python\dev\py34\lib\unittest\main.py, line 124, in __init__
self.parseArgs(argv)
  File F:\Python\dev\py34\lib\unittest\main.py, line 148, in parseArgs
options, args = parser.parse_args(argv[1:])
  File F:\Python\dev\py34\lib\optparse.py, line 1393, in parse_args
self.error(str(err))
  File F:\Python\dev\py34\lib\optparse.py, line 1573, in error
self.exit(2, %s: error: %s\n % (self.get_prog_name(), msg))
  File F:\Python\dev\py34\lib\optparse.py, line 1563, in exit
sys.exit(status)
SystemExit: 2

So back up and test it by itself first. It passes under unittest, but 
needs the revised boilerplate for regrtest. The two together work fine. 
When I rerun the suite, poof!, the mysterious tracback is gone and both 
pass.


The following appears after the 2nd. '''
..
--
Ran 2 tests in 0.000s

OK
'''
Perhaps this