Re: [Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)
Peter Astrand wrote: I'd like to have your opinion on this bug. Personally, I'd prefer to keep test_no_leaking as it is, but if you think otherwise... One thing that actually can motivate that test_subprocess takes 20% of the overall time is that this test is a good generic Python stress test - this test might catch some other startup race condition, for example. test_decimal has a short version which tests basic functionality and always runs, but enabling -udecimal also runs the specification tests (which take a fair bit longer). So keeping the basic subprocess tests unconditional, and running the long ones only if -uall or -usubprocess are given would seem reasonable. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)
I'd like to have your opinion on this bug. Personally, I'd prefer to keep test_no_leaking as it is, but if you think otherwise... One thing that actually can motivate that test_subprocess takes 20% of the overall time is that this test is a good generic Python stress test - this test might catch some other startup race condition, for example. A suite of unit tests is a precious thing. We want to test as much as we can, and as thoroughly as possible; but at the same time we want the test to run reasonably fast. If the test takes too long, human nature being what it is, this will actually cause less thorough testing because developers don't feel like running the test suite after each small change, and then we get frequent problems where someone breaks the build because they couldn't wait to run the unit test. (For example, where I work we have a Java test suite that takes 25 minutes to run. The build is broken on a daily basis by developers (including me) who make a small change and check it in believing it won't break anything.) The Python test suite already has a way (the -u flag) to distinguish between regular broad-coverage testing and deep coverage for specific (or all) areas. Let's keep the really long-running tests out of the regular test suite. There used to be a farm of machines that did nothing but run the test suite (snake-farm). This seems to have stopped (it was run by volunteers at a Swedish university). Maybe we should revive such an effort, and make sure it runs with -u all. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)
Guido van Rossum wrote: [...] There used to be a farm of machines that did nothing but run the test suite (snake-farm). This seems to have stopped (it was run by volunteers at a Swedish university). Maybe we should revive such an effort, and make sure it runs with -u all. I've changed the job that produces the data for http://coverage.livinglogic.de/ to run python Lib/test/regrtest.py -uall -T -N Unfortunately this job currently produces only coverage info, the output of the test suite is thrown away. It should be easy to fix this, so that the output gets put into the database. Bye, Walter Dörwald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)
Guido van Rossum wrote: The Python test suite already has a way (the -u flag) to distinguish between regular broad-coverage testing and deep coverage for specific (or all) areas. Let's keep the really long-running tests out of the regular test suite. There used to be a farm of machines that did nothing but run the test suite (snake-farm). This seems to have stopped (it was run by volunteers at a Swedish university). Maybe we should revive such an effort, and make sure it runs with -u all. Hello Guido and everybody else, I hacked together a simple distributed unittest runner for our projects. Requirements are a NFS-mounted home directory across the slave nodes and SSH-based automatic authentication, i.e. no passwords or passphrases necessary. It officially works-for-me for around three hosts (see below) so that cuts the time down basically to a third (real-life example ~600 seconds to ~200 seconds, so it does work :-). It also supports serialized tests, i.e. tests that must be run one after the other and cannot be run in parallel. http://mde.abo.fi/tools/disttest/ Comes with some problems; my blurb from advogato.org: Disttest is a distributed unittesting runner. You simply set the DISTTEST_HOSTS variable to a space-separated list of hostnames to connect to using SSH, and then run disttest. The nodes must all have the same filesystem (usually an NFS-mounted /home) and have the Disttest program installed. You even gain a bit with just one computer by setting the variable to localhost localhost. :-) There are currently two annoying problem with it, though. For some reason, 1) the unittest program connecting to the X server sometimes fails to provide the correct authentication, and 2) sometimes the actual connection to the X server can't be established. I think these are related to 1) congestion on the shared .Xauthority file, and 2) a too small listen() queue on the forwarding port by the SSH daemon. Both problems show up when using too many (over 4?) hosts, which is the whole point of the program! Sigh. Error checking probably bad. Anyway, feel free to check it out, modify, comment or anything. We're thinking of checking the assumptions in the blurb above, but no timetable is set. My guess is that the NFS-mounted home directory is the showstopper and people usually don't have lot's of machines hanging around, but that's for you to decide. Disclaimer: I don't know anything of CPython development nor of the tests in the CPython test suite. ;-) Best regards, and a big thank you for Python, Marcus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
