[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: Ok, no more regression / bug related to this issue, let close it. -- resolution: - fixed status: open - closed versions: -Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 949e0c0f5a38 by Victor Stinner in branch '3.2': Issue #12400: regrtest -W doesn't rerun the tests twice anymore, but captures http://hg.python.org/cpython/rev/949e0c0f5a38 New changeset 0172d40fdcd4 by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: regrtest -W doesn't rerun the tests twice anymore, http://hg.python.org/cpython/rev/0172d40fdcd4 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset af799df478c6 by Victor Stinner in branch 'default': Issue #12400: oops, remove debug code... http://hg.python.org/cpython/rev/af799df478c6 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 69e102bfed2b by Victor Stinner in branch '3.2': Issue #12400: remove unused variable http://hg.python.org/cpython/rev/69e102bfed2b New changeset 9d53612333c7 by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: remove unused import http://hg.python.org/cpython/rev/9d53612333c7 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: I will not backport the change in Python 2.7, because regrtest already captures stdout (but not in verbose mode), and it checks that the output is empty. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 80c7d63325f6 by Victor Stinner in branch 'default': Issue #12400: fix test_faulthandler if regrtest captures sys.stderr http://hg.python.org/cpython/rev/80c7d63325f6 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 74c6afbb524a by Victor Stinner in branch '3.2': Issue #12400: don't use sys.stderr in test_kqueue because it may be replaced by http://hg.python.org/cpython/rev/74c6afbb524a New changeset c0afc8d00c0a by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: don't use sys.stderr in test_kqueue because it may be http://hg.python.org/cpython/rev/c0afc8d00c0a -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 6c54c334ea7a by Victor Stinner in branch '3.2': Issue #12400: runtest() reuses the same io.StringIO instance for all calls http://hg.python.org/cpython/rev/6c54c334ea7a New changeset 8897c755633b by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: runtest() reuses the same io.StringIO instance for http://hg.python.org/cpython/rev/8897c755633b -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 2d07b1b4466a by Victor Stinner in branch '3.2': Issue #12400: regrtest, force verbose mode to True with option -W http://hg.python.org/cpython/rev/2d07b1b4466a New changeset 561974b760eb by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: regrtest, force verbose mode to True with option -W http://hg.python.org/cpython/rev/561974b760eb -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset d8ba59de860a by Victor Stinner in branch '3.2': Issue #12400: test.support.run_doctest() doesn't change sys.stdout anymore http://hg.python.org/cpython/rev/d8ba59de860a New changeset bb7c9308c846 by Victor Stinner in branch 'default': Issue #12400: test.support.run_doctest() doesn't change sys.stdout anymore http://hg.python.org/cpython/rev/bb7c9308c846 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
R. David Murray rdmur...@bitdance.com added the comment: Hmm. I thought Antoine got rid of the output checking when he added -j support. Well, he was the one who did it, anyway, so there may be a change set you could backport to get rid of the output checking if you want to backport this to 2.7. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset fdb98a7ef9be by Victor Stinner in branch '3.2': Issue #12400: Add missing import (os) to test_kqueue http://hg.python.org/cpython/rev/fdb98a7ef9be New changeset 37877d19c2c9 by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: Add missing import (os) to test_kqueue http://hg.python.org/cpython/rev/37877d19c2c9 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 7bd23115b6c4 by Victor Stinner in branch '3.2': Issue #12400: runtest() truncates the StringIO stream before a new test http://hg.python.org/cpython/rev/7bd23115b6c4 New changeset fc831c49216d by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: runtest() truncates the StringIO stream before a new http://hg.python.org/cpython/rev/fc831c49216d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Éric Araujo mer...@netwok.org added the comment: One more time, commit messages using the present tense are ambiguous: when you write “runtest() truncates the StringIO stream before a new test“, it’s not clear at all whether you describe the previous, incorrect behavior or the new, fixed one. These phrasings are better: “X used to do Y” (old behavior) “X now does Z” (new) “make X do Z” (new) -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: One more time, commit messages using the present tense are ambiguous: when you write “runtest() truncates the StringIO stream before a new test“, it’s not clear at all whether you describe the previous, incorrect behavior or the new, fixed one. You can guess by reading the patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Éric Araujo mer...@netwok.org added the comment: It’s irrelevant, please provide unambiguous commit message. This was discussed two or three times already. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset bc2f3fbda7e3 by Victor Stinner in branch '3.2': Issue #12400: test_zipimport_support doesn't restore original sys.stdout http://hg.python.org/cpython/rev/bc2f3fbda7e3 New changeset ff5abf93db80 by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: test_zipimport_support doesn't restore original http://hg.python.org/cpython/rev/ff5abf93db80 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Georg Brandl ge...@python.org added the comment: Eric is right. I don't see the patch when reading hg log output, or looking up revisions given by hg annotate. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: Eric is right. I don't see the patch when reading hg log output I'm using hg log -p. or looking up revisions given by hg annotate. hg annotate doesn't display the changelog, only the commit number or hash, which option do you use? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 450209efe272 by Victor Stinner in branch '3.2': Issue #12400: regrtest.runtest() uses stream.seek(0) before .truncate() http://hg.python.org/cpython/rev/450209efe272 New changeset 3ce22f1b9540 by Victor Stinner in branch 'default': (merge 3.2) Issue #12400: regrtest.runtest() uses stream.seek(0) before .truncate() http://hg.python.org/cpython/rev/3ce22f1b9540 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Ned Deily n...@acm.org added the comment: Eric is right. I don't see the patch when reading hg log output ... or when looking at the issue tracker. Yes, one can probably figure out the intent by looking at the whole patch but it's kind of rude to force people to do that. That's what the commit message is there for: to give a brief, unambiguous summary of what the change is about. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset c7bed90e4f31 by Victor Stinner in branch 'default': Issue #12400: test_faulthandler now uses sys.__stderr__ http://hg.python.org/cpython/rev/c7bed90e4f31 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
Roundup Robot devnull@devnull added the comment: New changeset 6edb7c153105 by Victor Stinner in branch 'default': Issue #12400: test_cprofile now restores correctly the previous sys.stderr http://hg.python.org/cpython/rev/6edb7c153105 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: Typical example: (... smtplib ...) Another example (yesterday): -- [355/356/2] test_subprocess ... Re-running test test_subprocess in verbose mode ... Ran 228 tests in 322.313s OK (skipped=20) -- -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
STINNER Victor victor.stin...@haypocalc.com added the comment: New patch updated according to bitdancer's comment on IRC: only change the -W option. With the patch, -W only runs the test once but captures the output. The main difference with my patch is that all output is written to stderr, even if the captured output was written into stdout. I don't want to capture stdout and stderr in two differents stream, I prefer to keep the original output order. In case of a failure, the output only concerns a failure. -- Added file: http://bugs.python.org/file22502/regrtest_verbose3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12400] regrtest: always run tests in verbose mode, but hide the output on success
New submission from STINNER Victor victor.stin...@haypocalc.com: Sporadic issues are difficult to analyze on the buildbots because sometimes a test fails, but when it runs again in verbose mode... it doesn't fail anymore. Typical example: === [ 99/356/1] test_smtplib Re-running test test_smtplib in verbose mode (...) testTimeoutValue (test.test_smtplib.GeneralTests) ... ok testBasic (test.test_smtplib.DebuggingServerTests) ... test test_smtplib failed -- multiple errors occurred; run in verbose mode for details ok testHELP (test.test_smtplib.DebuggingServerTests) ... ok testNOOP (test.test_smtplib.DebuggingServerTests) ... ok testNotImplemented (test.test_smtplib.DebuggingServerTests) ... ok (...) -- Ran 32 tests in 23.664s OK === I propose to always run the tests in verbose mode, write the output into a buffer, but only display the output on failure (or ignore the output on success). Attached patch simplifies regrtest.py usage: replace 4 verbose levels (no verbose option, -v, -w, -W) by only 2 (no verbose option, -v). Always display the full output in case of an error (as we used -w before): no more multiple error occured without output trap, no more oh, rerunning the test doesn't fail trap. The -v option works as before (directly write the output in verbose mode). regrtest is much more verbose than before in case of a failure, but I prefer too much information than no information when I try to debug something. If it is too much verbose by default, I can try to use 3 verbose levels (e.g. don't display the full output if only one test failed). If the patch is accepted, the devguide should also be patched. -- components: Tests files: regrtest_verbose.patch keywords: patch messages: 138981 nosy: ezio.melotti, georg.brandl, haypo, pitrou, r.david.murray priority: normal severity: normal status: open title: regrtest: always run tests in verbose mode, but hide the output on success versions: Python 2.7, Python 3.2, Python 3.3 Added file: http://bugs.python.org/file22446/regrtest_verbose.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12400 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
