[issue17908] Unittest runner needs an option to call gc.collect() after each test

[STINNER Victor]

STINNER Victor  added the comment:

This issue is no newcomer friendly, I remove the "easy" keyword.

--
keywords:  -easy
versions: +Python 3.9 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Adam]

Adam added the comment:

Any comments about this proposed patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[STINNER Victor]

STINNER Victor added the comment:

While I like the idea of adding an option to force a garbage collection after 
running each time, I don't like how unittest is growing :-( It looks more and 
more like regrtest and I hate regrtest (I reworked its code recently to create 
a less ugly "libregrtes").

Once Antoine Pitrou proposed to add "plugins" to unittest. I agree that we need 
a gate between unittest and all other test frameworks: nose, py.test, 
testtools/testrepository/testr, etc.

Seriously, don't you think that something is wrong with such API?

def __init__(self, module='__main__', defaultTest=None, argv=None,
 testRunner=None, testLoader=loader.defaultTestLoader,
 exit=True, verbosity=1, failfast=None, catchbreak=None,
 buffer=None, warnings=None, *, tb_locals=False,
 gc_enabled=False):

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[STINNER Victor]

STINNER Victor added the comment:

Well, maybe we can add yet another parameter to all unittest classes
(push this change), but IMHO someone must rework the unittest API to
make it more extensible instead of adding more and more stuff in the
base API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Ankit Baruah]

Changes by Ankit Baruah :


--
nosy: +@nkit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Robert Collins]

Robert Collins added the comment:

say, something to run a test until failure, or to
watch for reference leaks, or to run tests in multiple processes :-))

I think a few complimentary things.

unittest extensability currently requires a new CLI entry point for each thing. 
I'd like to fix that.

The actual plumbing is fairly extensible, though its not always obvious. I'd 
like to fix that too - without any global state getting involved.

Of the things you mention, running a given command line until failure and 
checking for reference leaks are both straight forward, very common requests 
(as is the gc check) and I'd like to see those implemented as extensions 
shipped in the stdlib.

Running in parallel becomes important when one is doing slow (e.g. functional) 
tests with unittest, and I think thats important to support. It is however much 
harder to do well: some of the current idioms that have snuck in (like the 
handling of stdout/stderr capturing without the buffer flag) are not well 
matched to the needs of reporting on concurrent tests to users. I'm not in any 
way opposed to a good implementation, but it would need to be good I think - 
there's not much point having a poor implementation, given the rich set of 
parallel test runners that are out there that already build on the unittest 
core (green, nose, testrepository just for starters). The only unique audience 
for stdlib test facilities is the stdlib itself and I think a better way to 
solve that is to enable the use of alternative runners for our own tests 
(moving them to be a little cleaner should enable that) - and then the point 
where it matters is really 'when buildbots would be enough faster to make a di
 fference'.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le 28/04/2015 21:00, Robert Collins a écrit :
 So you could add this as a hook to the loader (decorate each test
 with some new thing) and a CLI option to use that hook for a gc
 collect call.

Can I take this as meaning you're not opposed to adding other options to
the unittest core? (say, something to run a test until failure, or to
watch for reference leaks, or to run tests in multiple processes :-))

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Guido van Rossum]

Guido van Rossum added the comment:

It's trivial to add to a single test or even a single TestCase subclass, yes. 
The use case is more that you have a ton of tests and you suspect there's a 
problem due to GC. Being able to call gc.collect() after each test through the 
flip of a flag would be useful.

But I agree it's iffy whether this belongs in the unittest package; we could 
add it to the CPython run_tests.py script, or you could request this as a 
feature from py.test.

If someone wants to add this to run_tests.py I think this issue would be the 
place to review the patch.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Antoine Pitrou]

Antoine Pitrou added the comment:

I still think that it's something that people can trivially implement and that 
has no place in the standard unittest package.

--
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue23839 where proposed not only call gc.collect(), but clear all 
caches in test.regrtest.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Robert Collins]

Robert Collins added the comment:

I'm going to disagree with michael and antoine here.

The *internals* should be clean and pluggable for sure, but this is actually a 
pretty common thing to try, so there's no reason to force it to only be done by 
external plugins.

Right now the way to plug this in has been complicated by the addition of 
module / class suites, which already perform extra work around individual 
tests, but in a non-introspectable / extensible fashion.

So you could add this as a hook to the loader (decorate each test with some new 
thing) and a CLI option to use that hook for a gc collect call.

Alternatively, we could face down the class/module stuff and rearrange it to be 
extensible (e.g. something along the lines of testresources internals - generic 
groups pre-post stuff) or via some interaction with TestResult but I really 
dislike using TestResult to control the run - I have a better layout mostly 
sketched in mind but haven't had time to formalise it.

So I recommend the TestLoader hook point - its well within the current 
responsibilities for it to do this, and I don't see any long term maintenance 
issues.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Adam]

Adam added the comment:

Is this enhancement still open? I've run into this problem previously, and 
would be more than happy to implement this feature.

--
nosy: +azsorkin

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Ned Deily]

Changes by Ned Deily n...@acm.org:


--
nosy: +rbcollins

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Guido van Rossum]

Guido van Rossum added the comment:

Does the cpython test runner have this? Might be nice there. Not sure if
the default test runner is something to extend at this point. Eveybody is
using py.test anyway...

On Monday, April 27, 2015, Ned Deily rep...@bugs.python.org wrote:


 Changes by Ned Deily n...@acm.org javascript:;:


 --
 nosy: +rbcollins

 ___
 Python tracker rep...@bugs.python.org javascript:;
 http://bugs.python.org/issue17908
 ___


--
title: Unittest runner needs an option to call gc.collect() after each test 
- Unittest runner needs an option to call gc.collect() after each test

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy:  -demian.brecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Andrew Svetlov]

Changes by Andrew Svetlov andrew.svet...@gmail.com:


--
nosy: +asvetlov

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Demian Brecht]

Changes by Demian Brecht demianbre...@gmail.com:


--
nosy: +dbrecht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Phil Connell]

Changes by Phil Connell pconn...@gmail.com:


--
nosy: +pconnell

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Antoine Pitrou]

Antoine Pitrou added the comment:

 TBH I would *love* for there to be a standardized way to extend the
 command line options!  If it existed I would help myself.  As it is,
 that's way to complicated.  (Then again, most customizations to the
 default test runner are too complicated IMO. :-)

+1. I found myself wanting to add a -F option à la regrtest, and I
ended up processing sys.argv manually :-/

--
title: Unittest runner needs an option to call gc.collect() after each test - 
Unittest runner needs an option to call gc.collect() after each test

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Michael Foord]

Michael Foord added the comment:

The default test runner is quite horrible.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Michael Foord]

Michael Foord added the comment:

I'm afraid I'm inclined to agree with Antoine. This seems a relatively 
specialised problem and I'd expect to see it solved in the test system rather 
than in unittest itself.

One solution, and something I'd like to see, is a way for test systems to 
extend the unittest command line handling so that they can add extra options. 
At the moment it's very painful to do.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Guido van Rossum]

Guido van Rossum added the comment:

TBH I would *love* for there to be a standardized way to extend the command 
line options!  If it existed I would help myself.  As it is, that's way to 
complicated.  (Then again, most customizations to the default test runner are 
too complicated IMO. :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Giampaolo Rodola']

Changes by Giampaolo Rodola' g.rod...@gmail.com:


--
nosy: +giampaolo.rodola

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Antoine Pitrou]

Antoine Pitrou added the comment:

 OTOH it's a useful option to have in case you're tracking down
 something that happens (or doesn't happen) when an object is collected

IMO this is a good reason to implement your specific tearDown method (or call 
addCleanup if you prefer), possibly in a shared base class.

I don't think this is a good candidate for a command-line option, it's too 
specialized and it's also not something which should be enabled blindly. In 
other words, each test case should know whether it needs a collection or not.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Guido van Rossum]

Guido van Rossum added the comment:

On Sun, May 5, 2013 at 4:29 AM, Antoine Pitrou rep...@bugs.python.org wrote:

 Antoine Pitrou added the comment:

 OTOH it's a useful option to have in case you're tracking down
 something that happens (or doesn't happen) when an object is collected

 IMO this is a good reason to implement your specific tearDown method (or call 
 addCleanup if you prefer), possibly in a shared base class.

 I don't think this is a good candidate for a command-line option, it's too 
 specialized and it's also not something which should be enabled blindly. In 
 other words, each test case should know whether it needs a collection or not.

This is not for tests that know or expect they need a call to
gc.collect(). This is for the case where you have 500 tests that
weren't written with gc.collect() in mind, and suddenly you have a
nondeterministic failure because something goes wrong during
collection. The cause is probably many tests earlier -- and if you
could just call gc.collect() in each tearDown() it would be a cinch to
pinpoint the test that causes this. But (unless you had all that
foresight) that's a massive undertaking. However turning on the
command line option makes it trivial.

It's rare that extra collections cause tests to fail (and if it does
that's a flakey test anyway) so just turning this on for all tests
shouldn't affect correct tests -- however it can slow down your test
suite 3x or more so you don't want this to be unittest's default
behavior. Hence the suggestion of a command line flag.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Guido van Rossum]

New submission from Guido van Rossum:

It would be nice if there was a command-line option to tell the test runner to 
call gc.collect() after each test.

See https://groups.google.com/d/msg/python-tulip/tstjvOQowIU/IRihc8NCHZUJ -- 
Glyph points out that always doing this is problematic because it slows down 
tests too much; OTOH it's a useful option to have in case you're tracking down 
something that happens (or doesn't happen) when an object is collected (e.g. in 
__del__).

--
components: Library (Lib)
keywords: easy
messages: 188420
nosy: gvanrossum
priority: normal
severity: normal
status: open
title: Unittest runner needs an option to call gc.collect() after each test
type: enhancement
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17908] Unittest runner needs an option to call gc.collect() after each test

[Ezio Melotti]

Ezio Melotti added the comment:

See also #17534.

--
nosy: +ezio.melotti, michael.foord
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17908
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com