[issue32850] Run gc_collect() before complaining about dangling threads

[Nathaniel Smith]

Nathaniel Smith  added the comment:

@Victor: So to make sure I understand, the point of the check is to complain 
about reference cycles involving thread objects, because when those happen in 
the stdlib you consider them bugs?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Matthias Urlichs]

Matthias Urlichs  added the comment:

> It's a deliberate choice. It helped me to find real bugs. For example, I 
> found a very old reference cycle in socket.create_connection().

Fair enough; I will change the patch to complain before gc'ing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[STINNER Victor]

STINNER Victor  added the comment:

> This is not useful, esp. when the list of referrers to the "dangling" thread 
> looks like this: (...)

I wrote the test.bisect tool to help to debug dangling threads issues.

Sadly, sometimes the warning can occur randomly because of race conditions, and 
it can be painful to reproduce the bug, and to fix the root issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[STINNER Victor]

STINNER Victor  added the comment:

> Lib/test/support/__init__.py::threading_cleanup() complains about dangling 
> threads even if the reference in question would be cleaned up by the garbage 
> collector.

It's a deliberate choice. It helped me to find real bugs. For example, I found 
a very old reference cycle in socket.create_connection().

The error message ("dangling threads") doesn't help, but it means that 
"something is wrong" in your test.

Sadly, sometimes you have to explicitly do something like "self.thread = None" 
in your test.

I wrote an article to explain these things:
https://vstinner.github.io/contrib-cpython-2017q3-part2.html

Instead of calling support.gc_collect(), I suggest to write documentation, 
maybe as a comment in support.threading_cleanup()?, explaining the warning and 
how to fix it. Maybe with a link to this issue? :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Nathaniel Smith]

Nathaniel Smith  added the comment:

Yes, a PR would be better... also, the two versions of the patch appear to be 
identical?

CC'ing Antoine and Victor b/c they seem to have written this dangling_threads 
checking stuff and I'm not sure I understand it :-).

As some extra background: this is in Python's internal testing code, but 
@smurfix is hitting it because he's working on a third-party asyncio event loop 
implementation, and re-using test.asyncio to test it. I'm not sure what 
specifically is causing this to happen, though I guess it has something to do 
with how trio / trio-asyncio are using threads.

[1] https://github.com/python-trio/trio-asyncio

--
nosy: +pitrou, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Christian Heimes]

Christian Heimes  added the comment:

We generally prefer pull requests on github over patches on the bug tracker. 
Please follow the guidelines for contributing to CPython, 
https://devguide.python.org/pullrequest/

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Matthias Urlichs]

Matthias Urlichs  added the comment:

Apparently this patch has not been applied yet. Is there a reason for that, 
besides "it's obviously correct so there must be something wrong with it"? ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Nathaniel Smith]

Change by Nathaniel Smith :


--
nosy: +ezio.melotti, njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Matthias Urlichs]

Matthias Urlichs  added the comment:

Upon further consideration (and following the observation that my test cases no 
longer block for two seconds each after applying the first version of this 
patch): we do not want to clear the reference to "dangling_threads" since 
that's a weakset. We want to clear the "thread" variable, which holds a 
reference to a random member of that set, which will arbitrarily block the 
cleanup loop from ever succeeding.

Updated patch attached.

--
Added file: https://bugs.python.org/file47446/gc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32850] Run gc_collect() before complaining about dangling threads

[Matthias Urlichs]

New submission from Matthias Urlichs :

Lib/test/support/__init__.py::threading_cleanup() complains about dangling 
threads even if the reference in question would be cleaned up by the garbage 
collector.

This is not useful, esp. when the list of referrers to the "dangling" thread 
looks like this:

[, , ]

Thus I propose to check, run gc, check again, and only *then* 
complain-and-wait. Hence the attached patch for your consideration.

--
components: Tests
files: gc.patch
keywords: patch
messages: 312206
nosy: smurfix
priority: normal
severity: normal
status: open
title: Run gc_collect() before complaining about dangling threads
type: resource usage
versions: Python 3.7
Added file: https://bugs.python.org/file47445/gc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com