[issue36560] test_functools leaks randomly 1 memory block

Ken Jin added the comment: @Victor, with your patch applied, for test_typing (locally, Windows x64): $ ./python -m test test_typing -R 3:20 Different result on every run (race condition maybe?) $ ./python -m test test_typing -R 1:20 Same result as yours on every run. -- nosy: +kj

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: I can see the test_typing issue with this patch: diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py index b94826a5da..49e5f03414 100644 --- a/Lib/test/libregrtest/refleak.py +++ b/Lib/test/libregrtest/refleak.py @@ -124,7 +124,7 @@

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Recent failure on AMD64 Fedora Rawhide Refleaks 3.x: https://buildbot.python.org/all/#/builders/565/builds/131 test_typing leaked [1, 1, 1] memory blocks, sum=3 (...) 0:46:40 load avg: 0.00 Re-running test_typing in verbose mode test_typing leaked [1, 1, 1]

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: I didn't see this failure recently. I close the issue. -- resolution: -> out of date stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: AMD64 Fedora Rawhide Refleaks 3.8: https://buildbot.python.org/all/#/builders/79/builds/196 test_functools leaked [1, 1, 1] memory blocks, sum=3 -- ___ Python tracker

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Recent fail on AMD64 Fedora Rawhide Refleaks 3.x: https://buildbot.python.org/all/#builders/189/builds/162 test_functools leaked [1, 2, 1] memory blocks, sum=4 -- ___ Python tracker

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Another example: AMD64 Fedora Rawhide Refleaks 3.6 https://buildbot.python.org/all/#/builders/193/builds/5 test_contextlib leaked [8, 1, 1] memory blocks, sum=10 I cannot reproduce the issue on my ("idle") laptop. It sounds like a random failure (race

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Patch making check on memory block leaks stricter: diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py index 235d6bfd3a..dfadabdef6 100644 --- a/Lib/test/libregrtest/refleak.py +++ b/Lib/test/libregrtest/refleak.py @@ -130,7 +130,7

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: New changeset 86f0354fcb815312295b923c55e39364d85d0388 by Victor Stinner in branch '3.7': [3.7] bpo-36560: regrtest: don't collect the GC twice (GH-12747) (GH-12749) https://github.com/python/cpython/commit/86f0354fcb815312295b923c55e39364d85d0388

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: > _abc was written for optimization. Your PR makes it using slower Python code. > Could you please test how much this hits the performance of abc? I mostly wrote PR 12743 to debug this issue. Please see the discussion directly on the PR ;-) --

[issue36560] test_functools leaks randomly 1 memory block

Change by STINNER Victor : -- pull_requests: +12676 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: New changeset bb4447897a5f141eecf42987a1191a3330c5d7ed by Victor Stinner in branch 'master': bpo-36560: regrtest: don't collect the GC twice (GH-12747) https://github.com/python/cpython/commit/bb4447897a5f141eecf42987a1191a3330c5d7ed --

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: New changeset 9c14061a2c2df9a9b84d0aab190a50c24a0d52f4 by Victor Stinner in branch '2.7': bpo-36560: Fix reference leak hunting in regrtest (GH-12744) (GH-12745) https://github.com/python/cpython/commit/9c14061a2c2df9a9b84d0aab190a50c24a0d52f4 --

[issue36560] test_functools leaks randomly 1 memory block

Change by STINNER Victor : -- pull_requests: +12674 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36560] test_functools leaks randomly 1 memory block

Change by STINNER Victor : -- pull_requests: +12668 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: New changeset 5aaac94eeb44697e92b0951385cd557bc27e0f6a by Victor Stinner in branch 'master': bpo-36560: Fix reference leak hunting in regrtest (GH-12744) https://github.com/python/cpython/commit/5aaac94eeb44697e92b0951385cd557bc27e0f6a --

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: If I modify libregrtest with the following patch: diff --git a/Lib/test/libregrtest/refleak.py b/Lib/test/libregrtest/refleak.py index 0bb8a0a2bf..f0225a9768 100644 --- a/Lib/test/libregrtest/refleak.py +++ b/Lib/test/libregrtest/refleak.py @@ -128,7 +128,7

[issue36560] test_functools leaks randomly 1 memory block

Change by STINNER Victor : -- pull_requests: +12667 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36560] test_functools leaks randomly 1 memory block

Serhiy Storchaka added the comment: _abc was written for optimization. Your PR makes it using slower Python code. Could you please test how much this hits the performance of abc? -- nosy: +inada.naoki, levkivskyi ___ Python tracker

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: I wrote PR 12743 to modify _abc: use weakref.WeakSet rather than using directly a set of weak references. With this PR, calling _get_dump() should clear all broken weak references. I'm not sure if it's really useful for this specific issue, since

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: This issue isn't a real memory leak: if I use -R 3:10 instead of -R 3:3, the test doesn't fail anymore. But the issue is still annoying since it makes Refleaks buildbot workers fail randomly :-/ This issue remembers me the unstable multiprocessing tests:

[issue36560] test_functools leaks randomly 1 memory block

Change by STINNER Victor : -- keywords: +patch pull_requests: +12666 stage: -> patch review ___ Python tracker ___ ___

[issue36560] test_functools leaks randomly 1 memory block

Change by Serhiy Storchaka : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: The issue comes maybe from the complex code in Lib/test/libregrtest/refleak.py to handle the ABC cache: abcs = {} for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: if not isabstract(abc): continue

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Some progress. I'm now able to reproduce the issue with attached test_functools2.py (103 lines, a single test method): $ cp (...)test_functools2.py Lib/test/ $ ./configure -C --with-pydebug CFLAGS=-O0 $ make $ ./python -m test -F -j5 -R 3:3 test_functools2

[issue36560] test_functools leaks randomly 1 memory block

STINNER Victor added the comment: Current bisection progress: I'm able to reproduce a failure with attached test_functools.py (211 lines) using: ./python -m test -F -j5 -R 3:3 test_functools It takes between 4 and 200 runs to reproduce the failure, I don't understand what triggers the

[issue36560] test_functools leaks randomly 1 memory block

New submission from STINNER Victor : Sometimes, each run of test_functools leaks exactly 1 memory block, even when the whole test is "re-run in verbose mode". Sometimes, it doesn't leak. https://buildbot.python.org/all/#/builders/80/builds/550 test_functools leaked [1, 1, 1] memory blocks,