[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: Neil, my brief msg 10 minutes before yours suggested the same thing (just clear the weakref), so it must be right ;-) -- ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: Oops, I linked to wrong PR, my proposed fix is GH-16495. It is the same as what Tim suggests in his last comment. -- ___ Python tracker

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: > Why setting it back to release blocker? As far as I recall, this issue is not > a Python 3.8 regression. The regression which triggered this old and existing > bug has been fixed, see previous comments. I leave it up to our glorious release manager to

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: Neil, how about this alternative: leave the weakref implementation alone. If we find a trash weakref, simply clear it instead. That would prevent callbacks too, & would also prevent the weakref from being used to retrieve its possibly-trash-too referent.

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by Neil Schemenauer : -- pull_requests: +16083 pull_request: https://github.com/python/cpython/pull/16495 ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: > Would the attached rough patch (gc_disable_wr_callback.txt) > be a possible fix? When we find W inside handle_weakrefs(), > we mark it as trash and will not execute the callback. It's semantically correct since we never wanted to execute a callback from a

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > 2019-09-30 14:41:36 lukasz.langaset priority: release blocker Why setting it back to release blocker? As far as I recall, this issue is not a Python 3.8 regression. The regression which triggered this old and existing bug has been fixed, see

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by Łukasz Langa : -- priority: -> release blocker ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: Ah, nevermind my last comment - yes. handle_weakrefs will clear all weakrefs to the objects we know are trash. -- ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: > I see that handle_weakrefs() calls _PyWeakref_ClearRef() and that > will clear the weakref even if it doesn't have callback. So, I > think that takes care for the hole I was worried about. I.e. a > __del__ method could have a weakref to an non-valid object. >

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: > Note that my flags show that W *is* in 'unreachable'. It has > to be otherwise F would not have tp_clear called on it. Right! W holds a strong reference to F, so if W were thought to be reachable, F would be too. But F isn't. > But when delet

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: > We can have finalizer code running during delete_garbage(). That > code should not have access to non-valid objects. Weakrefs seem be > a way to violate that. handle_weakrefs() take care of some of them > but it seems there are other issues. I see that

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: > Fleshing out something I left implicit: if there's a trash object T > with a finalizer but we don't KNOW it's trash, we won't force-run its > finalizer before delete_garbage starts either. Then, really the same > thing: we may tp_clear some piece of trash

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: Since W is in the unreachable set, we should not be executing its callback. Would the attached rough patch (gc_disable_wr_callback.txt) be a possible fix? When we find W inside handle_weakrefs(), we mark it as trash and will not execute the callback.

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: I hacked up my copy of CPython to add flags to PyObject to see the GC state of objects. That makes it easier to know if an object is in the 'unreachable' list or not. > We must know that F is trash, else we never would have called tp_clear(F). Yup. In

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: Fleshing out something I left implicit: if there's a trash object T with a finalizer but we don't KNOW it's trash, we won't force-run its finalizer before delete_garbage starts either. Then, really the same thing: we may tp_clear some piece of trash T's

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: Sorry, this is very hard for me - broke an arm near the shoulder on Tuesday, and between bouts of pain and lack of good sleep, concentration is nearly impossible. Typing with one hand just makes it worse :-( We must know that F is trash, else we never would

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Neil Schemenauer added the comment: Victor: > I'm surprised that the function would be seen as "unreachable" if > it's reference counter was equal to 135: If all those references are contained within that garbage set, it is possible. Pablo: > For the weakref to b

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: > call_func and remove are part of a reference cycle. A forced garbage > collection breaks the cycle and removes the two objects, but they are > not removed in the expected order: > > * first: call_func > * then: remove > > The c

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Tim Peters added the comment: tp_clear implementations are necessary to reclaim trash cycles. They're always best practice for objects that may be in trash cycles. tuples are just "cute rebels" that way ;-) Best guess is that the (some) extension isn't playing by the rules. A weakref

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > If I understand Victor's test case correctly, the problem is caused if you > have an extension type that implements tp_traverse but not tp_clear and that > there is also a weakref involved in the trash cycle. I'm not sure that "implements tp_traverse but

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > Is the behavior of tp_clear the key to this bug? Once func_clear(my_func) is called, calling my_func() will crash: my_func() is unsuable. Because of a complex dance involving borrowed references, the function is called *after* it's cleared. Pablo's PR

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

plementing tp_clear should have been okay. It would mean that the GC might find garbage reference cycles that it couldn't cleanup. Those would leak memory but would not crash Python. I'm not sure what has changed since to require that tp_clear actually successfully clears the cycle. Tim was

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: I remove the "release blocker" priority since the regression has been fixed in Python 3.8. The main "regression" was the addition of the func_clear() function: I removed it. -- priority: release blocker -> ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

two fixes in the 3.8 branch for FreeIPA: * I removed func_clear() which triggered the bug * I fixed a reference cycle... in the weakref module :-) As Pablo wrote, this issue is still open since the real deepest root issue is not fixed yet. I was busy with random other stuff last weeks, so I didn't

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: There is this PR that avoids a hard crash in the interpreter: https://github.com/python/cpython/pull/15645 -- ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: Łukasz, I haven't been able to verify that the last patches are sufficient to prevent Python from segfaulting. We need to wait until rc1 is out. I need a proper release or RPM build to run the tests on our internal test infrastructure. You have to release

[issue33095] Cross-reference isolated mode from relevant locations

Julien Palard added the comment: New changeset 210dc3bb37ec57a8b8b16ffe0881181138a6e973 by Julien Palard in branch '3.7': [3.7] bpo-33095: Add reference to isolated mode in -m and script option (GH-7764) (GH-16181) https://github.com/python/cpython/commit

[issue33095] Cross-reference isolated mode from relevant locations

miss-islington added the comment: New changeset 07186c3959a61c3d73b5ccae431cdd32afe70e5e by Miss Islington (bot) in branch '3.8': bpo-33095: Add reference to isolated mode in -m and script option (GH-7764) https://github.com/python/cpython/commit/07186c3959a61c3d73b5ccae431cdd32afe70e5e

[issue33095] Cross-reference isolated mode from relevant locations

Change by Julien Palard : -- pull_requests: +15791 pull_request: https://github.com/python/cpython/pull/16181 ___ Python tracker ___

[issue33095] Cross-reference isolated mode from relevant locations

Change by miss-islington : -- pull_requests: +15790 pull_request: https://github.com/python/cpython/pull/16180 ___ Python tracker ___

[issue38152] AST change introduced tons of reference leaks

STINNER Victor added the comment: commit 0247e80f3c529900689425676342cb70ea31a13d Author: Eddie Elizondo Date: Sat Sep 14 09:38:17 2019 -0400 Fix leaks in Python-ast.c (#16127) -- ___ Python tracker

[issue38152] AST change introduced tons of reference leaks

Change by Ammar Askar : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue38152] AST change introduced tons of reference leaks

Eddie Elizondo added the comment: The PR has been merged so the issue can be closed now -- ___ Python tracker ___ ___

[issue33095] Cross-reference isolated mode from relevant locations

Nick Coghlan added the comment: New changeset bdd6945d4dbd1fe6a7fcff95f7d6908db7d791a1 by Nick Coghlan (Xtreak) in branch 'master': bpo-33095: Add reference to isolated mode in -m and script option (GH-7764) https://github.com/python/cpython/commit/bdd6945d4dbd1fe6a7fcff95f7d6908db7d791a1

[issue38152] AST change introduced tons of reference leaks

Change by Eddie Elizondo : -- keywords: +patch pull_requests: +15738 stage: -> patch review pull_request: https://github.com/python/cpython/pull/16127 ___ Python tracker ___

[issue38152] AST change introduced tons of reference leaks

Eddie Elizondo added the comment: I have a fix for this coming up. -- nosy: +eelizondo ___ Python tracker ___ ___ Python-bugs-list

[issue38152] AST change introduced tons of reference leaks

Change by Dino Viehland : -- assignee: -> dino.viehland nosy: +dino.viehland ___ Python tracker ___ ___ Python-bugs-list mailing

[issue38151] test_types: test_duck_gen() leaks reference

STINNER Victor added the comment: I opened bpo-38152 since much more tests are affected, not only test_types. -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> AST change introduced tons of ref

[issue38152] AST change introduced tons of reference leaks

New submission from STINNER Victor : The following change introduced tons of reference leaks: commit ac46eb4ad6662cf6d771b20d8963658b2186c48c Author: Dino Viehland Date: Wed Sep 11 10:16:34 2019 -0700 bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957) Summary

[issue38151] test_types: test_duck_gen() leaks reference

: vstinner priority: normal severity: normal status: open title: test_types: test_duck_gen() leaks reference versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue38

[issue16690] Reference leak with custom tp_dealloc in PyType_FromSpec

Petr Viktorin added the comment: This is fixed in Python 3.5+. -- resolution: -> fixed stage: -> resolved status: open -> closed ___ Python tracker ___

[issue36270] DOC: Add link to sys.exc_info for "Reference Manual"

Change by Julien Palard : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue36270] DOC: Add link to sys.exc_info for "Reference Manual"

miss-islington added the comment: New changeset f79a022d762edc749d0fecdc50c567d2bb910c53 by Miss Islington (bot) in branch '3.8': bpo-36270: Doc: add link to traceback object reference (GH-13119) https://github.com/python/cpython/commit/f79a022d762edc749d0fecdc50c567d2bb910c53

[issue36270] DOC: Add link to sys.exc_info for "Reference Manual"

Change by miss-islington : -- pull_requests: +15627 pull_request: https://github.com/python/cpython/pull/16002 ___ Python tracker ___

[issue36270] DOC: Add link to sys.exc_info for "Reference Manual"

Julien Palard added the comment: New changeset 9936371af298d465095ae70bc9c2943b4b16eac4 by Julien Palard (Björn Meier) in branch 'master': bpo-36270: Doc: add link to traceback object reference (GH-13119) https://github.com/python/cpython/commit/9936371af298d465095ae70bc9c2943b4b16eac4

[issue25753] Reference leaks in test_smtplib

Zackery Spytz added the comment: I cannot reproduce these leaks. I think this issue should be closed. -- nosy: +ZackerySpytz ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: I reverted the change which added func_clear() to get more time to investigate this bug: New changeset ccaea525885e41c5f1e566bb68698847faaa82ca by T. Wouters (Victor Stinner) in branch '3.8': Revert "bpo-33418: Add tp_clear for function object (GH-8058)"

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: New changeset 23669330b7d0d5ad1a9aac40315ba4c2e765f9dd by Victor Stinner in branch '3.7': bpo-38006: Avoid closure in weakref.WeakValueDictionary (GH-15641) (GH-15789) https://github.com/python/cpython/commit/23669330b7d0d5ad1a9aac40315ba4c2e765f9dd

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

miss-islington added the comment: New changeset 78d15faf6c522619098e94be3e7f6d88a9e61123 by Miss Islington (bot) in branch '3.8': bpo-38006: Avoid closure in weakref.WeakValueDictionary (GH-15641) https://github.com/python/cpython/commit/78d15faf6c522619098e94be3e7f6d88a9e61123 --

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by STINNER Victor : -- pull_requests: +15442 pull_request: https://github.com/python/cpython/pull/15789 ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by miss-islington : -- pull_requests: +15440 pull_request: https://github.com/python/cpython/pull/15787 ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: New changeset a2af05a0d3f0da06b8d432f52efa3ecf29038532 by Victor Stinner in branch 'master': bpo-38006: Avoid closure in weakref.WeakValueDictionary (GH-15641) https://github.com/python/cpython/commit/a2af05a0d3f0da06b8d432f52efa3ecf29038532 --

[issue38037] reference counter issue in signal module

STINNER Victor added the comment: Thanks for the fix animalize. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue38037] reference counter issue in signal module

miss-islington added the comment: New changeset b150d0bf1bb4c3203bb3293625e32aed01b25887 by Miss Islington (bot) in branch '3.8': bpo-38037: Fix reference counters in signal module (GH-15753) https://github.com/python/cpython/commit/b150d0bf1bb4c3203bb3293625e32aed01b25887 -- nosy

[issue38037] reference counter issue in signal module

Change by miss-islington : -- pull_requests: +15432 pull_request: https://github.com/python/cpython/pull/15779 ___ Python tracker ___

[issue38037] reference counter issue in signal module

STINNER Victor added the comment: New changeset 77643c486fd22d8030e0d82c13012684b4ab6df1 by Victor Stinner (animalize) in branch 'master': bpo-38037: Fix reference counters in signal module (GH-15753) https://github.com/python/cpython/commit/77643c486fd22d8030e0d82c13012684b4ab6df1

[issue38037] reference counter issue in signal module

Change by Ma Lin : -- pull_requests: +15407 pull_request: https://github.com/python/cpython/pull/15753 ___ Python tracker ___ ___

[issue38037] reference counter issue in signal module

Change by Ma Lin : -- title: Assertion failed: object has negative ref count -> reference counter issue in signal module ___ Python tracker <https://bugs.python.org/issu

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by Inada Naoki : -- nosy: +inada.naoki ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

te_garbage() in handle_weakrefs(): "Clear weakrefs and invoke callbacks as necessary". It seems like reproducer.tar.gz and gc_crash.py crashes involve a reference cycle. reproducer.tar.gz uses a weak reference which is *not* part of the reference cycle. -- __

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: gc_crash.py: * PyObject_GC_UnTrack() + PyObject_GC_Track() is used to order objects in the GC collection 0, the objects which are involved in the reference cycle. * BadGC2Type type is implemented in C, uses Py_TPFLAGS_HAVE_GC, implements tp_traverse

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by STINNER Victor : Added file: https://bugs.python.org/file48588/gc_crash.patch ___ Python tracker ___ ___ Python-bugs-list mailing

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: Aha! I reproduced the crash with attached gc_crash.py and gc_crash.patch: # cd /path/to/python/source $ git apply gc_crash.patch $ make $ ./python gc_crash.py Segmentation fault (core dumped) -- Added file:

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

== collect == CallFunc .remove at 0x7fab8a189eb0> call func from CallFunc.__del__ func_clear remove() -- in delete_garbage? 1 == exit == -- call_func and remove are part of a reference cycle. A forced garbage collection breaks the cycle and removes the two objects, but t

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: I failed to write a reproducer from scratch. So let me share my notes here. The first point is that remove() function of WeakValueDictionary keeps WeakValueDictionary.data alive like that: --- class NoisyDel: def __del__(self): print("dealloc

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: Attached crash.tar.gz: minimum reproducer which doesn't depend on FreeIPA anymore. Depends on: * cffi (_cffi_backend dynamic library) * pycparser * Python 3.8 (need "python3.8") To reproduce the crash: $ tar -xf reproducer.tar.gz $ cd reproducer/ $

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: Pablo, Victor, could you please assist Armin on the CFFI issue? As usual the situation is more complex than I anticipated. -- ___ Python tracker

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: Thanks Victor! I have opened a bug for CFFI, https://bitbucket.org/cffi/cffi/issues/416/python-38-segfault-cfield_type-does-not -- ___ Python tracker

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > * A and CB are seen as unreachable by the GC Oh, that's not correct: * Only CB is seen as unreachable by the GC * The GC "clears" CB which makes CB inconsistent (tp_clear) * A is deleted indirectly * Deleting A calls CB through the w

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: This bug is quite complex. Let me try to explain it more simply. * Create an object A * Create a weak reference to A with a callback CB * A and CB are part of a reference cycle * Removing the last references to A and CB are not enough to destroy them

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

function if it is indirectly used as a callback for a weak reference to a object which doesn't implement tp_traverse properly (like the CFFI type). The best that we can do is to detect the bug in C extension and emit a warning explaining the bug and suggesting a w

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > Another issue is that the GC doesn't prevent the crash. Would it be possible > to prevent the crash without changing the behavior (ex: still call weakref > callbacks)? Pablo opened bpo-38009 with a PR. I'm not sure if his PR is correct or not yet.

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: For the record, here is an annotated traceback of a FreeIPA crash. PyGC_list_contains() is a hack that I wrote to check if an object was in the unreachable argument of delete_garbage(). (gdb) where #0 0x00434b78 in _PyFunction_Vectorcall

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

has been cleared (by func_clear), but there was still a weak reference using this function as its callback. * Note: the function is called *during* it's being cleared by func_clear(). * The GC has a workaround for weak references part of "unreachable" objects, but its handle_weakrefs(

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by Pablo Galindo Salgado : -- Removed message: https://bugs.python.org/msg351022 ___ Python tracker ___ ___ Python-bugs-list

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: Wait, I just checked and the call is done manually: static void ctypedescr_dealloc(CTypeDescrObject *ct) { PyObject_GC_UnTrack(ct); if (ct->ct_weakreflist != NULL) PyObject_ClearWeakRefs((PyObject *) ct); I am not sure call to

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: For the weakref to be handled correctly the ctypedescr needs to be identified correctly as part of the isolated cycle. If is not in the isolated cycle something may be missing tp_traverse or the GC flags. Can you check if the ctypedescr is part of

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: The biggest CFFI based dependency of FreeIPA is cryptography, but cryptography does not use weakref in its sources. CFFI on the other hand has a WeakValueDictionary object attached to its internal typecache backend,

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: Unless we are missing something I think this may be caused by something in cffi that is not implementing gc-related functions correctly, as PyObject_ClearWeakRefs should not be called from a gc run. Given how complicated this is and even if the

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: I think the problem is that whatever is weak-referenced by the weak ref (CField_Type or similar) is not on the gc list. Because is not on the gc list, handle_weakrefs (https://github.com/python/cpython/blob/master/Modules/gcmodule.c#L1090) is not

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Petr Viktorin added the comment: The traceback does have some ctypedescr_dealloc and cfield_dealloc from _cffi. Could you check what those are? -- ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: It's going to be hard to figure that out. FreeIPA uses a ton of C extensions. Some are hand-written, some uses Cython, ctypes, and cffi. -- ___ Python tracker

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: The more I think about this the more I think there is something else at play. If the GC is able to follow the dependency chain, all weakrefs should have been already managed correctly and PyObject_ClearWeakRefs should have never be invoked. I think

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Petr Viktorin added the comment: > What downsides do we see raising an exception? Yeah, on second thought, that would probably be best. We still want PR #15641 as well, so the exception doesn't actually happen in practice. Note that _PyEval_EvalCodeWithName (the main use of func_globals)

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: In https://bugs.python.org/issue33418 I proposed reverting tp_clear on function objects. >What should be done when a function with func_code=NULL is called? We can set the error indicator at least. Although I agree that it seems suboptimal. At least

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

-8058) Without tp_clear, GC can't break cyclic reference. It will cause memory leak when cyclic reference is created intentionally. -- ___ Python tracker <https://bugs.python.org/issue38

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: I'm now able to reproduce the FreeIPA crash. In short: * Get a Fedora Rawhide VM (to get Python 3.8 as "python3", it's more convenient) * Install FreeIPA (dnf install freeipa-server) * Run: ipa-server-install --help * Python does crash at exit > In

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Petr Viktorin added the comment: I'm not sure adding a check would solve this. What should be done when a function with func_code=NULL is called? "Silently do nothing" is not really an option; raising an exception wouldn't help much in this case. I wonder if a function's tp_clear we should

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: See also https://bugs.python.org/issue33418 as the potential source of the problem. -- ___ Python tracker ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Pablo Galindo Salgado added the comment: In particular, this was not happening before because the function type did not implement tp_clear: https://github.com/python/cpython/blob/3.7/Objects/funcobject.c#L615 The new implementation of tp_clear without checks is allowing this to happen.

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

that the fields that are needed are valid. From the docs on tp_clear: > The Py_CLEAR() macro should be used, because clearing references is delicate: > the reference to the contained object must not be decremented until after the > pointer to the contained object is set to NULL. This is because dec

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by Pablo Galindo Salgado : -- nosy: +pablogsal, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Christian Heimes added the comment: Your workaround solves the segfault issue for me. -- ___ Python tracker ___ ___

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

STINNER Victor added the comment: > I don't understand why/how remove() gets a closure. Ok, I found the reason and I wrote PR 15641 to fix it. Previous fix of WeakValueDictionary remove() function: commit 9cd7e17640a49635d1c1f8c2989578a8fc2c1de6 Author: Łukasz Langa Date: Fri Feb 10

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by STINNER Victor : -- keywords: +patch pull_requests: +15307 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/15641 ___ Python tracker

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

re) is cleared, and there was a weak reference to this cffi object with a callback... which is the cleared function (Python object 0x7fffe6ebfd30)... -- I don't understand why/how remove() gets a closure. I modified weakref.py to ensure that remove.__closur

[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

Change by STINNER Victor : -- title: _PyFunction_Vectorcall() can segfault on process exit -> Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit ___ Python tracker <https://bugs.python.org/issu

[issue37955] mock.patch incorrect reference to Mock

Change by Paulo Henrique Silva : -- keywords: +patch pull_requests: +15204 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15521 ___ Python tracker

[issue37955] mock.patch incorrect reference to Mock

instead of Mock (even tough MagickMock inherits from it). -- assignee: docs@python components: Documentation messages: 350537 nosy: docs@python, phsilva priority: normal severity: normal status: open title: mock.patch incorrect reference to Mock versions: Python 3.9

<    4   5   6   7   8   9   10   11   12   13   >