[issue33930] Segfault with deep recursion into object().__dir__

miss-islington added the comment: New changeset ef36dfe4de1ee0df11cde94fd76465336aa8141d by Benjamin Peterson in branch '3.10': [3.10] bpo-33930: Fix typo in the test name. (GH-27736) https://github.com/python/cpython/commit/ef36dfe4de1ee0df11cde94fd76465336aa8141d -- _

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: Since the refleak investigation moved to its own issue, I'm re-closing this one for clarity. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue33930] Segfault with deep recursion into object().__dir__

miss-islington added the comment: New changeset f7635f0e542c916bfd62542e7b60ee23ff681d0d by Miss Islington (bot) in branch '3.9': [3.9] bpo-33930: Fix typo in the test name. (GH-27733) (GH-27734) https://github.com/python/cpython/commit/f7635f0e542c916bfd62542e7b60ee23ff681d0d -- _

[issue33930] Segfault with deep recursion into object().__dir__

Benjamin Peterson added the comment: New changeset d7f5796a1ec7ba223f6a844d7580559abef05238 by Miss Islington (bot) in branch '3.8': bpo-33930: Fix typo in the test name. (GH-27735) https://github.com/python/cpython/commit/d7f5796a1ec7ba223f6a844d7580559abef05238 -- ___

[issue33930] Segfault with deep recursion into object().__dir__

Change by Benjamin Peterson : -- pull_requests: +26216 pull_request: https://github.com/python/cpython/pull/27736 ___ Python tracker ___ ___

[issue33930] Segfault with deep recursion into object().__dir__

Change by miss-islington : -- pull_requests: +26214 pull_request: https://github.com/python/cpython/pull/27734 ___ Python tracker ___ __

[issue33930] Segfault with deep recursion into object().__dir__

Change by miss-islington : -- pull_requests: +26215 pull_request: https://github.com/python/cpython/pull/27735 ___ Python tracker ___ __

[issue33930] Segfault with deep recursion into object().__dir__

Benjamin Peterson added the comment: New changeset f08e6d1bb3c5655f184af88c6793e90908bb6338 by Benjamin Peterson in branch 'main': bpo-33930: Fix typo in the test name. (#27733) https://github.com/python/cpython/commit/f08e6d1bb3c5655f184af88c6793e90908bb6338 -- ___

[issue33930] Segfault with deep recursion into object().__dir__

Change by Benjamin Peterson : -- nosy: +benjamin.peterson nosy_count: 14.0 -> 15.0 pull_requests: +26213 pull_request: https://github.com/python/cpython/pull/27733 ___ Python tracker _

[issue33930] Segfault with deep recursion into object().__dir__

Irit Katriel added the comment: I've narrowed it down a bit more and created issue44895. -- ___ Python tracker ___ ___ Python-bugs-

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: Entirely possible, Irit. We've had just this sort of case just last week! See: https://bugs.python.org/issue44524#msg399157 -- ___ Python tracker _

[issue33930] Segfault with deep recursion into object().__dir__

Irit Katriel added the comment: It seems to have been introduced by https://github.com/python/cpython/pull/27626 I don't see how the code change could have introduced a leak. Maybe the new tests exposed a pre-existing leak? -- ___ Python tracker

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: (BTW, with this command, it is indeed possible to trigger the ref leak on `main` as well, which makes it somewhat less scary.) -- ___ Python tracker __

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: I gotta run for the day. The shortest way for me to reproduce is to focus on the tests that were added in 3.10 and shortening the repetitions a little: $ time ./python.exe -E -Wd -m test test_exceptions -R3:3 -m 'test_attribute*' -m 'test_getattr*' -m 'test_lin

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: Interestingly, I can't reproduce the refleaks on `main` either, only on 3.10. -- ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: Apparently we've got ourselves refleaks with the patches. It doesn't always reproduce but put test_exceptions -R: in a loop in 3.10 and soon enough you'll get: 0:00:00 load avg: 4.47 Run tests sequentially 0:00:00 load avg: 4.47 [1/1] test_exceptions beginning

[issue33930] Segfault with deep recursion into object().__dir__

Łukasz Langa added the comment: New changeset 09d814b3833c2c852f4f233297b4561028782d2a by Łukasz Langa in branch '3.8': [3.8] bpo-33930: Fix segfault with deep recursion when cleaning method objects (GH-27678) (GH-27721) https://github.com/python/cpython/commit/09d814b3833c2c852f4f233297b4561

[issue33930] Segfault with deep recursion into object().__dir__

Change by Łukasz Langa : -- nosy: +lukasz.langa nosy_count: 13.0 -> 14.0 pull_requests: +26204 pull_request: https://github.com/python/cpython/pull/27721 ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: Thanks for the fix Augusto Goulart and Pablo! I close the issue. Pablo asked in PR 27678 if a Python 3.8 Release Manager (Lukasz) wants a backport to 3.8. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed __

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: New changeset d6d2d549972422ccc6fa335ebf9907e4b3d71817 by Miss Islington (bot) in branch '3.10': bpo-33930: Fix segfault with deep recursion when cleaning method objects (GH-27678) (GH-27719) https://github.com/python/cpython/commit/d6d2d549972422ccc6fa335eb

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: New changeset 15f0a45b2822607f2c38685a72d06f050e24b215 by Miss Islington (bot) in branch '3.9': bpo-33930: Fix segfault with deep recursion when cleaning method objects (GH-27678) (GH-27720) https://github.com/python/cpython/commit/15f0a45b2822607f2c38685a72

[issue33930] Segfault with deep recursion into object().__dir__

Pablo Galindo Salgado added the comment: New changeset bfc2d5a5c4550ab3a2fadeb9459b4bd948ff61a2 by Pablo Galindo Salgado in branch 'main': bpo-33930: Fix segfault with deep recursion when cleaning method objects (GH-27678) https://github.com/python/cpython/commit/bfc2d5a5c4550ab3a2fadeb9459b

[issue33930] Segfault with deep recursion into object().__dir__

Change by miss-islington : -- pull_requests: +26203 pull_request: https://github.com/python/cpython/pull/27720 ___ Python tracker ___ __

[issue33930] Segfault with deep recursion into object().__dir__

Change by miss-islington : -- nosy: +miss-islington nosy_count: 12.0 -> 13.0 pull_requests: +26202 pull_request: https://github.com/python/cpython/pull/27719 ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

Neil Schemenauer added the comment: I'm thinking that the explicit call to PyObject_GC_UnTrack should be made unnecessary by integrating it into the trashcan code. That way, we avoid someone else running into this kind of bug in the future. See bpo-44881. -- nosy: +nascheme __

[issue33930] Segfault with deep recursion into object().__dir__

Change by Pablo Galindo Salgado : -- pull_requests: +26165 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27678 ___ Python tracker ___ ___

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: Correctness matters more than performance for 1 nanosecond. IMO a fix like attached bpo-33930.patch looks like the good approach to fix the crash. -- ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

Irit Katriel added the comment: I've reproduced the segfault on 3.11 on a Mac. -- nosy: +iritkatriel versions: +Python 3.11 -Python 3.6 ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

INADA Naoki added the comment: $ ./python.patched -m perf timeit --compare-to ./python.master -s 'o="foo"' 'o.encode' --duplicate=1 python.master: . 23.1 ns +- 0.5 ns python.patched: . 24.6 ns +- 0.5 ns Mean +- std dev: [python.master] 23.1 ns +- 0.

[issue33930] Segfault with deep recursion into object().__dir__

Serhiy Storchaka added the comment: Thank you Inada. There is a large overhead for iterating. Try to add --duplicate=1000. -- ___ Python tracker ___ _

[issue33930] Segfault with deep recursion into object().__dir__

INADA Naoki added the comment: microbench: $ ./python.patched -m perf timeit --compare-to ./python.master -s 'o="foo"' 'o.encode' python.master: . 29.3 ns +- 0.6 ns python.patched: . 30.9 ns +- 1.3 ns Mean +- std dev: [python.master] 29.3 ns +- 0.6 ns

[issue33930] Segfault with deep recursion into object().__dir__

Pablo Galindo Salgado added the comment: > Can someone please explain me why this change has any impact? Does it mean > that _PyObject_GC_UNTRACK() macro has a bug? Interestingly, this lonely and perplexing comment is left in PyObject_GC_UnTrack (the function): void PyObject_GC_UnTrack(voi

[issue33930] Segfault with deep recursion into object().__dir__

Serhiy Storchaka added the comment: It is hard to get a noticeable difference in macrobenchmarks. I want to see results of microbenchmarks. For tuples, the trashcan mechanism is the main culprit of the performance hit when use short-living tuples for passing arguments. For bound method objec

[issue33930] Segfault with deep recursion into object().__dir__

Ronald Oussoren added the comment: I don’t understand this either. For some reason the patch works when using the function and not when using the macro. I haven’t looked into this yet, and am currently traveling. I should have time to look into this starting December 5th when I’m back home.

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: -_PyObject_GC_UNTRACK(m); +PyObject_GC_UnTrack(m); Can someone please explain me why this change has any impact? Does it mean that _PyObject_GC_UNTRACK() macro has a bug? The function is quite straighforward :-) void PyObject_GC_Track(void *op) {

[issue33930] Segfault with deep recursion into object().__dir__

Gus Goulart added the comment: After a couple of rounds, this is the benchmark result I got for the attached patch. It seems to be a simple, non-performance degrading fix. python -m perf compare_to origin.json patch.json -G --min-speed=2 Slower (1): - sqlalchemy_declarative: 213 ms +- 4 ms -

[issue33930] Segfault with deep recursion into object().__dir__

Gus Goulart added the comment: Sorry, I forgot to include the trashcan API. I'll double check and update the issue. -- ___ Python tracker ___

[issue33930] Segfault with deep recursion into object().__dir__

Change by Gus Goulart : -- nosy: +taleinat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue33930] Segfault with deep recursion into object().__dir__

Gus Goulart added the comment: ronaldoussoren, I have tested (Debian 9) substituting "_PyObject_GC_UNTRACK(m);" to "PyObject_GC_UnTrack(m);", and although the tests pass, I'm can still see the segfault error. diff --git a/Objects/methodobject.c b/Objects/methodobject.c index cfea8cf410..cb8e

[issue33930] Segfault with deep recursion into object().__dir__

Change by Pablo Galindo Salgado : -- nosy: +pablogsal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:/

[issue33930] Segfault with deep recursion into object().__dir__

Ronald Oussoren added the comment: @victor: thanks for confirming my analysis. The trashcan API does fix this issue, although I did have to change "_PyObject_GC_UNTRACK(m);" to "PyObject_GC_UnTrack(m);" as well. I haven't checked yet why that's necessary. -- _

[issue33930] Segfault with deep recursion into object().__dir__

Alistair Buxton added the comment: I found this while chatting on IRC about the implementation details of objects - specifically, since objects have a dict of attributes, and dict subclasses object, it seems like every dict will have another dict inside it. (Eventually someone explained that

[issue33930] Segfault with deep recursion into object().__dir__

STINNER Victor added the comment: C traceback in gdb: Program received signal SIGSEGV, Segmentation fault. 0x004812a8 in meth_dealloc ( m=) at Objects/methodobject.c:86 86 { (gdb) where #0 0x004812a8 in meth_dealloc ( m=) at Objects/methodobject.c:86 #1 0

[issue33930] Segfault with deep recursion into object().__dir__

Raymond Hettinger added the comment: Alistair, how did you happen upon this case? -- nosy: +rhettinger ___ Python tracker ___ ___ P

[issue33930] Segfault with deep recursion into object().__dir__

Change by STINNER Victor : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.py

[issue33930] Segfault with deep recursion into object().__dir__

Serhiy Storchaka added the comment: The trashcan mechanism is used for slot wrappers. The following code doesn't crash: o = object() for x in range(100): o = o.__hash__ -- ___ Python tracker __

[issue33930] Segfault with deep recursion into object().__dir__

Ronald Oussoren added the comment: The crash is likely caused by recursion during the clean-up of the object().__dir__.__dir__ chain. The trashcan API (see Py_TRASHCAN_SAFE_BEGIN/Py_TRASHCAN_SAFE_END in Include/object.h) can help here. Those do some real work and have a cost, I can't

[issue33930] Segfault with deep recursion into object().__dir__

Terry J. Reedy added the comment: Serhiy, is this one of the cases we decided to leave as won't fix? -- nosy: +serhiy.storchaka, terry.reedy ___ Python tracker ___ ___

[issue33930] Segfault with deep recursion into object().__dir__

New submission from Alistair Buxton : The following small snippet of code will crash 3.6.5 with a segfault. The crash occurs at cleanup, so it won't happen in the interactive interpreter (until you exit). # --- code --- o = object() for x in range(100): o = o.__dir__ print(x, id