[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: > Fix extended to the other branch (and backported to 3.10), and both parts > backported to 3.9. I don't think it counts as a security issue, so not > backporting to 3.8. Thanks Thomas, I missed this code path! --

[issue44184] crash on windows invoking flake8

miss-islington added the comment: New changeset 298ee657ab8170adf75a186c0414b7ca3baf1991 by Miss Islington (bot) in branch '3.9': bpo-44184: Fix subtype_dealloc() for freed type (GH-26274) https://github.com/python/cpython/commit/298ee657ab8170adf75a186c0414b7ca3baf1991 --

[issue44184] crash on windows invoking flake8

Thomas Wouters added the comment: New changeset 0b4704973dbef712d05bdd62349bb4244f545430 by Miss Islington (bot) in branch '3.9': bpo-44184: Apply GH-26274 to the non-GC-type branch of subtype_dealloc (GH-27165) (GH-27175)

[issue44184] crash on windows invoking flake8

Change by Thomas Wouters : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Thomas Wouters added the comment: Fix extended to the other branch (and backported to 3.10), and both parts backported to 3.9. I don't think it counts as a security issue, so not backporting to 3.8. -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Thomas Wouters added the comment: New changeset 6aa59c68dc7910c0675ad23c1f9d88edfb81dfcb by Miss Islington (bot) in branch '3.10': bpo-44184: Apply GH-26274 to the non-GC-type branch of subtype_dealloc (GH-27165) (GH-27174)

[issue44184] crash on windows invoking flake8

Change by miss-islington : -- pull_requests: +25713 pull_request: https://github.com/python/cpython/pull/27176 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Change by miss-islington : -- pull_requests: +25712 pull_request: https://github.com/python/cpython/pull/27175 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Change by miss-islington : -- pull_requests: +25711 pull_request: https://github.com/python/cpython/pull/27174 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Thomas Wouters added the comment: New changeset 074e7659f208051b6b973f7fdb654dd22b93aaa2 by T. Wouters in branch 'main': bpo-44184: Apply GH-26274 to the non-GC-type branch of subtype_dealloc (GH-27165) https://github.com/python/cpython/commit/074e7659f208051b6b973f7fdb654dd22b93aaa2

[issue44184] crash on windows invoking flake8

Change by Thomas Wouters : -- pull_requests: +25701 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/27165 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Thomas Wouters added the comment: Reopening this issue, as there is another branch (for non-GC heaptypes) earlier in subtype_dealloc that I believe suffers from the same problem. Actually triggering the error in a test has been difficult because as far as I can tell it relies on garbage

[issue44184] crash on windows invoking flake8

Ammar Askar added the comment: Indeed, it's quite a tricky issue so I'm glad it was caught in the beta. Thank you for the report Anthony. Thanks for tracing the root cause and the fix Victor and thank you to everyone who helped debug. -- ___

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: Anthony Sottile: "that version of flake8 uses multiprocessing (even for 1 file) -- would the ast objects be involved in that way? (pyflakes also makes reference cyles to handle "parent" relationships)" I expect flake8 to use the ast somewhere to analyze

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: The issue is fixed by: commit 615069eb08494d089bf24e43547fbc482ed699b8 Author: Victor Stinner Date: Fri May 21 19:19:54 2021 +0200 bpo-44184: Fix subtype_dealloc() for freed type (GH-26274) Fix a crash at Python exit when a deallocator

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: New changeset 50b0d148a68072292832eb69bdf1815b8059355f by Miss Islington (bot) in branch '3.10': bpo-44184: Fix subtype_dealloc() for freed type (GH-26274) (GH-26290) https://github.com/python/cpython/commit/50b0d148a68072292832eb69bdf1815b8059355f

[issue44184] crash on windows invoking flake8

Anthony Sottile added the comment: that version of flake8 uses multiprocessing (even for 1 file) -- would the ast objects be involved in that way? (pyflakes also makes reference cyles to handle "parent" relationships) -- ___ Python tracker

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: Anthony Sottile: I'm surprised that AST nodes survive until the last GC collection. It seems like somehow a reference cycle prevent to delete these nodes, and this reference cycle is kept alive somehow until the last GC collection at Python exit. It would

[issue44184] crash on windows invoking flake8

Change by miss-islington : -- nosy: +miss-islington nosy_count: 12.0 -> 13.0 pull_requests: +24894 pull_request: https://github.com/python/cpython/pull/26290 ___ Python tracker

[issue44184] crash on windows invoking flake8

Erlend E. Aasland added the comment: > Crash related to AST in interpreter_clear() remains me bpo-41796. Well remembered, Victor! Bisecting using Pablo's reproducer: fd957c124c1d9c5eaf61f7af8cf266bafcb1 is the first bad commit commit fd957c124c1d9c5eaf61f7af8cf266bafcb1 Author:

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: Valgrind detects the bug (unmodified Python): $ PYTHONMALLOC=debug valgrind ./python ref.py (...) ==607098== Invalid read of size 8 ==607098==at 0x493FBE: subtype_dealloc (typeobject.c:1456) ==607098==by 0x47C914: _Py_Dealloc (object.c:2288)

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: Ok, I got a crash under the address sanitizer using ref.py: ./python lel.py exit Cycle.__del__ Cycle.__del__ = ==77503==ERROR: AddressSanitizer: heap-use-after-free on address

[issue44184] crash on windows invoking flake8

Change by STINNER Victor : Added file: https://bugs.python.org/file50056/debug_subtype_dealloc.patch ___ Python tracker ___ ___

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: Apply attached debug_subtype_dealloc.patch to reproduce the issue on Linux with attached ref.py script: $ ./python ref.py exit subtype_dealloc(_ABC): call basedealloc() with Py_REFCNT(type)=1 subtype_dealloc(_Precedence): call basedealloc() with

[issue44184] crash on windows invoking flake8

Change by STINNER Victor : -- keywords: +patch pull_requests: +24880 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26274 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: I have a reproducer (.py script) on Windows, but the crash rate is between 1/3 and 1/2. Also, in VS, if I run "import bug" (bug.py) in the REPL, I fail to reproduce the crash. Using "exec(open('../bug.py').read())" is more likely to trigger the crash, but

[issue44184] crash on windows invoking flake8

Anthony Sottile added the comment: would it maybe be helpful to bisect a history where the dataclasses / inspect import change is introduced earlier? this would perhaps help pinpoint the other commit which is causing this? -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Steve Dower added the comment: > I've found that it's reproducible if you run flake8 on a file that just > contains: `a.b` ISTR there were some changes made to assigning attributes on AST classes recently? I forget who did them, but I remember discussing it during the sprints last year.

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: This bug is really hard to reproduce on Windows. It depends on flake8 is run. It's a random crash in the last GC collection at Python exit. Crash related to AST in interpreter_clear() remains me bpo-41796. Extract of the code: static void

[issue44184] crash on windows invoking flake8

Erlend E. Aasland added the comment: FWIW, I'm also unable to reproduce on macOS (so far). -- ___ Python tracker ___ ___

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: I'm trying to reproduce on Linux but I am unable to make it fail even with the address sanitizer activated :( -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: My bet is because this is caused by the AST module using heap typed here. There is probably a missing strong reference somewhere. -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Ammar Askar added the comment: Just to thicken the mystery a bit further, if you comment out import inspect in dataclasses.py, the error also goes away. -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Change by Erlend E. Aasland : -- nosy: +erlendaasland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue44184] crash on windows invoking flake8

Steve Stagg added the comment: So, for me, the single line from 11159d that causes the problem is this: pprint.py: import dataclasses as _dataclasses Obviously importing dataclasses here is having some side-effect -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Steve Stagg added the comment: I got the same result from a bisect: 11159d2c9d6616497ef4cc62953a5c3cc8454afb -- ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Ammar Askar added the comment: The bisect was bizarre to say the least... It points to commit 11159d2c9d6616497ef4cc62953a5c3cc8454afb bpo-43080: pprint for dataclass instances (GH-24389) * Added pprint support for dataclass instances which don't have a custom __repr__. which doesn't touch

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: Also, why this doesn't happen on Linux? -- ___ Python tracker ___ ___ Python-bugs-list

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: Given how hairy finding these GC errors is, I would recommend to bisect this so we can trim down the search to some commit, although my bet is some of the heap type conversions. -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Steve Stagg added the comment: I've found that it's reproducible if you run flake8 on a file that just contains: `a.b` whereas: `a` and `a(b)` don't seem to trigger the error. The object being cleaned up when the segfault appears, seems to be a subclass of ast.AST --

[issue44184] crash on windows invoking flake8

Ammar Askar added the comment: The segfault seems to be occuring on typeobject.c:1456 during interpreter finalization: if (type->tp_flags & Py_TPFLAGS_HEAPTYPE && !(base->tp_flags & Py_TPFLAGS_HEAPTYPE)) where `type` seems to have already been deallocated. Here's the stack trace:

[issue44184] crash on windows invoking flake8

Change by Steve Stagg : -- nosy: +stestagg ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue44184] crash on windows invoking flake8

Shreyan Avigyan added the comment: This is a flake8 error not tox error. Though if it was a complete flake8 error then it would fail in all Python. But it only fails in 3.10 (don't know about 3.11 because haven't tested it yet.) -- ___ Python

[issue44184] crash on windows invoking flake8

Anthony Sottile added the comment: vstinner I showed the directions above, but here they are again: venv\Scripts\pip install flake8==3.6.0 venv\Scripts\pip install -e . --no-deps # ignore the conflict, but fix pyflakes -- ___ Python tracker

[issue44184] crash on windows invoking flake8

STINNER Victor added the comment: Where does flake8.exe come from? Is it created by pip? Or by Python? Is it part of the flake8 project? -- nosy: +vstinner ___ Python tracker

[issue44184] crash on windows invoking flake8

Ammar Askar added the comment: Can recreate on the latest 3.10 checkout, taking a look. -- nosy: +ammar2 ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Anthony Sottile added the comment: ah sorry, the branch got squash-merged this is the equivalent revision after the merge: f3b1b44bf3d2d5927004fa1c2fcf1ab2def816b9 -- ___ Python tracker

[issue44184] crash on windows invoking flake8

Pablo Galindo Salgado added the comment: Is b02ba019e16f7c156ec63c2ea05c627a0fe86c48 the correct commit? I cannot checkout this: ❯ git clone https://github.com/pycqa/pyflakes Cloning into 'pyflakes'... remote: Enumerating objects: 2651, done. remote: Counting objects: 100% (53/53), done.

[issue44184] crash on windows invoking flake8

Change by Pablo Galindo Salgado : -- priority: deferred blocker -> release blocker ___ Python tracker ___ ___ Python-bugs-list

[issue44184] crash on windows invoking flake8

Christian Heimes added the comment: Pablo, Steve, please take a look. -- assignee: -> steve.dower keywords: +3.10regression nosy: +pablogsal priority: normal -> deferred blocker versions: +Python 3.11 ___ Python tracker

[issue44184] crash on windows invoking flake8

Anthony Sottile added the comment: everything in this virtualenv is pure python so I don't think it's a faulty third party extension module -- ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Shreyan Avigyan added the comment: It's reproducible. I reproduced it on my Windows 10 with Python 3.10.0b1 -- ___ Python tracker ___

[issue44184] crash on windows invoking flake8

Christian Heimes added the comment: Anthony, could you please check if any of your dependencies has a native C extension? On Windows they have a .pyd extension. -- nosy: +christian.heimes ___ Python tracker

[issue44184] crash on windows invoking flake8

Shreyan Avigyan added the comment: This looks like a pyflakes error to me. And you've also mentioned you can reproduce it outside tox with pyflakes. -- nosy: +shreyanavigyan title: crash on windows invoking flake8 under tox -> crash on windows invoking flake8 type: -> crash

[issue44184] crash on windows invoking flake8 under tox

Anthony Sottile added the comment: I can reproduce it outside of tox using: venv\Scripts\pip install flake8==3.6.0 venv\Scripts\pip install -e . --no-deps # ignore the conflict, but fix pyflakes C:\Users\asott\AppData\Local\Temp\y\pyflakes>venv\Scripts\flake8.exe setup.py

[issue44184] crash on windows invoking flake8 under tox

Change by Karthikeyan Singaravelan : -- components: +Windows nosy: +paul.moore, steve.dower, tim.golden, zach.ware ___ Python tracker ___

[issue44184] crash on windows invoking flake8 under tox

New submission from Anthony Sottile : I installed python using the installers from python.org -- I originally reproduced this using github actions using pyflakes's testsuite >ver Microsoft Windows [Version 10.0.19041.985] >venv\Scripts\python --version Python 3.10.0b1