[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: The bug is now fixed in Python 3.6 and master. Thanks for the bug report and analysis, Stefan Behnel! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: New changeset 8afd7ab12d7f8915b549cf04af384b495ec73d22 by Victor Stinner (Miss Islington (bot)) in branch '3.6': [3.6] bpo-31499, xml.etree: Fix xmlparser_gc_clear() crash (GH-3641) (#3645)

[issue31499] ElementTree crash while cleaning up ParseError

Changes by Roundup Robot : -- pull_requests: +3639 ___ Python tracker ___

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: New changeset e727d41ffcd91b21ce82026ec8c8381d34a16209 by Victor Stinner in branch 'master': bpo-31499, xml.etree: Fix xmlparser_gc_clear() crash (#3641) https://github.com/python/cpython/commit/e727d41ffcd91b21ce82026ec8c8381d34a16209 --

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: test_issue31499.diff: Oh great, it works (to reproduce the crash). I modified your test and included it in my PR, I added you as a co-author of my PR ;-) -- ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

Changes by Serhiy Storchaka : Added file: https://bugs.python.org/file47146/test_issue31499.diff ___ Python tracker ___

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: Serhiy Storchaka: I tried your pattern, but failed to write a reliable unit test. Can you please write a full patch / test? -- ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

Serhiy Storchaka added the comment: def test_issue31499(self): def test(): ... test() test.support.gc_collect() -- ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: Attached PR 3641 fixes bug1.py and bug2.py crashes. Sadly, I failed to write a reliable unit test using bug2.py. The bug requires to trigger the garbage collector in a specific order which depends on how frames are tracked by the GC... -- versions:

[issue31499] ElementTree crash while cleaning up ParseError

Changes by STINNER Victor : -- keywords: +patch pull_requests: +3636 stage: -> patch review ___ Python tracker ___

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: >> The question is more why/how the code didn't crash before? :-) > > Typical case of a Schroedinbug. I don't believe in the chaos :-) I ran a "git bisect" since January 1st, 2017. Attached bug2.py started to crash since the following commit related to

[issue31499] ElementTree crash while cleaning up ParseError

Stefan Behnel added the comment: > The question is more why/how the code didn't crash before? :-) Typical case of a Schroedinbug. -- ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: Python 2.7 is not affected because it doesn't implement tp_clear (it doesn't have xmlparser_gc_clear()), only xmlparser_dealloc() calls EXPAT(ParserFree)(self->parser). I'm unable to reproduce the bug in Python 3.5 nor 3.6. bug2.py creates a reference cycle

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: The bug is at this line: Breakpoint 6, xmlparser_gc_clear (self=0x77e28c08) at /home/haypo/prog/python/master/Modules/_elementtree.c:3414 static int xmlparser_gc_clear(XMLParserObject *self) { EXPAT(ParserFree)(self->parser); // <--- HERE ... }

[issue31499] ElementTree crash while cleaning up ParseError

Stefan Behnel added the comment: Thanks for confirming, Victor. I hadn't realised that the first update of expat was already back in June. That means it's not ruled out yet as a source of this crash. Bisecting is probably a good idea. -- ___ Python

[issue31499] ElementTree crash while cleaning up ParseError

Changes by STINNER Victor : Added file: https://bugs.python.org/file47145/bug2.py ___ Python tracker ___

[issue31499] ElementTree crash while cleaning up ParseError

STINNER Victor added the comment: I confirm the crash using attached bug1.py (first example, completed) and bug2.py (second example). -- Added file: https://bugs.python.org/file47144/bug1.py ___ Python tracker

[issue31499] ElementTree crash while cleaning up ParseError

Stefan Behnel added the comment: Minimal reproducer seems to be this: -- import xml.etree.ElementTree as etree def test(): parser = etree.XMLParser() try: parser.close() except etree.ParseError as exc: e = exc # must keep local reference! test() --