[issue16089] _elementtree causes segfault in GC

[Stefan Krah]

Stefan Krah added the comment:

Nice find. -- The Python version does this:

  _Element = _ElementInterface = Element


So (naively) I would think the same should be done for the C version
after importing Element.


But then one runs into the object layouts conflict that you mentioned.

On the other hand, in the original documentation direct use of
_ElementInterface was discouraged:

http://effbot.org/zone/pythondoc-elementtree-ElementTree.htm#elementtree.ElementTree._ElementInterface-class

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Eli Bendersky]

Eli Bendersky added the comment:

Thank you, Antoine, for looking into this. I wish I could participate in a 
meaningful way, but alas it will be days or weeks before I can recreate a 
suitable setup to get back hacking on Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Roundup Robot]

Roundup Robot added the comment:

New changeset f9224f23f473 by Antoine Pitrou in branch '3.3':
Sanitize and modernize some of the _elementtree code (see issue #16089).
http://hg.python.org/cpython/rev/f9224f23f473

New changeset 9fb0a8fc5d79 by Antoine Pitrou in branch 'default':
Sanitize and modernize some of the _elementtree code (see issue #16089).
http://hg.python.org/cpython/rev/9fb0a8fc5d79

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'll still commit my cleanup patch in the meantime :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

Example of this is the following code in treebuilder_handle_start:

if (this != Py_None) {
if (element_add_subelement((ElementObject*) this, node) < 0)
goto error;

(note the overly optimistic cast)

but this is really a pervasive problem, since in many places TreeBuilder is 
hard-wired to use a Element instance and nothing else (despite the 
element_factory).

Note that simpleTAL cannot use the _elementtree.Element class, since their 
subclass also inherits from an exception subclass, and the object layouts 
conflict with each other (yeah, crappy design).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

Ok, the problem is that _elementtree.TreeBuilder expects to receive 
_elementtree.Element instances, but simpleTAL's element_factory instead gives 
_ElementInterface instances.

In other words, TreeBuilder is completely broken.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

By the way, the crash involves an _ElementInterface subclass named 
SimpleElementTreeVar:

#0  0x00524c0f in visit_decref (op=Traceback (most recent call last):
  File "/home/antoine/cpython/33/python-gdb.py", line 1298, in to_string
pyop = PyObjectPtr.from_pyobject_ptr(self.gdbval)
  File "/home/antoine/cpython/33/python-gdb.py", line 370, in from_pyobject_ptr
cls = cls.subclass_from_type(p.type())
  File "/home/antoine/cpython/33/python-gdb.py", line 318, in subclass_from_type
tp_name = t.field('tp_name').string()
  File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x95 in position 1: invalid 
start byte
, data=0x0) at Modules/gcmodule.c:361
#1  0x005bcbc5 in BaseException_traverse (self=0x7f08f390a8b0, 
visit=0x524b98 , arg=0x0)
at Objects/exceptions.c:104
#2  0x004336d9 in subtype_traverse (self=
, 
visit=0x524b98 , arg=0x0) at Objects/typeobject.c:837
#3  0x00524cba in subtract_refs (containers=0x8dcf40) at 
Modules/gcmodule.c:386
#4  0x00525afa in collect (generation=0, n_collected=0x7fffed18be00, 
n_uncollectable=0x7fffed18bdf8)
at Modules/gcmodule.c:891
#5  0x005260b2 in collect_with_callback (generation=0) at 
Modules/gcmodule.c:1048
#6  0x0052615c in collect_generations () at Modules/gcmodule.c:1071
#7  0x0052707d in _PyObject_GC_Malloc (basicsize=48) at 
Modules/gcmodule.c:1580
[snip]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

More assorted celementtree cleanups.

--
Added file: http://bugs.python.org/file27378/ctree2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Antoine Pitrou]

Antoine Pitrou added the comment:

Here is a collection of assorted small fixes for celementtree. There are 
probably other problems lurking (the coding style there is quite old).

I cannot say anything about the crasher until there's a simple reproducer :)

--
nosy: +pitrou
Added file: http://bugs.python.org/file27377/ctree.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Georg Brandl]

Georg Brandl added the comment:

Let's make sure this gets into 3.3.1.

--
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Stefan Krah]

Stefan Krah added the comment:

Do note that the patch is somewhat cargo-cult. I don't understand yet why
it works; it may very well just silence the real problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Eli Bendersky]

Eli Bendersky added the comment:

I'm currently somewhat "offline" for a while (cross-continental move), but I'll 
do my best to try to recreate my setup to test this problem and the proposed 
solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Christian Heimes]

Christian Heimes added the comment:

I can confirm that Stefan's fix works. I ran the SimpleTAL test suite about 
hundred times in a loop without a segfault. But I don't understand why the 
change fixes the issue. Could the alteration just lead to another code path so 
the erroneous code isn't triggered?

+1 for Py_CLEAR()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Georg Brandl]

Changes by Georg Brandl :


--
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Stefan Krah]

Stefan Krah added the comment:

I'm now able to reproduce the issue with a non-debug build. As
Christian says, using Py_CLEAR() does not help (though I wonder
if it shouldn't be used anyway).

Reverting part of 20b8f0ee3d64 "fixes" the segfault, see the
attached diff. I don't know the code well enough to say if this
is a valid possibility.

--
keywords: +patch
Added file: http://bugs.python.org/file27355/issue16089-pseudo-fix.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Christian Heimes]

Christian Heimes added the comment:

No, it doesn't make a difference when I replace Py_XDECREF() with Py_CLEAR(). 
I've also replaced Py_(X)DECREF() in the other *_gc_clear() methods without 
success.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Stefan Krah]

Stefan Krah added the comment:

I can't reproduce this, so just a wild guess: Does the segfault still
happen if you replace Py_XDECREF() with Py_CLEAR() in xmlparser_gc_clear()?

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Florent Xicluna]

Changes by Florent Xicluna :


--
components: +XML
nosy: +eli.bendersky, flox

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Christian Heimes]

Christian Heimes added the comment:

The issue could be related to #14007.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16089] _elementtree causes segfault in GC

[Christian Heimes]

New submission from Christian Heimes:

The issue was reported by Arfrever on #python-dev. The test suite of simpletal 
[1] was segfaulting with Python 3.3. It doesn't segfault without the 
_elementtree C extension.

I'm able to reproduce the issue. It may take a couple of runs with a debug 
build of Python but eventually Python segfaults. So far segfaults occur inside 
a GC collection run. Benjamin suspects the issue is related to the new GC code 
in _elementtree.

#0  visit_decref (op=, data=0x0) at 
Modules/gcmodule.c:361
#1  0x004bccd5 in subtract_refs (containers=) at 
Modules/gcmodule.c:386
#2  collect (generation=0, n_collected=0x7fffe3eedb70, 
n_uncollectable=0x7fffe3eedb78) at Modules/gcmodule.c:891
#3  0x004bd7e6 in collect_with_callback (generation=0) at 
Modules/gcmodule.c:1048
#4  collect_generations () at Modules/gcmodule.c:1071
#5  _PyObject_GC_Malloc (basicsize=) at 
Modules/gcmodule.c:1580
#6  0x004bddcc in _PyObject_GC_Malloc (basicsize=) 
at Modules/gcmodule.c:1567
#7  _PyObject_GC_New (tp=0x8322a0) at Modules/gcmodule.c:1590
#8  0x00548a3a in new_dict (values=0x0, keys=0xd02e60) at 
Objects/dictobject.c:395
#9  _PyDict_NewPresized (minused=) at 
Objects/dictobject.c:1044
#10 0x0047a449 in PyEval_EvalFrameEx (f=, 
throwflag=) at Python/ceval.c:2245
#11 0x00480b03 in fast_function (nk=, na=1, 
n=, pp_stack=0x7fffe3eede40, func=
) at Python/ceval.c:4150


[1] http://www.owlfish.com/software/simpleTAL/downloads/SimpleTAL-5.1.tar.gz

--
components: Extension Modules
keywords: 3.3regression
messages: 171603
nosy: christian.heimes
priority: critical
severity: normal
stage: needs patch
status: open
title: _elementtree causes segfault in GC
type: crash
versions: Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com