[issue27863] multiple issues in _elementtree module

[Eryk Sun]

Change by Eryk Sun :


--
nosy:  -ahmedsayeed1982

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Eryk Sun]

Change by Eryk Sun :


--
Removed message: https://bugs.python.org/msg405690

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Ahmed Sayeed]

Ahmed Sayeed  added the comment:

amd64-linux-siginfo.c: Adjust include order to avoid gnulib error 
http://www.compilatori.com/computers/latest-car-deals/
On Fedora rawhide, after updating to glibc-2.33, I'm seeing the
following build failure:
http://www.acpirateradio.co.uk/travel/good/
  CXXnat/amd64-linux-siginfo.o
In file included from /usr/include/bits/sigstksz.h:24, 
http://www.logoarts.co.uk/services/affordable-printer/ 
 from /usr/include/signal.h:315,
 from ../gnulib/import/signal.h:52, 
http://www.slipstone.co.uk/property/hp-of-cars/ 
 from /ironwood1/sourceware-git/rawhide-gnulib 
http://www.mconstantine.co.uk/category/health/ 
/bld/../../worktree-gnulib/gdbserver/../gdb/nat/amd64-linux-siginfo.c:20: 
http://embermanchester.uk/services/whatsapp-number-change/ 
../gnulib/import/unistd.h:663:3: error: #error "Please include config.h 
first."
  663 |  #error "Please include config.h first." 
http://connstr.net/property/mars-researches/ 
  |   ^

glibc-2.33 has changed signal.h to now include 
http://www.go-mk-websites.co.uk/category/health/  which,
in turn, includes . http://joerg.li/technology/b-class-cars/ For 
a gdb build, this causes the gnulib
version of unistd.h to be pulled in first.  The build failure shown 
https://www.mktrade.fi/
above happens because gnulib's config.h has not been included before
the include of . http://www.jopspeech.com/health/nvidia-rtx-2060/

The fix is simple - we just rearrange the order of the header file
includes to make sure that gdbsupport/ 
http://fishingnewsletters.co.uk/category/health/ commondefs.h is included before
attempting to include signal.h.  Note that gdbsupport/commondefs.h
includes . http://www.wearelondonmade.com/health/check-ups/

Build and regression tested on Fedora 33.  On Fedora rawhide, GDB
builds again. https://waytowhatsnext.com/shopping/xbox-release-date/

gdb/ChangeLog:
glibc-2.33 has changed signal.h to now include  which,
in turn, includes . 
http://www.iu-bloomington.com/computers/invisible-with-vpn/ For a gdb build, 
this causes the gnulib
version of unistd.h to be http://the-hunters.org/category/health/ pulled in 
first.  The build failure shown
above happens because gnulib's https://komiya-dental.com/crypto/new-coins/ 
config.h has not been included before
the include of . http://www-look-4.com/tech/nvidia-and-samsung/

The fix is simple - we just rearrange the order of the header file 
https://www.webb-dev.co.uk/crypto/crypto-for-investing/
includes to make sure that gdbsupport/commondefs.h is included before
attempting to include signal.h.  Note that gdbsupport/commondefs.h
includes .

--
nosy: +ahmedsayeed1982 -eli.bendersky, ericvw, scoder, serhiy.storchaka, 
tehybel, xiang.zhang
versions:  -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies:  -various issues due to misuse of PySlice_GetIndicesEx
resolution:  -> fixed
stage: backport needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 9c2c42c221d7996070c0c0a2a114ab42fe3ddb9d by Serhiy Storchaka in 
branch '2.7':
bpo-27863: Fixed multiple crashes in ElementTree. (#765) (#903) (#963)
https://github.com/python/cpython/commit/9c2c42c221d7996070c0c0a2a114ab42fe3ddb9d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset a6b4e1902250d6f28ca6d083ce1c8d7e9b91974b by Serhiy Storchaka in 
branch '3.6':
bpo-27863: Fixed multiple crashes in ElementTree. (#765) (#903)
https://github.com/python/cpython/commit/a6b4e1902250d6f28ca6d083ce1c8d7e9b91974b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Since it is hard to backport the bugfix to 2.7 without test, issue15083 is a 
dependence.

--
dependencies: +Rewrite ElementTree tests in a cleaner and safer way
stage: needs patch -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset c90ff1b78cb79bc3762184e03fa81f11984aaa45 by Serhiy Storchaka in 
branch '3.5':
bpo-27863: Fixed multiple crashes in ElementTree. (#765) (#904)
https://github.com/python/cpython/commit/c90ff1b78cb79bc3762184e03fa81f11984aaa45


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +804

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +803

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 576def096ec7b64814e038f03290031f172886c3 by Serhiy Storchaka in 
branch 'master':
bpo-27863: Fixed multiple crashes in ElementTree. (#765)
https://github.com/python/cpython/commit/576def096ec7b64814e038f03290031f172886c3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +674

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +various issues due to misuse of PySlice_GetIndicesEx

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Eric N. Vander Weele]

Changes by Eric N. Vander Weele :


--
nosy: +ericvw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you for your report tehybel.

--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[SilentGhost]

Changes by SilentGhost :


--
nosy: +eli.bendersky, scoder
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27863] multiple issues in _elementtree module

[tehybel]

New submission from tehybel:

I'll describe 7 issues in the /Modules/_elementtree.c module here. They
include multiple use-after-frees, type confusions and instances of
out-of-bounds array indexing.



Issue 1: use-after-free in element_get_text

The problematic code looks like this:

LOCAL(PyObject*)
element_get_text(ElementObject* self)
{ 
/* return borrowed reference to text attribute */

PyObject* res = self->text;

if (JOIN_GET(res)) { 
res = JOIN_OBJ(res);
if (PyList_CheckExact(res)) {
res = list_join(res);
if (!res)
return NULL;
self->text = res;
}
}

return res;
}

As we can see, if res is a list, we call list_join with res, which is
self->text. list_join will decrease self->text's reference count. When that
happens, arbitrary python code can run. If that code uses self->text, a
use-after-free occurs.

PoC (Proof-of-Concept segfaulting script):

---

import _elementtree as et

class X(str):
def __del__(self):
print(elem.text)

b = et.TreeBuilder()
b.start("test")
b.data(["", X("")])
b.start("test2")

elem = b.close()
print(elem.text)

---


Issue 2: use-after-free in element_get_tail

The same type of issue also exists in element_get_tail and should be fixed
there, too.



Issue 3: type confusion in elementiter_next

The function elementiter_next iterates over a tree consisting of elements.
Each element has an array of children. 

Before doing any casts, most of the elementtree code is careful to check that
these children are, indeed, elements; that is, that their type is correct. The
problem is that elementiter_next does not validate these child objects before
performing a cast. Here is the relevant line:

elem = (ElementObject *)cur_parent->extra->children[child_index];

If the child is not an element, a type confusion occurs. Here's a PoC:

-

import _elementtree as et

state = {
"tag": "tag",
"_children": [1,2,3],
"attrib": "attr",
"tail": "tail",
"text": "text",
}

e = et.Element("ttt")
e.__setstate__(state)

for x in e.iter():
print(x)

-



Issue 4: array-out-of-bounds indexing in element_subscr

This issue occurs when taking the subscript of an element. This is done using
the element_subscr function. The function calls PySlice_GetIndicesEx:

if (PySlice_GetIndicesEx(item,
self->extra->length,
, , , ) < 0) {
return NULL; 
}

The problem is that to evaluate the indices, PySlice_GetIndicesEx might call
back into python code. That code might cause the element's self->extra->length
to change. If this happens, the variables "start", "stop" and "step" might no
longer be appropriate.

The code then uses these variables for array indexing:

for (cur = start, i = 0; i < slicelen;
 cur += step, i++) {
PyObject* item = self->extra->children[cur]; 
...
}

But this could go out of bounds and interpret arbitrary memory as a PyObject.
Here's a PoC that reproduces this:

---

import _elementtree as et

class X:
def __index__(self):
e[:] = []
return 1

e = et.Element("elem")
for _ in range(10):
e.insert(0, et.Element("child"))

print(e[0:10:X()])

---

Running it results in a segfault:

(gdb) r ./poc14.py
Starting program: /home/xx/Python-3.5.2/python ./poc14.py

Program received signal SIGSEGV, Segmentation fault.
0x0049f933 in PyObject_Repr (v=0x768af058) at Objects/object.c:471
471 if (Py_TYPE(v)->tp_repr == NULL)
(gdb) print *v
$37 = {_ob_next = 0xdbdbdbdbdbdbdbdb, _ob_prev = 0xdbdbdbdbdbdbdbdb, ob_refcnt 
= 0xdbdbdbdbdbdbdbdc,
  ob_type = 0xdbdbdbdbdbdbdbdb}

As we can see, "v" is freed memory with arbitrary contents.



Issue 5: array-out-of-bounds indexing in element_ass_subscr

A separate issue of the same type, also due to a call to PySlice_GetIndicesEx,
exists in element_ass_subscr. Here's a proof-of-concept script for that:

---

import _elementtree as et

class X:
def __index__(self):
e[:] = []
return 1

e = et.Element("elem")
for _ in range(10):
e.insert(0, et.Element("child"))

e[0:10:X()] = []

---

To fix these, I believe we could check whether self->extra->length changed
after calling PySlice_GetIndicesEx, and bail out if so. (You can grep the
codebase for "changed size during iteration" for examples of some similarish
cases.)





Issue 6: use-after-free in treebuilder_handle_start

In the function treebuilder_handle_start we see these lines:

if (treebuilder_set_element_text(self->last, self->data))
return NULL;

Here self->last is the most recent element, and we are setting its text to
self->data. This assignment happens via the function
treebuilder_set_element_text which in turn calls
treebuilder_set_element_text_or_tail. There, if the element self->last is not
an exact instance of Element_Type, we run these