[issue25873] Faster ElementTree iterating

[Roundup Robot]

Roundup Robot added the comment:

New changeset 5a5d5268afd5 by Serhiy Storchaka in branch 'default':
Issue #25873: Optimized iterating ElementTree.
https://hg.python.org/cpython/rev/5a5d5268afd5

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you for your review Martin. I have opened a separate issue25902 for 
refcount bugs in iteration. Its patch is purposed for all versions, while this 
issue is for developer version only.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch refactors and optimize the code for iterating ElementTree. 
Refactoring and minor optimization makes the code about 30 lines smaller and 
2-3% faster. Using a continuous array instead of linked list makes the code up 
to 40% faster (lxml benchmark was used to measurement).

In addition the patch fixes possible issue with owning references.

--
components: Extension Modules
files: etree_iter.patch
keywords: patch
messages: 256472
nosy: brett.cannon, scoder, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Faster ElementTree iterating
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file41318/etree_iter.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Martin Panter]

Martin Panter added the comment:

Looks technically correct as far as my knowledge of the malloc routines goes. 
What was the problem with references that you fixed? Maybe with 
parent_stack_push_new() failure?

The main reference counting bug that sticks out to me is with the text and tail 
elements, because element_get_text() etc return a borrowed reference. It might 
be good to fix this while you are at it:

>>> element = Element("tag")
>>> class Text:
... def __bool__(self):
... element.text = "changed"
... return True
... 
>>> element.text = Text()
>>> i = element.itertext()
>>> next(i)
Segmentation fault (core dumped)
[Exit 139]

--
components: +XML
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: http://bugs.python.org/file41323/etree_iter2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file41324/etree_iter2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Good catch Martin. I missed this case. Following patch fixes it.

Other bugs look similar. They presumably can be triggered when remove the 
element itself during executing the __next__() method. I have no a reproducer 
though.

I don't know wherever it is worth to fix these bugs in maintained releases.

--
Added file: http://bugs.python.org/file41323/etree_iter2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25873] Faster ElementTree iterating

[Martin Panter]

Martin Panter added the comment:

The new changes look good.

I see these kind of reference counting bugs as fairly low priority. I suspect 
there are plenty more in this module, at least Issue 24103 and Issue 24104.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com