https://github.com/python/cpython/commit/f5b4289e0156bfae237e4d07defd13d4b9ed1308 commit: f5b4289e0156bfae237e4d07defd13d4b9ed1308 branch: 3.14 author: Miss Islington (bot) <[email protected]> committer: serhiy-storchaka <[email protected]> date: 2026-09-13T12:43:31Z summary:
[3.14] gh-93618: Document the memory usage of the incremental parsers (GH-156763) (GH-157420) It was said that iterparse() can be useful for reading a large document without holding it wholly in memory, but the tree is only built incrementally, it is not freed incrementally. Document how to remove the processed elements, and that a custom target does not build a tree at all. (cherry picked from commit e7a393797a0df47a001673f4510c51aa272bc002) Co-authored-by: Serhiy Storchaka <[email protected]> files: M Doc/library/xml.etree.elementtree.rst diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 66a97682ed5923..592a9db50f828a 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -162,8 +162,37 @@ some storage device. In such cases, blocking reads are unacceptable. Because it's so flexible, :class:`XMLPullParser` can be inconvenient to use for simpler use-cases. If you don't mind your application blocking on reading XML data but would still like to have incremental parsing capabilities, take a look -at :func:`iterparse`. It can be useful when you're reading a large XML document -and don't want to hold it wholly in memory. +at :func:`iterparse`. + +Note that both parsers build the tree incrementally: it is not freed +incrementally, so every parsed element is kept until the whole document is +read. To keep the memory usage low, get rid of the data which is not needed +any more. + +If the processed elements are large, it is enough to clear them. +This works wherever they are in the tree, +but the emptied elements are left in it:: + + for event, elem in ET.iterparse(source): + if elem.tag == 'record': + process(elem) + elem.clear() + +If an element has a large number of children, +remove the processed children from it:: + + for event, elem in ET.iterparse(source, events=('start', 'end')): + if event == 'start' and elem.tag == 'parent': + parent = elem + elif event == 'end' and elem.tag == 'child': + process(elem) + parent.remove(elem) + +These examples are not universal, +they only give an idea for two common cases. +If you do not need a tree at all, +parse with :class:`XMLParser` and a custom target instead; +it is not built then, and nothing has to be removed. Where *immediate* feedback through events is wanted, calling method :meth:`XMLPullParser.flush` can help reduce delay; @@ -637,6 +666,10 @@ Functions for applications where blocking reads can't be made. For fully non-blocking parsing, see :class:`XMLPullParser`. + The tree is only built incrementally, it is not freed incrementally: + every parsed element is kept until the whole document is read. + See :ref:`elementtree-pull-parsing` for how to keep the memory usage low. + .. note:: :func:`iterparse` only guarantees that it has seen the ">" character of a _______________________________________________ Python-checkins mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/python-checkins.python.org Member address: [email protected]
