[issue46932] Please update bundled libexpat to 2.4.7 with an important fix
New submission from sping : Hi! I believe the fix documented at https://github.com/libexpat/libexpat/blob/27d5b8ba1771f916d9cfea2aac6bdac72071dc66/expat/Changes#L7-L11 is important to users of CPython. Please update bundled libexpat to 2.4.7. Thank you! Best, Sebastian -- components: XML messages: 414591 nosy: sping priority: normal severity: normal status: open title: Please update bundled libexpat to 2.4.7 with an important fix type: behavior versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46932> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46794] Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)
sping added the comment: Hi mattip, at the core the problem is not the use of non-URI character "}" for a namespace separator but the use of non-URI character "}" in a namespace URI. test_issue3151 is mistaken (meaning that non-URI characters in URIs are malformed XML) and the test has been removed in CPython pull request https://github.com/python/cpython/pull/31453/files . Expat pull request https://github.com/libexpat/libexpat/pull/577 is related but it's about URI characters not about non-URI ones, so it does not change anything about test_issue3151 in PyPy. Does that make sense? Best, Sebastian -- ___ Python tracker <https://bugs.python.org/issue46794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46793] expose expat XML billion laughs attack mitigation APIs
sping added the comment: First mention at https://bugs.python.org/issue44394#msg395642 -- nosy: +sping ___ Python tracker <https://bugs.python.org/issue46793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46811] Test suite needs adjustments for Expat >=2.4.5
Change by sping : -- keywords: +patch pull_requests: +29581 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31453 ___ Python tracker <https://bugs.python.org/issue46811> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46794] Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)
sping added the comment: I have created a dedicated ticket bpo-46811 now, test suite pull request upcoming. -- ___ Python tracker <https://bugs.python.org/issue46794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46811] Test suite needs adjustments for Expat >=2.4.5
New submission from sping : It has been reported at https://bugs.python.org/issue46794#msg413587 that the current CPython test suite needs some adjustments for Expat >=2.4.5. Since that is somewhat separate from updating the bundled copy of Expat to >=2.4.6, I am creating this dedicated ticket. I pull request for discussion will follow shortly. -- components: XML messages: 413605 nosy: mgorny, sping priority: normal severity: normal status: open title: Test suite needs adjustments for Expat >=2.4.5 versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46811> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46794] Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)
sping added the comment: I'm busy with the release upstream at the moment. I'll see what I can do. -- ___ Python tracker <https://bugs.python.org/issue46794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46794] Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)
sping added the comment:
Hi Michal,
TL;DR would be:
- There is a regression but none of these test fails are related.
- There will be a release Expat 2.4.6 with the regression fixed later today.
- The 3 failing tests need (small) adjustments to Expat 2.4.5
and these fails are not considered bugs in Expat.
I will demo a fix to 2 of the 3 test fails below:
# git diff -U1 | cat
diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py
index 1663b1f114..38cea97a97 100644
--- a/Lib/test/test_minidom.py
+++ b/Lib/test/test_minidom.py
@@ -12,2 +12,3 @@
from xml.dom.minidom import getDOMImplementation
+from xml.parsers.expat import ExpatError
@@ -1149,4 +1150,6 @@ def testEncodings(self):
# of crashing
-self.assertRaises(UnicodeDecodeError, parseString,
-b'Comment \xe7a va ? Tr\xe8s bien
?')
+self.assertRaises(ExpatError, parseString,
+b'')
+self.assertRaises(ExpatError, parseString,
+b'Comment \xe7a va ? Tr\xe8s bien ?')
@@ -1611,3 +1614,3 @@ def testEmptyXMLNSValue(self):
def testExceptionOnSpacesInXMLNSValue(self):
-with self.assertRaisesRegex(ValueError, 'Unsupported syntax'):
+with self.assertRaisesRegex(ExpatError, "syntax error"):
parseString('')
For the third test, the key is that the closing curly brace is used as the
namespace separator in line 3660…
self->parser = EXPAT(ParserCreate_MM)(encoding, , "}");
…in file Modules/_elementtree.c (which is okay but part of the test fail).
Best
Sebastian
--
title: Please update bundled libexpat to 2.4.5 with security fixes (5 CVEs) ->
Please update bundled libexpat to 2.4.6 with security fixes (5 CVEs)
___
Python tracker
<https://bugs.python.org/issue46794>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please update bundled libexpat to 2.4.4 with security fixes (CVE-2021-45960)
sping added the comment: Overlooked your reference, so you already know, my bad, nevermind. -- ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please update bundled libexpat to 2.4.4 with security fixes (CVE-2021-45960)
sping added the comment: Yes, I have already created bpo-46794 for 2.4.5. -- ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46794] Please update bundled libexpat to 2.4.5 with security fixes (5 CVEs)
New submission from sping : Thank you! https://github.com/libexpat/libexpat/blob/97a4840578693a346e79302909b67d97492e1880/expat/Changes#L6-L35 -- components: XML messages: 413517 nosy: sping priority: normal severity: normal status: open title: Please update bundled libexpat to 2.4.5 with security fixes (5 CVEs) type: security versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46794> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please update bundled libexpat to 2.4.4 with security fixes (CVE-2021-45960)
sping added the comment: Just to understand, why has Python 3.7 and 3.8 been dropped? Neither seems to be end-of-life but affected. Thank you! -- ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please update bundled libexpat to 2.4.4 with security fixes
sping added the comment: 2.4.4 with more security fixes has been released, adjusting the ticket to be about updating to 2.4.4 now. -- title: Please update bundled libexpat to 2.4.3 with security fixes -> Please update bundled libexpat to 2.4.4 with security fixes ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38487] expat infinite loop
sping added the comment:
Hi StyXman,
I had a closer look at the files you shared, thanks for those, very helpful!
What I found is that expat_test.py uses a single scalar variable
(_DictSAXHandler.parser) to keep track of the related parser, while it would
need a stack to allow recursion. In a way, the current approach is equivalent
to walking up the stack as expected but never going back down.
Once I make the code use a stack, the loop goes away. I'm pasting the patch
inline (with two spaces indented globally) below.
During debugging, these are commands I used to compare internal libexpat
behavior,
that may be of interest:
EXPAT_ACCOUNTING_DEBUG=2 python expat_test.py |& sed 's,0x[0-9a-f]\+,XXX,' |
tee pyexpat.txt
EXPAT_ACCOUNTING_DEBUG=2 xmlwf -x test1.xml |& sed 's,0x[0-9a-f]\+,XXX,' |
tee xmlwf.txt
diff -u xmlwf.txt pyexpat.txt
Here's how I quick-fixed expat_test.py to make things work:
# diff -u expat_test.py_ORIG expat_test.py
--- expat_test.py_ORIG 2022-01-26 21:15:27.506458671 +0100
+++ expat_test.py 2022-01-26 22:15:08.741384932 +0100
@@ -7,11 +7,21 @@
parser.ExternalEntityRefHandler = handler.externalEntityRef
-# store the parser in the handler so we can recurse
-handler.parser = parser
-
class _DictSAXHandler(object):
+def __init__(self):
+self._parsers = []
+
+def push_parser(self, parser):
+self._parsers.append(parser)
+
+def pop_parser(self):
+self._parsers.pop()
+
+@property
+def parser(self):
+return self._parsers[-1]
+
def externalEntityRef(self, context, base, sysId, pubId):
print(context, base, sysId, pubId)
external_parser = self.parser.ExternalEntityParserCreate(context)
@@ -19,7 +29,9 @@
setup_parser(external_parser, self)
f = open(sysId, 'rb')
print(f)
+self.push_parser(external_parser)
external_parser.ParseFile(f)
+self.pop_parser()
print(f)
# all OK
@@ -36,12 +48,13 @@
namespace_separator
)
setup_parser(parser, handler)
+handler.push_parser(parser)
if hasattr(xml_input, 'read'):
parser.ParseFile(xml_input)
else:
parser.Parse(xml_input, True)
-return handler.item
+# return handler.item # there is no .item
parse(open('test1.xml', 'rb'))
What do you tink?
PS: Please note that processing external entities has security implications
(see https://en.wikipedia.org/wiki/XML_external_entity_attack).
Best, Sebastian
--
nosy: +sping
___
Python tracker
<https://bugs.python.org/issue38487>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45713] gcc warning when compiling Modules/expat/xmltok_ns.c
sping added the comment: This has been fixed upstream in pull request https://github.com/libexpat/libexpat/pull/527 that is included with latest release libexpat 2.4.3. bpo-46400 will fix this as a side effect. -- nosy: +sping ___ Python tracker <https://bugs.python.org/issue45713> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please update bundled libexpat to 2.4.3 with security fixes
Change by sping : -- title: Please updated bundled libexpat to 2.4.3 with security fixes -> Please update bundled libexpat to 2.4.3 with security fixes ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46400] Please updated bundled libexpat to 2.4.3 with security fixes
New submission from sping : Expat 2.4.3 released, includes security fixes https://blog.hartwork.org/posts/expat-2-4-3-released/ Thank you! PS: This is similar to bpo-44394 excect now it's 2.4.3. -- components: XML messages: 410700 nosy: sping priority: normal severity: normal status: open title: Please updated bundled libexpat to 2.4.3 with security fixes type: security versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46400> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45321] Module xml.parsers.expat.errors misses error code constants of libexpat >=2.0
New submission from sping : (This has been mention at https://bugs.python.org/issue44394#msg395642 before, but issue 44394 has been closed as fixed despite that part being forgotten, hence the dedicated ticket...) Module `xml.parsers.expat.errors` and its docs need 6 more error code entries to be complete: /* Added in 2.0. */ 38 XML_ERROR_RESERVED_PREFIX_XML 39 XML_ERROR_RESERVED_PREFIX_XMLNS 40 XML_ERROR_RESERVED_NAMESPACE_URI /* Added in 2.2.1. */ 41 XML_ERROR_INVALID_ARGUMENT /* Added in 2.3.0. */ 42 XML_ERROR_NO_BUFFER /* Added in 2.4.0. */ 43 XML_ERROR_AMPLIFICATION_LIMIT_BREACH The source for this is: - https://github.com/libexpat/libexpat/blob/72d7ce953827fe08a56b8ea64092f208be6ffc5b/expat/lib/expat.h#L120-L129 The place where additions is needed is: - https://github.com/python/cpython/blob/f76889a88720b56c8174f26a20a8e676a460c7a6/Modules/pyexpat.c#L1748 - https://github.com/python/cpython/blame/f76889a88720b56c8174f26a20a8e676a460c7a6/Doc/library/pyexpat.rst#L867 Thanks in advance. -- components: Extension Modules messages: 402866 nosy: sping priority: normal severity: normal status: open title: Module xml.parsers.expat.errors misses error code constants of libexpat >=2.0 versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue45321> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44394] [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1
sping added the comment: For the AIX link error that Pablo brought up, there is merged pull request https://github.com/libexpat/libexpat/pull/510 upstream. -- ___ Python tracker <https://bugs.python.org/issue44394> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44394] [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1
Change by sping : -- title: [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Updated to vendoed copy to expat 2.4.1 -> [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Update vendored copy to expat 2.4.1 ___ Python tracker <https://bugs.python.org/issue44394> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44394] [security] CVE-2013-0340 "Billion Laughs" fixed in Expat >=2.4.0: Updated to vendoed copy to expat 2.4.1
sping added the comment: FTR that^^ Sebastian is me :) -- nosy: +sping ___ Python tracker <https://bugs.python.org/issue44394> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31170] expat: utf8_toUtf8 cannot properly handle exhausting buffer
sping added the comment: Expat 2.2.4 with a fix has been released now: https://github.com/libexpat/libexpat/releases -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31170> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31170] expat: utf8_toUtf8 cannot properly handle exhausting buffer
sping added the comment: For the record, the upstream fix is commit https://github.com/libexpat/libexpat/commit/74a7090a6eb92c27b7010287a4082de6b357fa42 and it will be part of Expat 2.2.4. -- nosy: +sping versions: +Python 2.7, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue31170> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27369] [PATCH] Tests break with --with-system-expat and Expat 2.2.0
Changes by sping <sebast...@pipping.org>: Added file: http://bugs.python.org/file43515/0001-Fix-Python-3.x.x-tests-for-Expat-2.2.0.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27369> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27369] [PATCH] Tests break with --with-system-expat and Expat 2.2.0
New submission from sping: Expat commit https://sourceforge.net/p/expat/code_git/ci/550eb6bbaa7e6efc44e94cab945fe9c781c01211 introduced with the recent release of Expat 2.2.0 fixed a bug that the current Python test suite relies on. Attached you find patches to fix the test suite for all versions in a backwards-compatible way. Please consider application upstream. Thanks! Best, Sebastian -- components: Tests files: 0001-Fix-Python-2.7.11-tests-for-Expat-2.2.0.patch keywords: patch messages: 269079 nosy: sping priority: normal severity: normal status: open title: [PATCH] Tests break with --with-system-expat and Expat 2.2.0 type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file43514/0001-Fix-Python-2.7.11-tests-for-Expat-2.2.0.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27369> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24266] raw_input + readline: Ctrl+C during search breaks readline
sping added the comment: I guess supporting older upstream versions would be nice in this case. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24266 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24266] raw_input function (with readline): Ctrl+C (during search mode but not only) leaves readline in broken state
Changes by sping sebast...@pipping.org: Added file: http://bugs.python.org/file39468/raw_input__minimal.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24266 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24266] raw_input function (with readline): Ctrl+C (during search mode but not only) leaves readline in broken state
New submission from sping: Hi! A college of mine ran into a bug with raw_input. We have a shell derived from stdlib module cmd here but the bug shows with plain raw_input, as well (see demo code). For the symptoms: the shell is executing commands from history that the user explicitly chose not to run. Since our shell does things like deallocation of LVM volumes, that behavior is rather troublesome :) How does it happen? * When pressing Ctrl+R, an incremental reverse history search is started. During that search, you normalled press Escape/Ctrl+J/Ctrl+G to end search or Return to pick a result. * When you press Ctrl+C though, the next call to the raw_input function believes to be in search mode still (while not showing (reverse-i-search) or indicating search some way). * Now when (entering some text and) pressing return now, the string last shown during incremental search is being return from raw_input (and executed in context of the cmd module). I have re-produced the issue with Python 2.7.3, 2.7.9, 3.2.3, 3.4.2. For a workaround, one can handle KeyboardInterrupt and internal adjust variable rl_readline_state of the C readline library. I'm attaching (a minimal bug demo and) the ctypes based workaround that works well over here. (The workaround demo also shows that readline state is not fully reset when Ctrl+C was pressed outside of search mode, since flag RL_STATE_DONE is not set after.) It would rock the house if this could be fixed in Python. I'm looking forward to your reply. Best, Sebastian -- components: Interpreter Core files: raw_input__workaround_demo.py messages: 243828 nosy: sping priority: normal severity: normal status: open title: raw_input function (with readline): Ctrl+C (during search mode but not only) leaves readline in broken state type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file39467/raw_input__workaround_demo.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24266 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
