[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Ezio Melotti]

Ezio Melotti  added the comment:

Fixed, thanks Ivan for the report and Alex for suggesting Py_EnterRecursiveCall!

--
resolution:  -> fixed
stage: commit review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 6fa20e360e5e by Ezio Melotti in branch '2.7':
#12017: Fix segfault in json.loads() while decoding highly-nested objects using 
the C accelerations.
http://hg.python.org/cpython/rev/6fa20e360e5e

New changeset 61164d09337e by Ezio Melotti in branch '3.1':
#12017: Fix segfault in json.loads() while decoding highly-nested objects using 
the C accelerations.
http://hg.python.org/cpython/rev/61164d09337e

New changeset db97968379dd by Ezio Melotti in branch '3.2':
#12017: merge with 3.1.
http://hg.python.org/cpython/rev/db97968379dd

New changeset 389620c9e609 by Ezio Melotti in branch 'default':
#12017: merge with 3.2.
http://hg.python.org/cpython/rev/389620c9e609

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
versions:  -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Not applicable to 2.6 after all.

--
priority: release blocker -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Thanks for the patch.  I'll apply this to 2.6svn for the 2.6.7rc1 release 
today.  Feel free to apply this to 2.6hg and forward port it to the relevant 
releases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Ezio Melotti]

Ezio Melotti  added the comment:

Attached patch fixes the issues. The patch includes 6 tests that caused a 
segfault and that now raise a "RuntimeError: maximum recursion depth exceeded" 
error, matching the behavior of the Python version.

The recursion happen because scan_once_str/unicode might call 
_parse_object_str/unicode (for objects/dicts) and _parse_array_str/unicode (for 
arrays/lists), and these functions might call scan_once_str again for their 
inner elements.
To fix the problem I added Py_Enter/LeaveRecursiveCall around the calls to 
_parse_object_str and _parse_array_str in scan_once_str/unicode.

For some reason the message raised with json.loads('{"a":' * 10 + '1' + '}' 
* 10), is a generic "maximum recursion depth exceeded while calling a 
Python object", but that's probably not too important.
The other messages work fine.

--
keywords: +patch
nosy: +benjamin.peterson, georg.brandl
priority: high -> release blocker
stage: needs patch -> commit review
Added file: http://bugs.python.org/file21911/issue12017.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Ezio Melotti]

Changes by Ezio Melotti :


--
assignee:  -> ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +barry
priority: normal -> high
stage: test needed -> needs patch
versions: +Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Alex Gaynor]

Alex Gaynor  added the comment:

Why not use Py_EnterRecursiveCall?

--
nosy: +alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[ivank]

ivank  added the comment:

 any opinions here on what the default max_depth should be? is there any 
safe number?
 I'm curious as to how many C stack frames I can use after reaching the 
Python recursion limit
 and what ulimit -s looks like on every platform
 http://hg.python.org/cpython/file/tip/Lib/test/test_parser.py#l537
 ivan`, maybe this is useful

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[Ezio Melotti]

Changes by Ezio Melotti :


--
components: +Extension Modules -Library (Lib)
nosy: +ezio.melotti
stage:  -> test needed
versions: +Python 3.1, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12017] Decoding a highly-nested object with json (_speedups enabled) causes segfault

[ivank]

New submission from ivank :

Decoding a highly-nested object with json (_speedups enabled) can cause a 
segfault due to a stack overflow:

# python -c "import json; json.loads('[' * 10 + '1' + ']' * 10)"
zsh: segmentation fault  python -c "import json; json.loads('[' * 10 + '1' 
+ ']' * 10)"

# python -c "import json; json.loads('{\"a\":' * 10 + '1' + '}' * 10)"
zsh: segmentation fault  python -c "import json; json.loads('{\"a\":' * 10 
+ '1' + '}' * 10)"

simplejson has the same problem:
https://github.com/simplejson/simplejson/pull/11

I've started on a fix (see patch at that URL), but it doesn't support 
customizing max_depth yet.

--
components: Library (Lib)
messages: 135281
nosy: ivank
priority: normal
severity: normal
status: open
title: Decoding a highly-nested object with json (_speedups enabled) causes 
segfault
type: security
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com