[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Matej Cepl]

Matej Cepl  added the comment:

Does it even make sense to make a security patch for 2.7 for this one?

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am not sure this issue should be classified as a security issue. It can cause 
DDOS, because pickle should not be used with untrusted data. If it is used, the 
program has more severe security issues than just DDOS.

The crash could be triggered by accident, but this is very unlikely. I doubts 
that this happened even once in real world.  Libraries used for handling a 
large amount of data (like NumPy) use more efficient pickle representation, and 
can provide even more efficient alternate serialization methods. Note that 
integers and floats are not memoized, this increases the complexity and size of 
data that could be affected by this bug.

But I think that this fix needs a news entry. Do you mind to add it Benjamin?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Miro Hrončok]

Miro Hrončok  added the comment:

Should this go to 3.4 and 3.5 as well, since it is a security thing?

http://people.canonical.com/~ubuntu-security/cve/2018/CVE-2018-20406.html

--
nosy: +hroncok

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[miss-islington]

miss-islington  added the comment:


New changeset 71a9c65e74a70b6ed39adc4ba81d311ac1aa2acc by Miss Islington (bot) 
in branch '3.6':
closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261)
https://github.com/python/cpython/commit/71a9c65e74a70b6ed39adc4ba81d311ac1aa2acc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[miss-islington]

miss-islington  added the comment:


New changeset ef4306b24c9034d6b37bb034e2ebe82e745d4b77 by Miss Islington (bot) 
in branch '3.7':
closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261)
https://github.com/python/cpython/commit/ef4306b24c9034d6b37bb034e2ebe82e745d4b77


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[miss-islington]

Change by miss-islington :


--
pull_requests: +8879

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[miss-islington]

Change by miss-islington :


--
pull_requests: +8878

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd by Benjamin Peterson in 
branch 'master':
closes bpo-34656: Avoid relying on signed overflow in _pickle memos. (GH-9261)
https://github.com/python/cpython/commit/a4ae828ee416a66d8c7bf5ee71d653c2cc6a26dd


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

>>> import pickletools
>>> pickletools.dis(b'\x80\x04\x95\x1d\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x03age\x94K\x17\x8c\x03jobr\x8c\x07student\x94u.')
0: \x80 PROTO  4
2: \x95 FRAME  29
   11: }EMPTY_DICT
   12: \x94 MEMOIZE(as 0)
   13: (MARK
   14: \x8c SHORT_BINUNICODE 'age'
   19: \x94 MEMOIZE(as 1)
   20: KBININT123
   22: \x8c SHORT_BINUNICODE 'job'
   27: rLONG_BINPUT 1953695628
   32: uSETITEMS   (MARK at 13)
   33: dDICT   no MARK exists on stack
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/serhiy/py/cpython/Lib/pickletools.py", line 2457, in dis
raise ValueError(errormsg)
ValueError: no MARK exists on stack

Ignore the error of unbalanced MARK. The problem code is LONG_BINPUT with the 
excessive large argument 1953695628. The C implementation of pickle tries to 
resize the the memo list to the size twice larger than this index. And here an 
integer overflow occurred.

This unlikely occurred in real world. The pickle needs to have more than 
2**30-1 ≈ 10**9 memoized items for encountering this bug. It means that its 
size on disk and in memory should be tens or hundreds of gigabytes. Pickle is 
not the best format for serializing such amount of data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Kubilay Kocak]

Change by Kubilay Kocak :


--
nosy:  -koobs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Benjamin Peterson]

Change by Benjamin Peterson :


--
keywords: +patch
pull_requests: +8718
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +alexandre.vassalotti, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[shuoz]

shuoz  added the comment:

[--registers---]
RAX: 0x7ff9d401e010 --> 0x0 
RBX: 0x77f48d00 --> 0x1 
RCX: 0x7ff8ab58c800 --> 0x77ea5d80 --> 0x2 
RDX: 0x73ac47d8 --> 0x1 
RSI: 0x25152303 
RDI: 0xfff3a803c00 --> 0x0 
RBP: 0x7473078c 
RSP: 0x7fffcf20 --> 0x73ac47d8 --> 0x1 
RIP: 0x728a8a64 (<_Unpickler_MemoPut+1668>: addr11,0x20)
R8 : 0xfff3a803bff --> 0x0 
R9 : 0xfff3a803c01 --> 0x0 
R10: 0xefe91a3 --> 0x0 
R11: 0x128a917f8 --> 0x0 
R12: 0xfff156b1922 --> 0x0 
R13: 0xe8e60f18 --> 0x0 
R14: 0x77f48d18 --> 0x7ff8ab58c800 --> 0x77ea5d80 --> 0x2 
R15: 0xfff3a803c02 --> 0x0
EFLAGS: 0x216 (carry PARITY ADJUST zero sign trap INTERRUPT direction overflow)
[-code-]
   0x728a8a52 <_Unpickler_MemoPut+1650>:cmpBYTE PTR 
[r15+0x7fff8000],0x0
   0x728a8a5a <_Unpickler_MemoPut+1658>:jne0x728a8ae1 
<_Unpickler_MemoPut+1793>
   0x728a8a60 <_Unpickler_MemoPut+1664>:addrsi,0x4
=> 0x728a8a64 <_Unpickler_MemoPut+1668>:addr11,0x20
   0x728a8a68 <_Unpickler_MemoPut+1672>:cmpBYTE PTR 
[r10+0x7fff8000],0x0
   0x728a8a70 <_Unpickler_MemoPut+1680>:movQWORD PTR [rax],0x0
   0x728a8a77 <_Unpickler_MemoPut+1687>:je 0x728a896d 
<_Unpickler_MemoPut+1421>
   0x728a8a7d <_Unpickler_MemoPut+1693>:nopDWORD PTR [rax]
[stack-]
| 0x7fffcf20 --> 0x73ac47d8 --> 0x1 
0008| 0x7fffcf28 --> 0xefe91a3 --> 0x0 
0016| 0x7fffcf30 --> 0x77f48da8 --> 0x20 (' ')
0024| 0x7fffcf38 --> 0x77f48d00 --> 0x1 
0032| 0x7fffcf40 --> 0xa00 --> 0x0 
0040| 0x7fffcf48 --> 0x0 
0048| 0x7fffcf50 --> 0x77f48da0 --> 0x28 ('(')
0056| 0x7fffcf58 --> 0x77f48da8 --> 0x20 (' ')
[--]
Legend: code, data, rodata, value
0x728a8a64  1392for (i = self->memo_size; i < new_size; i++)
gdb-peda$ p new_size
$5 = 0xe8e60f18
gdb-peda$ p self->memo_size
$6 = 0x20
gdb-peda$ p i


.
for (i = self->memo_size; i < new_size; i++)
self->memo[i] = NULL;
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[shuoz]

Change by shuoz :


Added file: https://bugs.python.org/file47802/pk.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34656] memory exhaustion in Modules/_pickle.c:1393

[shuoz]

New submission from shuoz :

python version:
   Python 3.8.0a0 (heads/master:4ae8ece, Sep 13 2018, 09:48:16) 
   [GCC 5.4.0 20160609] on linux


I found a bug in python pickle.load func. Can cause memory exhaustion DDOS.

./python pk.py poc


cat ./pk.py
import pickle
import sys
filename = sys.argv[1]
with open(filename, 'rb') as f:
aa = pickle.load(f)
print(aa)

--
components: FreeBSD
files: poc
messages: 325230
nosy: koobs, shuoz
priority: normal
severity: normal
status: open
title: memory exhaustion in Modules/_pickle.c:1393
type: security
versions: Python 3.8
Added file: https://bugs.python.org/file47801/poc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com