[issue41422] C Unpickler memory leak via memo

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41422] C Unpickler memory leak via memo

[kale-smoothie]

kale-smoothie  added the comment:

The leak demonstrated in the attachment is, to my understanding, caused by 
memoizing the closure returned from the `find_class` method that's used to 
intercept global references. The cycle is then: Unpickler, memo table, closure, 
Unpickler (via cell reference to `self`).

My proposed patch visits every entry in the memo table.

Pre-patch run of valgrind on leak_pickler.py:

==20339== HEAP SUMMARY:
==20339== in use at exit: 190,189,238 bytes in 2,406,919 blocks
==20339==   total heap usage: 3,150,288 allocs, 743,369 frees, 233,766,596 
bytes allocated
==20339== 
==20339== LEAK SUMMARY:
==20339==definitely lost: 0 bytes in 0 blocks
==20339==indirectly lost: 0 bytes in 0 blocks
==20339==  possibly lost: 190,176,150 bytes in 2,406,835 blocks
==20339==still reachable: 13,088 bytes in 84 blocks
==20339== suppressed: 0 bytes in 0 blocks
==20339== Rerun with --leak-check=full to see details of leaked memory

Post-patch run of valgrind on leak_pickler.py:

==20880== HEAP SUMMARY:
==20880== in use at exit: 667,277 bytes in 6,725 blocks
==20880==   total heap usage: 2,853,739 allocs, 2,847,014 frees, 216,473,216 
bytes allocated
==20880== 
==20880== LEAK SUMMARY:
==20880==definitely lost: 0 bytes in 0 blocks
==20880==indirectly lost: 0 bytes in 0 blocks
==20880==  possibly lost: 654,624 bytes in 6,646 blocks
==20880==still reachable: 12,653 bytes in 79 blocks
==20880== suppressed: 0 bytes in 0 blocks
==20880== Rerun with --leak-check=full to see details of leaked memory

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41422] C Unpickler memory leak via memo

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 1.0 -> 2.0
pull_requests: +20809
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21664

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41422] C Unpickler memory leak via memo

[kale-smoothie]

New submission from kale-smoothie :

I'm not familiar with the workings of GC/pickle, but it looks like the traverse 
code in the C Unpickler omits a visit to the memo, potentially causing a memory 
leak?

--
components: Library (Lib)
files: leak_pickler.py
messages: 374518
nosy: kale-smoothie
priority: normal
severity: normal
status: open
title: C Unpickler memory leak via memo
type: resource usage
versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 
3.9
Added file: https://bugs.python.org/file49345/leak_pickler.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com