New submission from Pierre Glaser <[email protected]>:
The new Pickler reducer_override mechanism introduced in `Python3.8` generates
a reference cycle: for optimization purposes, a the pickler.reducer_override
bound method is referenced into the reducer_override attribute of the Pickler's
struct. Thus, until as a gc.collect call is performed, both the Pickler and all
the elements it pickled (as they are part of its memo), wont be collected.
We should break this cycle a the end of the dump() method.
See reproducer below:
```
import threading
import weakref
import pickle
import io
class MyClass:
pass
my_object = MyClass()
collect = threading.Event()
_ = weakref.ref(my_object, lambda obj: collect.set()) # noqa
class MyPickler(pickle.Pickler):
def reducer_override(self, obj):
return NotImplemented
my_pickler = MyPickler(io.BytesIO())
my_pickler.dump(my_object)
del my_object
del my_pickler
# import gc
# gc.collect()
for i in range(5):
collected = collect.wait(timeout=0.1)
if collected:
print('my_object was successfully collected')
break
```
----------
components: Library (Lib)
messages: 360995
nosy: pierreglaser, pitrou
priority: normal
severity: normal
status: open
title: reference cycle affecting Pickler instances (Python3.8+)
type: resource usage
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue39492>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
