New submission from Serhiy Storchaka <[email protected]>:
There is possible resource leak in glob on alternate Python implementation
witch do not use reference counting to immediate releasing resources.
It is in the code
names = list(_iterdir(dirname, dir_fd, dironly))
where _iterdir() is a generator function which calls os.scandir() and yields
entry names after some filtering. If an exception happens inside _iterdir(),
the scandir iterator will be immediately closed, because of using the with
statement. But an exception can happens outside of _iterdir(), in the list
constructor (MemoryError). In this case the generator will be closed
immediately in CPython because of reference counting (newly created generator
has only one reference), but on other implementation it can be deferred on
arbitrary time. And even in CPython we rely on garbage collector if there
reference loops in the generator frame. This issue has very small chance to
occur but still.
The right way is to close the generator explicitly:
it = _iterdir(dirname, dir_fd, dironly)
try:
names = list(it)
finally:
it.close()
or
with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
names = list(it)
We should analyze all other generator functions which acquire some resources
and ensure that they are always properly closed.
----------
components: Library (Lib)
messages: 396306
nosy: gvanrossum, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Possible resource leeak in glob in non-refcount implementations
type: resource usage
versions: Python 3.10, Python 3.11, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue44482>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
