New submission from Walter Doekes:
If you define a FileHandler in your logging dictConfig but fail to use it in
the same configuration, the file handle is leaked and a ResourceWarning is
thrown.
Versions:
$ python3 -V
Python 3.4.0
$ apt-cache show python3.4 | grep ^Version
Version: 3.4.0-2ubuntu1
Expected:
$ PYTHONWARNINGS="default,ignore::DeprecationWarning" \
python3 problem.py
imported once
Observed:
$ PYTHONWARNINGS="default,ignore::DeprecationWarning" \
python3 problem.py
.../lib/python3.4/importlib/_bootstrap.py:656:
ResourceWarning: unclosed file
<_io.FileIO name='/tmp/debug.log' mode='ab'>
code = marshal.loads(data)
imported once
To reproduce, save below as problem.py:
if __name__ == '__main__':
LOGGING = {
'version': 1,
'handlers': {
'logfile': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/tmp/debug.log',
},
},
}
from logging.config import dictConfig
dictConfig(LOGGING)
# using importlib on a new file triggers the warnings
import importlib, shutil, os
shutil.copy(__file__, __file__ + '.new')
os.unlink(__file__)
os.rename(__file__ + '.new', __file__)
importlib.import_module('problem')
else:
print('imported once')
Fixed by:
--- /usr/lib/python3.4/logging/config.py.orig 2014-12-08
14:06:24.911460799 +0100
+++ /usr/lib/python3.4/logging/config.py 2014-12-08 14:16:17.519460799
+0100
@@ -637,6 +637,16 @@ class DictConfigurator(BaseConfigurator)
except Exception as e:
raise ValueError('Unable to configure root '
'logger: %s' % e)
+
+ # Close unused loggers
+ used_handlers = set()
+ for logger in logging.root.manager.loggerDict.values():
+ if hasattr(logger, 'handlers') and logger.handlers:
+ used_handlers.add(*logger.handlers)
+ for name, handler in handlers.items():
+ if handler not in used_handlers:
+ if hasattr(handler, 'close'):
+ handler.close()
finally:
logging._releaseLock()
The fix above might not be the best fix, but it proves where the problem is.
Perhaps the handlers shouldn't be instantiated until the first user of such a
handler is instantiated.
This issue was found by using Django 1.7 with Python 3.4 when enabling the
PYTHONWARNINGS.
Cheers,
Walter Doekes
OSSO B.V.
----------
components: Library (Lib)
messages: 232302
nosy: wdoekes
priority: normal
severity: normal
status: open
title: "unclosed file" warning when defining unused logging FileHandler in
dictConfig
versions: Python 3.4
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue23010>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
