[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Brett Cannon added the comment: While you're right, Marcel, that code which pulls out the file object form FieldStorage would probably have the file closed from underneath it, I don't know if I agree that it's a bad thing. The FieldStorage object created that file, implicitly putting it in charge of managing it. Having it guarantee that it gets closed seems to me totally reasonable and a bug not to do so. And having Bottle break as-is doesn't sway me as this is a bug fix and so updating Bottle as part of the process to support Python 3.4 is reasonable. That use of the term leisure is definitely a problem, though. So I'm going to make this a bug report for updating the docs to no longer use the term leisure and add a versionchanged note that FieldStorage will close the 'file' attribute upon deletion. I'll also opened http://bugs.python.org/issue20289 to add context manager support to FieldStorage. But since leaving files randomly open is not good I can't bring myself to revert this change. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Changes by Brett Cannon br...@python.org: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Roundup Robot added the comment: New changeset 13d04a8713ad by Brett Cannon in branch 'default': Issue #18394: Document that cgi.FieldStorage now cleans up after its http://hg.python.org/cpython/rev/13d04a8713ad -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Marcel Hellkamp added the comment: This change breaks existing applications. The cgi.FieldStorage.file attribute is public and mentioned in the documentation. It even states You can then read the data at leisure from the file attribute. Consider this example:: form = cgi.FieldStorage() fileitem = form.getfirst(userfile) if fileitem and fileitem.file: executor.submit(store_file, fileitem.file, fileitem.filename) This code is no longer safe. The garbage collector might close the file handle while it is still referenced and accessed from the worker thread. Another example is the bottle web framework. It uses cgi.FieldStorage for parsing only, extracts the valuable information and stores the result in its own data structures. The cgi.FieldStorage instance is lost. Python 3.4 breaks every single bottle application that works with file uploads. How about implementing the context manager protocol for cgi.FieldStorage to resolve this issue? -- nosy: +Marcel.Hellkamp ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Changes by Brett Cannon br...@python.org: -- assignee: - brett.cannon keywords: +3.3regression -patch nosy: +larry priority: normal - release blocker status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Roundup Robot added the comment: New changeset c0e9ba7b26d5 by Brett Cannon in branch 'default': Issue #18394: Explicitly close the file object cgi.FieldStorage http://hg.python.org/cpython/rev/c0e9ba7b26d5 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Brett Cannon added the comment: I managed to write a similar patch to Vajrasky independently, so at least I know the approach is reasonable. =) I'm not backporting since it really isn't that critical; I fixed it just to turn off the ResourceWarning while running the test suite. -- nosy: +brett.cannon resolution: - fixed status: open - closed versions: -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Vajrasky Kok added the comment: It happens because if the length of data is more than 1000: def __write(self, line): line is always bytes, not string if self.__file is not None: if self.__file.tell() + len(line) 1000: self.file = self.make_file() data = self.__file.getvalue() self.file.write(data) self.__file = None .. it will create a temporary file. Attached the patch to close the temporary file in the destructor method. About the 1000 number, should we put it in constant? Why 1000? This number is so random. For now, I just leave it as it is. -- keywords: +patch nosy: +vajrasky Added file: http://bugs.python.org/file31239/fix_resource_warning_in_test_cgi.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Madison May added the comment: I ran into a similar issue (see #18700) with test_cgi. ``/home/mmay/cpython/Lib/test/test_cgi.py:276: ResourceWarning: unclosed file _io.BufferedRandom name=3`` -- nosy: +madison.may ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
New submission from Florent Xicluna:
It happens when POSTing a file for example.
When running the test suite:
./python.exe -m test test_cgi
Or with the script attached:
$ ./python test_fieldstorage.py
test_fieldstorage.py:28: ResourceWarning: unclosed file _io.BufferedRandom
name=3
check('x' * 1010) # ResourceWarning
test_fieldstorage.py:29: ResourceWarning: unclosed file _io.BufferedRandom
name=3
check('x' * (maxline - 1)) # ResourceWarning
--
components: Library (Lib)
files: test_fieldstorage.py
messages: 192530
nosy: flox, orsenthil, serhiy.storchaka
priority: normal
severity: normal
status: open
title: cgi.FieldStorage triggers ResourceWarning sometimes
type: behavior
versions: Python 3.3, Python 3.4
Added file: http://bugs.python.org/file30837/test_fieldstorage.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18394
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18394] cgi.FieldStorage triggers ResourceWarning sometimes
Changes by Barry A. Warsaw ba...@python.org: -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18394 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
