[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Марк Коренберг added the comment: Is issue 21579 fixed in that bug? too many letters..sorry... -- nosy: +mmarkk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- title: tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variablwe - tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Antoine Pitrou added the comment: Here is a proper patch (for 3.3). Untested under Windows, but should work. -- nosy: +pitrou Added file: http://bugs.python.org/file33250/tempfile_lifetime.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Antoine Pitrou added the comment: Oops, removed two debug lines. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Antoine Pitrou pit...@free.fr: Removed file: http://bugs.python.org/file33250/tempfile_lifetime.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Antoine Pitrou pit...@free.fr: Added file: http://bugs.python.org/file33252/tempfile_lifetime.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: LGTM. Thank you Antoine. -- assignee: serhiy.storchaka - pitrou stage: patch review - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Roundup Robot added the comment: New changeset f3b7a76fb778 by Antoine Pitrou in branch '3.3': Issue #18879: When a method is looked up on a temporary file, avoid closing the file before the method is possibly called. http://hg.python.org/cpython/rev/f3b7a76fb778 New changeset d68ab2eb7a77 by Antoine Pitrou in branch 'default': Issue #18879: When a method is looked up on a temporary file, avoid closing the file before the method is possibly called. http://hg.python.org/cpython/rev/d68ab2eb7a77 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Antoine Pitrou added the comment: Fixed in 3.3 and 3.4. I think 2.7 is irrelevant at this point. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Antoine Pitrou pit...@free.fr: -- stage: commit review - committed/rejected versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: Unfortunately last patch causes resource leak on 3.3+. $ ./python -Wall -m test.regrtest test_tempfile [1/1] test_tempfile 1 test OK. sys:1: ResourceWarning: gc: 5 uncollectable objects at shutdown; use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: Jakub's patch has a bug: import tempfile f = tempfile.NamedTemporaryFile(dir=.,delete=False) write = f.write write function BufferedRandom.write at 0xb716b4f4 write2 = f.write write2 built-in method write of _io.BufferedRandom object at 0xb721deac del f write(b'foo') 3 del write write2(b'bar') Traceback (most recent call last): File stdin, line 1, in module ValueError: write to closed file -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: Here is fixed Jakub's patch. -- assignee: - serhiy.storchaka Added file: http://bugs.python.org/file33153/tempfile_wrap_methods.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Zdeněk Pavlas added the comment: Hit this issue too, but with the read method. I think that providing a custom read() and write() methods in the wrapper class that override __getattr__ is a sufficient solution. These are the attributes most likely to be used as the tail. -- nosy: +Zdeněk.Pavlas ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Jakub Stasiak added the comment: I don't see an obvious way of solving that. One thing I could think of is creating a wrapper for file method being returned from __getattr__ that holds reference to _TemporaryFileWrapper instance until the method gets called, please find a patch attached. -- keywords: +patch nosy: +jstasiak Added file: http://bugs.python.org/file31598/tempfile.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: Yet one way is just remove __del__() and monkey-patch underlied file. -- Added file: http://bugs.python.org/file31603/tempfile_simplify.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Meador Inge added the comment: I was experimenting with something similar to what Jakub has. Rebinding the call the the wrapper seems like a simple and clean way to do it. Although, I wasn't absolutely sure whether this approach covers all cases. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Meador Inge added the comment: The monkey-patched version breaks the auto-close on __del__ feature: [meadori@li589-207 cpython]$ ./python Python 3.4.0a1+ (default:c41c68a18bb6, Sep 5 2013, 17:51:03) [GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux Type help, copyright, credits or license for more information. import tempfile tempfile.NamedTemporaryFile(dir=.,delete=False).write(bhello) __main__:1: ResourceWarning: unclosed file _io.BufferedRandom name=4 5 Also run the test suite with '-R :' and you will see tons of resource warnings. Breaking auto-close is because now unlinking *and* closing are being guarded by 'delete'. Even if that is fixed you still run into resource leak problems. Looks like the monkey-patching introduces a reference cycle. The wrapper binding approach works fine in all the tests I did. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Serhiy Storchaka added the comment: The monkey-patched version breaks the auto-close on __del__ feature: Files were closed. Just with a warning. I consider this as an enhancement. If you don't close file explicitly you will get a leak on non-refcounted Python implementations. It's true for both ordinal files so tempfiles. Perhaps we should add a warning in any __del__ method (even if we will commit Jakub's patch, this way LGTM too). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
jort bloem added the comment: I am only new to Python, but... Having looked at the code, I am surprised that the _TemporaryFileWrapper is not a subclass of file. Surely that would be the python way, and would solve this problem, too? __getattr__ would no longer be needed. The opening of the file is within the library, so _os.open() could be replaced with file()... It would require rewriting/reorganising chunks of this code... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
New submission from jort bloem: Issue occurs in the following code: tempfile.NamedTemporaryFile(dir=.,delete=False).write(hello) The NamedTemporaryFile is deleted before the write is complete; deleting the object closes the file, so the write fails. -- messages: 196485 nosy: jort.bloem priority: normal severity: normal status: open title: tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable type: behavior versions: Python 2.6, Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by Serhiy Storchaka storch...@gmail.com: -- components: +IO, Library (Lib) nosy: +georg.brandl, ncoghlan, serhiy.storchaka versions: +Python 3.3, Python 3.4 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
jort bloem added the comment: Sorry, my python-foo is not quite up to providing a solution. A partial solution would be for the file not to be closed at NamedTemporaryFile destruction, if delete==False. If delete==True, then there is no point writing to a file (though it shouldn't raise an exception, either). Alternatively, on a unix based system, the file could be deleted, but left open. Read write could continue as per normal, and the file would be closed automatically later. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Nick Coghlan added the comment: The method call should keep the file object alive until it completes (due to the self reference). What behaviour prompted the conclusion that it is being closed early? If the symptom is that the data isn't being written to disk, we may have a missing flush() call somewhere causing a race condition. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Meador Inge added the comment: Confirmed in trunk: tempfile.NamedTemporaryFile(dir=.,delete=False).write(bhello) Traceback (most recent call last): File stdin, line 1, in module ValueError: write to closed file -- nosy: +meador.inge ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue18879 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18879] tempfile.NamedTemporaryFile can close the file too early, if not assigning to a variable
Meador Inge added the comment:
'tempfile.NamedTemporaryFile' returns an instance of '_TemporaryFileWrapper'
that wraps the created file object. The wrapper object implements
'__getattr__' and we end up with a situation like the following simplified
example where the wrapper gets destroyed before the looked up method is called:
$ cat test.py
class S(object):
def __init__(self):
print(S.__init__) def __del__(self): print(S.__del__)
def foo(self):
print(S.foo)
class T(object):
def __init__(self, s):
self.s = s
print(T.__init__)
def __getattr__(self, name):
s = self.__dict__['s']
a = getattr(s, name)
print(__getattr__('%s') % name)
if not isinstance(a, int):
setattr(self, name, a)
return a
def __del__(self):
print(T.__del__)
T(S()).foo()
$ ./python test.py
S.__init__
T.__init__
__getattr__('foo')
T.__del__
S.foo
S.__del__
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18879
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
