[issue16477] tarfile fails to close file handles in case of exception
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Andrew Svetlov added the comment: Fixed. Thanks, Serhiy -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Roundup Robot added the comment: New changeset b7bdc0b3c2fe by Andrew Svetlov in branch '3.2': Issue #16477: Close tarfile internal handlers in case of exception. http://hg.python.org/cpython/rev/b7bdc0b3c2fe New changeset 80749ddc30b6 by Andrew Svetlov in branch '3.3': Merge issue #16477: Close tarfile internal handlers in case of exception. http://hg.python.org/cpython/rev/80749ddc30b6 New changeset 2d887691a99b by Andrew Svetlov in branch 'default': Merge issue #16477: Close tarfile internal handlers in case of exception. http://hg.python.org/cpython/rev/2d887691a99b New changeset ea4bdf5a2e69 by Andrew Svetlov in branch '2.7': Issue #16477: Close tarfile internal handlers in case of exception. http://hg.python.org/cpython/rev/ea4bdf5a2e69 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Andrew Svetlov added the comment: LGTM -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Serhiy Storchaka added the comment: Would with source = self.extractfile(tarinfo): work? No. extractfile() can return an instance of custom class. But in 3.x this not used at all. Here are patches for 2.7 and 3.x. -- keywords: +patch Added file: http://bugs.python.org/file28012/tarfile_fd_leaks-2.7.patch Added file: http://bugs.python.org/file28013/tarfile_fd_leaks.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___diff -r 457c0c9c7893 Lib/tarfile.py --- a/Lib/tarfile.pyThu Nov 15 07:10:27 2012 +0100 +++ b/Lib/tarfile.pySat Nov 17 21:57:57 2012 +0200 @@ -1987,9 +1987,8 @@ # Append the tar header and data to the archive. if tarinfo.isreg(): -f = bltn_open(name, rb) -self.addfile(tarinfo, f) -f.close() +with bltn_open(name, rb) as f: +self.addfile(tarinfo, f) elif tarinfo.isdir(): self.addfile(tarinfo) @@ -2197,10 +2196,11 @@ Make a file called targetpath. source = self.extractfile(tarinfo) -target = bltn_open(targetpath, wb) -copyfileobj(source, target) -source.close() -target.close() +try: +with bltn_open(targetpath, wb) as target: +copyfileobj(source, target) +finally: +source.close() def makeunknown(self, tarinfo, targetpath): Make a file from a TarInfo object with an unknown type diff -r e9af9b1ca67e Lib/tarfile.py --- a/Lib/tarfile.pySat Nov 17 19:18:10 2012 + +++ b/Lib/tarfile.pySat Nov 17 21:37:45 2012 +0200 @@ -1924,9 +1924,8 @@ # Append the tar header and data to the archive. if tarinfo.isreg(): -f = bltn_open(name, rb) -self.addfile(tarinfo, f) -f.close() +with bltn_open(name, rb) as f: +self.addfile(tarinfo, f) elif tarinfo.isdir(): self.addfile(tarinfo) @@ -2131,16 +2130,15 @@ source = self.fileobj source.seek(tarinfo.offset_data) -target = bltn_open(targetpath, wb) -if tarinfo.sparse is not None: -for offset, size in tarinfo.sparse: -target.seek(offset) -copyfileobj(source, target, size) -else: -copyfileobj(source, target, tarinfo.size) -target.seek(tarinfo.size) -target.truncate() -target.close() +with bltn_open(targetpath, wb) as target: +if tarinfo.sparse is not None: +for offset, size in tarinfo.sparse: +target.seek(offset) +copyfileobj(source, target, size) +else: +copyfileobj(source, target, tarinfo.size) +target.seek(tarinfo.size) +target.truncate() def makeunknown(self, tarinfo, targetpath): Make a file from a TarInfo object with an unknown type ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Terry J. Reedy added the comment: Would with source = self.extractfile(tarinfo): work? -- nosy: +terry.reedy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
New submission from Sam Thursfield: Exceptions such as disk full during extraction cause tarfile to leak file handles. Besides being messy, it causes real problems if, for example, the target file is on a mount that should be unmounted before the program exits - in this case, the unmount will fail as there are still open file handles. Simplest solution I can see is to change: def makefile(self, tarinfo, targetpath): Make a file called targetpath. source = self.extractfile(tarinfo) target = bltn_open(targetpath, wb) copyfileobj(source, target) source.close() target.close() to this: def makefile(self, tarinfo, targetpath): Make a file called targetpath. source = self.extractfile(tarinfo) try: with open(targetpath, wb) as target: shutil.copyfileobj(source, target) finally: source.close() -- components: Library (Lib) messages: 175616 nosy: ssam priority: normal severity: normal status: open title: tarfile fails to close file handles in case of exception type: behavior versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Sam Thursfield added the comment: sorry, replace 'open' with 'bltn_open' in the above comment - no need to change it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16477] tarfile fails to close file handles in case of exception
Serhiy Storchaka added the comment: Issue16408 is similar issue for zipfile. -- keywords: +easy nosy: +lars.gustaebel, serhiy.storchaka stage: - needs patch type: behavior - resource usage versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 -Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16477 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com