[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Roundup Robot added the comment: New changeset 4dfe0634d11a by Serhiy Storchaka in branch '2.7': Issue #21560: An attempt to write a data of wrong type no longer cause https://hg.python.org/cpython/rev/4dfe0634d11a New changeset 6eb48b22ff5c by Serhiy Storchaka in branch '3.4': Issue #21560: An attempt to write a data of wrong type no longer cause https://hg.python.org/cpython/rev/6eb48b22ff5c -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Serhiy Storchaka added the comment: Tests are taken from issue23688. Thanks for your contribution Wolfgang. -- assignee: - serhiy.storchaka nosy: +serhiy.storchaka resolution: - fixed stage: test needed - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: needs patch - test needed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
New submission from Wolfgang Maier: I ran into this: gzout = gzip.open('test.gz','wb') gzout.write('abcdefgh') # write expects bytes not str Traceback (most recent call last): File pyshell#2, line 1, in module gzout.write('abcdefgh') File /usr/lib/python3.4/gzip.py, line 343, in write self.crc = zlib.crc32(data, self.crc) 0x TypeError: 'str' does not support the buffer interface gzout.write(b'abcdefgh') # ok, use bytes instead 8 gzout.close() But now the file is not recognized as valid gzip format anymore (neither by the gzip module nor by external software): gzin = gzip.open('test.gz','rb') next(gzin) Traceback (most recent call last): File pyshell#32, line 1, in module next(gzin) File /usr/lib/python3.4/gzip.py, line 594, in readline c = self.read(readsize) File /usr/lib/python3.4/gzip.py, line 365, in read if not self._read(readsize): File /usr/lib/python3.4/gzip.py, line 465, in _read self._read_eof() File /usr/lib/python3.4/gzip.py, line 487, in _read_eof raise OSError(Incorrect length of data produced) OSError: Incorrect length of data produced Turns out that gzip.write increased the ISIZE field value by 8 already during the failed call with the str object, so it is now 16 instead of 8: raw = open('test.gz','rb') [n for n in raw.read()] # ISIZE is the fourth last element [31, 139, 8, 8, 51, 46, 127, 83, 2, 255, 116, 101, 115, 116, 0, 75, 76, 74, 78, 73, 77, 75, 207, 0, 0, 80, 42, 239, 174, 16, 0, 0, 0] in other words: gzip.GzipFile.write() leaps (and modifies) before it checks its input argument. -- components: Library (Lib) messages: 218961 nosy: wolma priority: normal severity: normal status: open title: gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Changes by Serhiy Storchaka storch...@gmail.com: -- keywords: +easy stage: - needs patch versions: +Python 2.7, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Wolfgang Maier added the comment: ok, this seems to be really easy: patch attached -- keywords: +patch Added file: http://bugs.python.org/file35323/GzipFile_write.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Wolfgang Maier added the comment: or not - my patch just causes a different error in my example :( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Claudiu.Popa added the comment: Moving `self.crc = zlib.crc32(data, self.crc) 0x` before `self.size = self.size + len(data)` should be enough. Also, your patch needs a test. -- nosy: +Claudiu.Popa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string
Wolfgang Maier added the comment: isn't this exactly what I did in my patch ? actually, it is working, I just had an error in my preliminary test script. I may be able to work on an official test at some point, but definitely not over the next week -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21560 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com