[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Serhiy Storchaka added the comment: The reason of using bytes concatenating rather than accumulating in the list, is that in most cases one of arguments is an empty bytes object (appending to the empty buffer or uncompressing a file with large compression block), and this case is optimized in

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Changes by Serhiy Storchaka : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker ___

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Shubha Ramani added the comment: Upon comparing the PyPy changes from attached diff to the latest cpython3 github, I don't find a need for improvement. Looks like cpython3 zipfile.py has the same changes and the read() method in class ZipExtFile(io.BufferedIOBase) is vastly improved compared

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Serhiy Storchaka added the comment: I'm not the initial author of zipfile. That initial code for decrypting was added in issue698833. I like your idea about replacing the generating code with the precomputed table. Please open a separate issue for this. --

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Alecsandru Patrascu added the comment: Serhiy, I am curious why did you have chosen to compute the CRC32 table everytime? It is standard (the generator polynomial does not change) and always will output the same values. And it is also less computational intensive to loading a precomputed

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Shubha Ramani added the comment: Serhiy yes what you said makes sense. Thanks for clarifying. Updates (benchmarking results) shortly...stay tuned. -- ___ Python tracker

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Serhiy Storchaka added the comment: I assigned this issue to me as the maintainer of the zipfile module. This means that I undertake to examine the results of benchmarking, make the review of the proposed patch, and merge it if it improves the performance and doesn't have negative side

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Shubha Ramani added the comment: serhiy sure I will attach proof of the performance bottle-neck on 2.7 and 3.7 before I submit a patch. Please assign this bug to me. -- ___ Python tracker

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Shubha Ramani added the comment: Please assign this bug to me. I will submit a patch -- ___ Python tracker ___

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Serhiy Storchaka added the comment: Could you provide a benchmarkin script that demonstrates a performance bottleneck? The patch is against 2.7. Please provide a patch against the master branch. -- ___ Python tracker

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

Changes by Serhiy Storchaka : -- assignee: -> serhiy.storchaka components: +Library (Lib) nosy: +alanmcintyre, serhiy.storchaka, twouters ___ Python tracker

[issue30468] Propagate zipfile.py pypy issue #905 patch to CPython 3.7

New submission from Shubha Ramani: PyPy had a longstanding issue : ZipFile.extractall is very slow compared to CPython 2.6 https://bitbucket.org/pypy/pypy/issues/905/zipfileextractall-is-very-slow-compared-to which has been fixed in the PyPy code base. The changes were entirely in zipfile.py