[issue25711] Rewrite zipimport from scratch

Change by Serhiy Storchaka : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Nick Coghlan added the comment: Noticed this was still open when reviewing Elvis's zipimport patch for issue 34022. Given the Python implementation has been merged, should we close this as resolved, and open new issues for any further changes (performance or otherwise)? -- nosy:

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: New changeset b2984ab9a7c458f8b7ed8978c0c95b109116895d by Serhiy Storchaka in branch 'master': bpo-25711: Remove outdated zipimport tests. (GH-9404) https://github.com/python/cpython/commit/b2984ab9a7c458f8b7ed8978c0c95b109116895d --

[issue25711] Rewrite zipimport from scratch

Change by Decorater : -- pull_requests: +8832 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: New changeset 9da3961f364da2a3ced740230b85ffb4309238d3 by Serhiy Storchaka in branch 'master': bpo-25711: Move _ZipImportResourceReader from importlib to zipimport. (GH-9406) https://github.com/python/cpython/commit/9da3961f364da2a3ced740230b85ffb4309238d3

[issue25711] Rewrite zipimport from scratch

Change by Serhiy Storchaka : -- pull_requests: +8828 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25711] Rewrite zipimport from scratch

Change by Serhiy Storchaka : -- pull_requests: +8827 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: New changeset 79d1c2e6c9d1bc1cf41ec3041801ca1a2b9a995b by Serhiy Storchaka in branch 'master': bpo-25711: Rewrite zipimport in pure Python. (GH-6809) https://github.com/python/cpython/commit/79d1c2e6c9d1bc1cf41ec3041801ca1a2b9a995b --

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: I'm personally in no rush and I assume Serhiy isn't either with 3.8 cut-off quite a ways out, so I see no rush. -- ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Barry A. Warsaw added the comment: Not sure if I'll have time before the core sprints to check the implementation with shiv, but I can give it a try then. Do you mind waiting until then? -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: I think Serhiy's PR is basically done, so now the question is do we want to merge it in and drop the C code? ;) I obviously say yes because this is I/O-bound code and so the switch shouldn't be enough of a performance hit to warrant blocking the gain in

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: Fixed issues on Windows. -- versions: +Python 3.8 -Python 3.7 ___ Python tracker ___ ___

[issue25711] Rewrite zipimport from scratch

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: I'm planning to review the PR at some point. On Sun, Jul 8, 2018, 00:42 Serhiy Storchaka, wrote: > > Serhiy Storchaka added the comment: > > Could anybody please make a review? This is just a first step, we need to > do it before making other steps:

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: Could anybody please make a review? This is just a first step, we need to do it before making other steps: implementing more modern import API, supporting large ZIP files and ZIP files with comments, implementing loading binary extensions from ZIP files,

[issue25711] Rewrite zipimport from scratch

Change by pmpp : -- nosy: +pmpp ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: PR 6809 freezes zipimport.py, adds changes made in zipimport.c since writing the initial Python implementation, and fixes few bugs exposed in the frozen module. Seems Python now works with the zipped stdlib. --

[issue25711] Rewrite zipimport from scratch

Change by Serhiy Storchaka : -- pull_requests: +6497 ___ Python tracker ___ ___

[issue25711] Rewrite zipimport from scratch

Decorater added the comment: Alright, just wanted to make sure because I did not want to have something break when loading up the entire standard library from an zip file with it. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: zipimport.py would be frozen just like importlib, so there's no bootstrapping issue if that's what you're asking. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Decorater added the comment: So, after reviewing this it started to make me rethink about the zipimport.py file. So my question is how would that file work if it is an pyc file in python37.zip or something just to zipimport other modules? There is got to be some sort

[issue25711] Rewrite zipimport from scratch

Barry A. Warsaw added the comment: I've ported the patch to a branch on GH. See PR 4023. I started from zipimport-2.patch. It doesn't work, but it will be easier to work on as a PR rather than a patch. Contributions welcome! Let's see if we can make this work.

[issue25711] Rewrite zipimport from scratch

Change by Barry A. Warsaw : -- pull_requests: +3998 ___ Python tracker ___ ___

[issue25711] Rewrite zipimport from scratch

Barry A. Warsaw added the comment: I've landed here after chatting with @brett.cannon. I have a use case for this (making pex startup faster by bypassing pkg_resources) but I need to hack around the limitation of dlopen'ing .so's from zips. Our idea was to have a

[issue25711] Rewrite zipimport from scratch

Change by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list

[issue25711] Rewrite zipimport from scratch

Yaron de Leeuw added the comment: What is the status of this work? Is there anything I can do to help make this happen? -- nosy: +jarondl ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: Just FYI, if this lands I will probably try to build off of it to make a pathlib-like zip module to eventually replace zipfile. So if there's any API design decisions that need to be made, it would be great if we try to keep the zip-specific bits separate and

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: And having a private copy of os.fsdecode() isn't difficult as os.fspath() is in posix and after that it's four lines that only need access to the sys module. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: > Technically, will it be possible to freeze it? I think yes. But I don't know how to do this. I hope on Brett's (or other import machinery expert) help. Since zipimporter constructor is called only with string path by import machinery, the os module is

[issue25711] Rewrite zipimport from scratch

Gregory P. Smith added the comment: So long as this code block that imports os is avoided, I believe that this can be properly frozen: +if not isinstance(path, str): +import os +path = os.fsdecode(path) But it should be easy to avoid that code path when the

[issue25711] Rewrite zipimport from scratch

STINNER Victor added the comment: > Here is preliminary translation of zipimport to Python. It is not frozen and > imports other modules. Technically, will it be possible to freeze it? It seems useful to keep the ability to put the whole stdlib into a single ZIP. Using a ZIP is sometimes

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: Got rid of dependencies from os, stat and encodings. -- Added file: http://bugs.python.org/file45814/zipimport-2.patch ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: Here is preliminary translation of zipimport to Python. It is not frozen and imports other modules. I tried to keep the implementation close to C implementation. As a consequence, some raised exceptions look arbitrary. -- keywords: +patch stage: ->

[issue25711] Rewrite zipimport from scratch

Rose Ames added the comment: Sorry for the late response. I didn't have much time over the holidays. I think I better let someone else take this one. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Changes by diana : -- nosy: +diana ___ Python tracker ___ ___ Python-bugs-list

[issue25711] Rewrite zipimport from scratch

STINNER Victor added the comment: Can you both publish your WIP work? -- nosy: +haypo ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: I were on very early stage, and stopped this work few weeks ago in favor of other issues. I would be glad to make a review of your work when you have finished it Rose. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Rose Ames added the comment: Serhiy, how far along are you on this? I have a wip from this summer that I could finish over the holidays. -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Changes by Eric Snow : -- nosy: +eric.snow, superluser ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: It was my intension. -- ___ Python tracker ___ ___ Python-bugs-list

[issue25711] Rewrite zipimport from scratch

Changes by Serhiy Storchaka : -- Removed message: http://bugs.python.org/msg255223 ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Changes by Serhiy Storchaka : -- Removed message: http://bugs.python.org/msg255222 ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: I was my intention. -- ___ Python tracker ___ ___ Python-bugs-list

[issue25711] Rewrite zipimport from scratch

New submission from Brett Cannon: No one wants to work on zipimport, and yet it's full of bugs. It needs a rewrite so that it's more maintainable. An idea floated at PyCon 2015 was to writing the zip-reading code in C and to keep it as simple as possible -- e.g., don't worry about supporting

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: FYI I'm at the early stage of rewriting zipimport in Python. -- nosy: +serhiy.storchaka ___ Python tracker ___

[issue25711] Rewrite zipimport from scratch

Changes by Robert Byrnes : -- nosy: +byrnes ___ Python tracker ___ ___ Python-bugs-list

[issue25711] Rewrite zipimport from scratch

Brett Cannon added the comment: Are you writing it in such a way that it can be bootstspped in with importlib so the stslib can be loaded from it? -- ___ Python tracker

[issue25711] Rewrite zipimport from scratch

Serhiy Storchaka added the comment: It was my intention. -- ___ Python tracker ___ ___ Python-bugs-list