On Fri, Sep 3, 2021 at 5:32 AM Paul Moore <p.f.mo...@gmail.com> wrote: > On Fri, 3 Sept 2021 at 10:29, Simon Cross <hodgestar+python...@gmail.com> > wrote: > > I think adding a meta path importer that reads from a standard > > optimized format could be a great addition. > > I think the biggest open question would be "what benefits does this > have over the existing zipimport?"
+1 > > As you mentioned in your email, this is a big detour from the current > > start-up performance work, so I think practically the people working > > on performance are unlikely to take a detour from their detour right > > now. > > Agreed, it would probably have to be an independent development > initially. If it delivers better performance, then switching the > startup work to use it would give a second set of performance > improvements, which no-one is going to object to. Similarly, if it's > simpler to manage, then the maintainability benefits could justify > switching over. +1 > > * Write the meta path importer in a separate package (it sounds like > > you've already done a lot of the work and gained a lot of > > understanding of the issues while writing PyOxidizer!) > > This is the key thing, though. The import machinery allows new > importers to be written as standalone modules, so I'd strongly > recommend that the proposed format/importer gets developed as a PyPI > module initially, with the PEP then being simply a proposal that the > module gets added to the stdlib and/or built into the interpreter. FWIW, I'm a big fan of folks taking advantage of the flexibility of the import machinery and writing importers like this (especially ones that folks must explicitly enable). As noted elsewhere, it would need to prove its worth before we consider putting it into importlib. > The key argument would be bootstrapping, IMO. I would definitely expect > interest in something like this to be lower if it's an external module > (needing a dependency to load your other dependencies is suboptimal). > Conversely, though, if no-one shows any interest in a PyPI version of > this idea, that would strongly imply that it's not as useful in > practice as you'd hoped. Excellent point! > In particular, I'd involve the maintainers of pyinstaller in the > design. If a new "frozen module importer" mechanism isn't of interest > to them, it's probably not going to get the necessary support to be > worth adding to the stdlib. +1 > On a personal note, I love the flexibility of Python's import system, > and I've always wanted to write importers for additional storage > formats (import from a sqlite database, for instance). But I've never > actually done so, because a zipfile is basically always sufficient for > any practical use case I've had. One day I hope to find a real use > case, though :-) Cool! I'd love to see what you make. -eric _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YV2K3BPVDZRZTGLM4HWQEJWMVPI6BGHD/ Code of Conduct: http://python.org/psf/codeofconduct/
