On Fri, 3 Sept 2021 at 10:29, Simon Cross <hodgestar+python...@gmail.com> wrote: > > Hi Gregory, > > I think adding a meta path importer that reads from a standard > optimized format could be a great addition.
I think the biggest open question would be "what benefits does this have over the existing zipimport?" Maybe it could be a little faster? But would the downside of it not being possible to manage the format with existing standard tools outweigh that? A clear description of how to decide which is the most appropriate to use in a given situation between the new format and a zipfile would be a benefit here. > As you mentioned in your email, this is a big detour from the current > start-up performance work, so I think practically the people working > on performance are unlikely to take a detour from their detour right > now. Agreed, it would probably have to be an independent development initially. If it delivers better performance, then switching the startup work to use it would give a second set of performance improvements, which no-one is going to object to. Similarly, if it's simpler to manage, then the maintainability benefits could justify switching over. > * Ask if there are any Python core developers who would be willing to > look at the early stages of the code and/or PEP that you might produce > in the next couple of steps. Perhaps also ask on one of the packaging > mailing lists. If you get others involved as reviewers or contributors > from the start, convincing them later that it is a good idea will be > much easier. :) I'd be willing to look. I'm more interested in the design at this stage than in looking at code, as it's awfully easy to develop something that ends up being a "solution looking for a problem", so a solid case for having a general solution would be important for me. > * Write the meta path importer in a separate package (it sounds like > you've already done a lot of the work and gained a lot of > understanding of the issues while writing PyOxidizer!) This is the key thing, though. The import machinery allows new importers to be written as standalone modules, so I'd strongly recommend that the proposed format/importer gets developed as a PyPI module initially, with the PEP then being simply a proposal that the module gets added to the stdlib and/or built into the interpreter. The key argument would be bootstrapping, IMO. I would definitely expect interest in something like this to be lower if it's an external module (needing a dependency to load your other dependencies is suboptimal). Conversely, though, if no-one shows any interest in a PyPI version of this idea, that would strongly imply that it's not as useful in practice as you'd hoped. In particular, I'd involve the maintainers of pyinstaller in the design. If a new "frozen module importer" mechanism isn't of interest to them, it's probably not going to get the necessary support to be worth adding to the stdlib. > * Write a PEP. > > It seems to me that PEPs that come with an implementation and the > support of a few existing core developers have a much less painful PEP > review process. Agreed. In particular, existing code with a clearly demonstrated user base only has to persuade people that being in the core is important. Most proposals that I've seen which could be developed as a PyPI module never get anywhere because it turns out no-one is willing to do the work. You don't need to be a core developer to write a PyPI module, so if no-one has done that, it's likely to be either because the implementation needs tight integration into the core, or because nobody is actually as interested in the issue as you thought... On a personal note, I love the flexibility of Python's import system, and I've always wanted to write importers for additional storage formats (import from a sqlite database, for instance). But I've never actually done so, because a zipfile is basically always sufficient for any practical use case I've had. One day I hope to find a real use case, though :-) > Thank you for writing PyOxidizer and offering some of your time to > help make Python itself better. +1 Paul. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2THOX327G2SBUBNC4CEFK2JOH7VICFHC/ Code of Conduct: http://python.org/psf/codeofconduct/
