[Python-Dev] Re: A better way to freeze modules

2021-09-13 Thread Brett Cannon
On Sat, Sep 11, 2021 at 7:08 PM Gregory Szorc wrote: > Thanks for all the replies, everyone! I'll reply to a few comments > individually. But I first wanted to address the common theme around > zipimport. > > First, one idea that nobody mentioned (and came to me after reading the > replies) was

[Python-Dev] Re: A better way to freeze modules

2021-09-11 Thread Gregory Szorc
Thanks for all the replies, everyone! I'll reply to a few comments individually. But I first wanted to address the common theme around zipimport. First, one idea that nobody mentioned (and came to me after reading the replies) was to possibly leverage zipimport for freezing the standard library

[Python-Dev] Re: A better way to freeze modules

2021-09-03 Thread Eric Snow
On Fri, Sep 3, 2021 at 5:32 AM Paul Moore wrote: > On Fri, 3 Sept 2021 at 10:29, Simon Cross > wrote: > > I think adding a meta path importer that reads from a standard > > optimized format could be a great addition. > > I think the biggest open question would be "what benefits does this > have

[Python-Dev] Re: A better way to freeze modules

2021-09-03 Thread Eric Snow
On Thu, Sep 2, 2021 at 10:46 PM Gregory Szorc wrote: > Over in https://bugs.python.org/issue45020 there is some exciting work around > expanding the use of the frozen importer to speed up Python interpreter > startup. I wholeheartedly support the effort and don't want to discourage > progress

[Python-Dev] Re: A better way to freeze modules

2021-09-03 Thread Paul Moore
On Fri, 3 Sept 2021 at 10:29, Simon Cross wrote: > > Hi Gregory, > > I think adding a meta path importer that reads from a standard > optimized format could be a great addition. I think the biggest open question would be "what benefits does this have over the existing zipimport?" Maybe it could

[Python-Dev] Re: A better way to freeze modules

2021-09-03 Thread Simon Cross
Hi Gregory, I think adding a meta path importer that reads from a standard optimized format could be a great addition. As you mentioned in your email, this is a big detour from the current start-up performance work, so I think practically the people working on performance are unlikely to take a

[Python-Dev] Re: A better way to freeze modules

2021-09-03 Thread Paul Moore
My quick reaction was somewhat different - it would be a great idea, but it’s entirely possible to implement this outside the stdlib as a 3rd party module. So the fact that no-one has yet done so means there’s less general interest than the OP is suggesting. And from my experience, the reason for

[Python-Dev] Re: A better way to freeze modules

2021-09-02 Thread Guido van Rossum
Quick reaction: This feels like a bait and switch to me. Also, there are many advantages to using a standard format like zip (many formats are really zip with some conventions). Finally, the bytecode format you are using is “marshal”, and is fully portable — as is zip. On Thu, Sep 2, 2021 at