[Python-Dev] Worried about Python release schedule and lack of stable C-API

2021-09-27 Thread Stephen J. Turnbull
jack.jan...@cwi.nl writes: > I’m getting increasingly worried about the future of Python, You mean about the fact that Python has a plethora (rapidly growing!) of powerful, efficient extension packages and you want to use them all (perhaps recursively)? :-) I really don't think this bodes ill

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-27 Thread Ronald Oussoren via Python-Dev
> On 26 Sep 2021, at 19:03, Christian Heimes wrote: > > On 26/09/2021 13.07, jack.jan...@cwi.nl wrote: >> The problem with the stable ABI is that very few developers are targeting >> it. I’m not sure why not, whether it has to do with incompleteness of the >> ABI, or with issues targeting it

[Python-Dev] The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
We've frozen most of the stdlib modules imported during "python -c pass" [1][2], to make startup a bit faster. Import of those modules is controlled by "-X frozen_modules=[on|off]". Currently it defaults to "off" but we'd like to default to "on". The blocker is the impact on contributors. I

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Chris Angelico
On Tue, Sep 28, 2021 at 3:21 AM Eric Snow wrote: > > On Mon, Sep 27, 2021 at 11:09 AM Chris Angelico wrote: > > When exactly does the freezing happen? > > When you build the executable (e.g. "make -j8", > ".\PCbuild\build.bat"). So your changes to those .py files wouldn't > show up until then.

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 10:51 AM Eric Snow wrote: > Possible solutions: > > 1. always default to "on" (the annoyance for contributors isn't big enough?) > 2. default to "on" if it's a PGO build (and "off" otherwise) > 3. default to "on" unless running from the source tree FWIW, I'm planning on

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Patrick Reader
How about checking each non-frozen module's hash and/or and comparing it to that of the frozen module? Would that defeat the performance improvement of freezing? Is it just a terrible idea? On 27/09/2021 17:51, Eric Snow wrote: > We've frozen most of the stdlib modules imported during "python

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Steve Dower
On 9/27/2021 6:40 PM, Patrick Reader wrote: I accidentally a word. "... module's hash and/or *timestamp* and comparing it ..." On 27/09/2021 18:38, Patrick Reader wrote: How about checking each non-frozen module's hash and/or and comparing it to that of the frozen module? Would that defeat

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Steve Dower
On 9/27/2021 7:15 PM, Ethan Furman wrote: On 9/27/21 10:50 AM, Guido van Rossum wrote: > So my *actual* proposal (call it #2') is to use a separate compile-time flag, which is set by `./configure > --enable-optimizations` regardless of whether PGO/LTO are possible, and which on Windows can

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Chris Angelico
On Tue, Sep 28, 2021 at 2:58 AM Eric Snow wrote: > > We've frozen most of the stdlib modules imported during "python -c > pass" [1][2], to make startup a bit faster. Import of those modules > is controlled by "-X frozen_modules=[on|off]". Currently it defaults > to "off" but we'd like to

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Guido van Rossum
On Mon, Sep 27, 2021 at 10:40 AM Steve Dower wrote: > On 9/27/2021 5:51 PM, Eric Snow wrote: > > Possible solutions: > > > > 1. always default to "on" (the annoyance for contributors isn't big > enough?) > > 2. default to "on" if it's a PGO build (and "off" otherwise) > > Just to air my concerns

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Ethan Furman
On 9/27/21 10:50 AM, Guido van Rossum wrote: > So my *actual* proposal (call it #2') is to use a separate compile-time flag, which is set by `./configure > --enable-optimizations` regardless of whether PGO/LTO are possible, and which on Windows can be set by > `PCbuild\build.bat --pgo` (we

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 11:09 AM Chris Angelico wrote: > When exactly does the freezing happen? When you build the executable (e.g. "make -j8", ".\PCbuild\build.bat"). So your changes to those .py files wouldn't show up until then. -eric ___

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Steve Dower
On 9/27/2021 5:51 PM, Eric Snow wrote: Possible solutions: 1. always default to "on" (the annoyance for contributors isn't big enough?) 2. default to "on" if it's a PGO build (and "off" otherwise) Just to air my concerns regarding option 2 (which I've already spoken to Eric about): I feel

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Patrick Reader
I accidentally a word. "... module's hash and/or *timestamp* and comparing it ..." On 27/09/2021 18:38, Patrick Reader wrote: > How about checking each non-frozen module's hash and/or and comparing it to > that of the frozen module? Would that defeat the performance improvement of > freezing?

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 12:40 PM Steve Dower wrote: > Having it be implied by an "--enable-optimizations" option is totally > fine (and we'd add one to build.bat for this), but I still think it > needs to be discoverable later whether the frozen modules build option > was used or not, independent

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Brett Cannon
On Mon, Sep 27, 2021 at 9:54 AM Eric Snow wrote: > We've frozen most of the stdlib modules imported during "python -c > pass" [1][2], to make startup a bit faster. Import of those modules > is controlled by "-X frozen_modules=[on|off]". Currently it defaults > to "off" but we'd like to default

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-27 Thread Brett Cannon
On Sun, Sep 26, 2021 at 3:51 AM Phil Thompson via Python-Dev < python-dev@python.org> wrote: > On 26/09/2021 05:21, Steven D'Aprano wrote: > > [snip] > > > These are not rhetorical questions, I genuinely do not know. I *think* > > that there was an attempt to make a stable C API back in 3.2

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Barry Warsaw
A couple of questions. If you’re planning a runtime -X option, then does that mean that the modules will be frozen at build time but Python will decide at runtime whether to use the frozen modules or the unfrozen ones? Are you planning on including the currently frozen importlib modules in

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 2:59 PM Brett Cannon wrote: > What about opting out when `--with-pydebug` is used? I'm not sure how many > people actively develop in a non-debug build other than testing something, > but at that point I would be having to run `make` probably anyway for > whatever I'm

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Victor Stinner
On Tue, Sep 28, 2021 at 12:52 AM Eric Snow wrote: > On Mon, Sep 27, 2021 at 3:31 PM Victor Stinner wrote: > > Which stdlib modules are currently frozen? If I really want to hack > > site.py or os.py for whatever reason, I just have to use "python3 -X > > frozen_modules=off"? > > The

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Victor Stinner
Hi Eric, Which stdlib modules are currently frozen? If I really want to hack site.py or os.py for whatever reason, I just have to use "python3 -X frozen_modules=off"? > 1. always default to "on" (the annoyance for contributors isn't big enough?) What is the annoyance? What is different between

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 3:04 PM Barry Warsaw wrote: > If you’re planning a runtime -X option, then does that mean that the modules > will be frozen at build time but Python will decide at runtime whether to use > the frozen modules or the unfrozen ones? Correct. FYI, this was already done. >

[Python-Dev] Re: The Default for python -X frozen_modules.

2021-09-27 Thread Eric Snow
On Mon, Sep 27, 2021 at 3:31 PM Victor Stinner wrote: > Which stdlib modules are currently frozen? If I really want to hack > site.py or os.py for whatever reason, I just have to use "python3 -X > frozen_modules=off"? The single-source-of-truth is Tools/scripts/freeze_modules.py. After running