[Python-Dev] Re: Latest PEP 554 updates.

On Wed, May 6, 2020 at 2:25 PM Jeff Allen wrote: > Many thanks for working on this so carefully for so long. I'm happy to see > the per-interpreter GIL will now be studied fully before final commitment to > subinterpreters in the stdlib. I would have chipped in in those terms to the > review,

[Python-Dev] Re: Issues with import_fresh_module

Thanks for the suggestion. I think I tried something similar for tests that involved an environment variable and found that it doesn't play nicely with coverage.py /at all/. Also, I will have to solve this problem at some point anyway because the property tests for the module (not currently

[Python-Dev] Re: Issues with import_fresh_module

On Wed, May 6, 2020 at 2:34 PM Paul Ganssle wrote: > I think I tried something similar for tests that involved an environment > variable and found that it doesn't play nicely with coverage.py at all. This is a solvable problem: https://coverage.readthedocs.io/en/coverage-5.1/subprocess.html

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

06.05.20 00:46, Victor Stinner пише: Subinterpreters and multiprocessing have basically the same speed on this benchmark. It does not look like there are some advantages of subinterpreters against multiprocessing. I am wondering how much 3.9 will be slower than 3.8 in single-thread

[Python-Dev] Re: Issues with import_fresh_module

I'm drowning in work this month, so if you need me to look at something then I unfortunately need a point-blank link of what you want me to look at with a targeted question. As for import_fresh_module() not being robust: of course it isn't because it's mucking with import stuff in a very

[Python-Dev] A PEP PR that I closed until someone discusses context

I saw a PR on the PEP repository that looked like a joke here : https://github.com/python/peps/pull/1396 The author can give context to re-open if it was intentional. -- Best, Joannah Nanjekye *"You think you know when you learn, are more sure when you can write, even more when you can teach,

[Python-Dev] Re: A PEP PR that I closed until someone discusses context

On 06May2020 2204, joannah nanjekye wrote: I saw a PR on the PEP repository that looked like a joke here : https://github.com/python/peps/pull/1396 The author can give context to re-open if it was intentional. Given there isn't a real email address on the PEP, I'd assume it was meant as a

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

On Wed, May 6, 2020 at 10:03 AM Antoine Pitrou wrote: > > On Tue, 5 May 2020 18:59:34 -0700 > Nathaniel Smith wrote: > > On Tue, May 5, 2020 at 3:47 PM Guido van Rossum wrote: > > > > > > This sounds like a significant milestone! > > > > > > Is there some kind of optimized communication

[Python-Dev] Re: Latest PEP 554 updates.

On 05/05/2020 16:45, Eric Snow wrote: On Mon, May 4, 2020 at 11:30 AM Eric Snow wrote: Further feedback is welcome, though I feel like the PR is ready (or very close to ready) for pronouncement. Thanks again to all. FYI, after consulting with the steering council I've decided to change the

[Python-Dev] Re: Issues with import_fresh_module

The subtle issue is of course performance if you get too cozy with this pattern... On Wed, May 6, 2020 at 1:59 PM Nathaniel Smith wrote: > On Wed, May 6, 2020 at 7:52 AM Paul Ganssle wrote: > > > > As part of PEP 399, an idiom for testing both C and pure Python versions > of a library is

[Python-Dev] Re: Issues with import_fresh_module

Have you also considered changes to the modules under test that might make it easier for both implementations to exist and be tested side-by-side (so with fewer hacks on the testing side)? —Chris On Wed, May 6, 2020 at 2:33 PM Paul Ganssle wrote: > Thanks for the suggestion. > > I think I

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

On Wed, May 6, 2020 at 5:41 AM Victor Stinner wrote: > > Hi Nathaniel, > > Le mer. 6 mai 2020 à 04:00, Nathaniel Smith a écrit : > > As far as I understand it, the subinterpreter folks have given up on > > optimized passing of objects, and are only hoping to do optimized > > (zero-copy) passing

[Python-Dev] Re: Issues with import_fresh_module

On Wed, May 6, 2020 at 7:52 AM Paul Ganssle wrote: > > As part of PEP 399, an idiom for testing both C and pure Python versions of a > library is suggested making use if import_fresh_module. > > Unfortunately, I'm finding that this is not amazingly robust. We have this > issue:

[Python-Dev] Re: Issues with import_fresh_module

No worries, I actually seem to have solved the immediately pressing problem that was blocking PEP 615 by doing this: @functools.lru_cache(1) def get_modules():     import zoneinfo as c_module     py_module = import_fresh_module("zoneinfo", blocked=["_czoneinfo"])     return py_module, c_module

[Python-Dev] Re: A PEP PR that I closed until someone discusses context

On 5/6/2020 5:28 PM, Steve Dower wrote: On 06May2020 2204, joannah nanjekye wrote: I saw a PR on the PEP repository that looked like a joke here : https://github.com/python/peps/pull/1396 The author can give context to re-open if it was intentional. Given there isn't a real email address on

[Python-Dev] Re: PEP 617: New PEG parser for CPython

Hi Guido, Pablo & Lysandros, I'm excited about this improvement to Python, and was interested to hear about it at the language summit as well. I happen to be friends with Alessandro Warth, whom you cited in the PEP as developing the packrat parsing technique you use (at least in part). I wrote

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

On 06May2020 23:05, Serhiy Storchaka wrote: 06.05.20 00:46, Victor Stinner пише: Subinterpreters and multiprocessing have basically the same speed on this benchmark. It does not look like there are some advantages of subinterpreters against multiprocessing. Maybe I'm missing something,

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

Main memory bus or cache contention? Integer execution ports full? Throttling? VTune is useful to find out where the bottleneck is, things like that tend to happen when you start loading every logical core. On Tue, May 5, 2020 at 4:45 PM Joseph Jenne via Python-Dev < python-dev@python.org>

[Python-Dev] Issues with import_fresh_module

As part of PEP 399 , an idiom for testing both C and pure Python versions of a library is suggested making use if import_fresh_module. Unfortunately, I'm finding that this is not amazingly robust. We have this issue: https://bugs.python.org/issue40058,

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

Hi Nathaniel, Le mer. 6 mai 2020 à 04:00, Nathaniel Smith a écrit : > As far as I understand it, the subinterpreter folks have given up on > optimized passing of objects, and are only hoping to do optimized > (zero-copy) passing of raw memory buffers. I think that you misunderstood the PEP 554.

[Python-Dev] Maintainer for Multiprocessing

Hello, it looks like davin is no longer active and there is a pending merge request open more than two years. https://github.com/python/cpython/pull/4819 How can this get merged? Kind regards Philipp Rehs ___ Python-Dev mailing list --

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

Okay, an image is appearing. It sounds like GIL-free subinterpreters may one day shine because IPC is faster and simpler within one process than between multiple processes. This is not exactly what I got from PEP 554 but it is sufficient for me to have confidence in the project. On Wed, May 6,

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

On Tue, 5 May 2020 18:59:34 -0700 Nathaniel Smith wrote: > On Tue, May 5, 2020 at 3:47 PM Guido van Rossum wrote: > > > > This sounds like a significant milestone! > > > > Is there some kind of optimized communication possible yet between > > subinterpreters? (Otherwise I still worry that it's

[Python-Dev] Re: Maintainer for Multiprocessing

On Wed, 06 May 2020 09:42:02 - "Philipp Helo Rehs" wrote: > Hello, > > it looks like davin is no longer active and there is a pending merge request > open more than two years. > > https://github.com/python/cpython/pull/4819 For the record, I've punted on this for a while because reviewing

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

> On 5 May 2020, at 23:40, Guido van Rossum wrote: > > Is there some kind of optimized communication possible yet between > subinterpreters? (Otherwise I still worry that it's no better than > subprocesses -- and it could be worse because when one subinterpreter > experiences a hard crash