[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Nathaniel Smith
On Wed, Dec 15, 2021 at 3:07 AM Victor Stinner wrote: > I wrote https://bugs.python.org/issue39511 and > https://github.com/python/cpython/pull/18301 to have per-interpreter > None, True and False singletons. My change is backward compatible on > the C API: you can still use "Py_None" in your C

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Nathaniel Smith
On Wed, Dec 15, 2021 at 2:21 AM Antoine Pitrou wrote: > > On Wed, 15 Dec 2021 10:42:17 +0100 > Christian Heimes wrote: > > On 14/12/2021 19.19, Eric Snow wrote: > > > A while back I concluded that neither approach would work for us. The > > > approach I had taken would have significant cache

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-14 Thread Nathaniel Smith
Whoops, never mind, I see that you started the "immortal objects" thread to discuss this. On Tue, Dec 14, 2021 at 4:54 PM Nathaniel Smith wrote: > > How did you end up solving the issue where Py_None is a static global > that's exposed as part of the stable C ABI? > >

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-14 Thread Nathaniel Smith
How did you end up solving the issue where Py_None is a static global that's exposed as part of the stable C ABI? On Tue, Dec 14, 2021 at 9:13 AM Eric Snow wrote: > > Hi all, > > I'm still hoping to land a per-interpreter GIL for 3.11. There is > still a decent amount of work to be done but

[Python-Dev] Re: Python multithreading without the GIL

2021-10-08 Thread Nathaniel Smith
On Thu, Oct 7, 2021 at 7:54 PM Sam Gross wrote: > Design overview: > https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsDFosB5e6BfnXLlejd9l0/edit Whoa, this is impressive work. I notice the fb.com address -- is this a personal project or something facebook is working on? what's the

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-28 Thread Nathaniel Smith
On Tue, Sep 28, 2021 at 12:40 AM Guido van Rossum wrote: > > What I have heard repeatedly, from people who are paid to know, is that most > users don’t care about the latest features, and would rather stick to a > release until it becomes unsupported. (Extreme example: Python 2.) > > Numpy

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-28 Thread Nathaniel Smith
On Sun, Sep 26, 2021 at 3:38 AM wrote: > Open3D is an example. Will finally move to Python 3.9 some time the coming > month. Its dependency graph contains about 70 other packages. > > In this specific case, the underlying problem was that TensorFlow was stuck > at 3.8. The TensorFlow codebase

[Python-Dev] Re: Worried about Python release schedule and lack of stable C-API

2021-09-25 Thread Nathaniel Smith
On Sat, Sep 25, 2021 at 5:40 PM wrote: > PyPI packages and wheels are targeted to specific Python versions, which > means that any project that depends on some of the larger extension packages > (of which there are many, and many of which are must-have for many projects) > now start lagging

[Python-Dev] Re: python-dev thread w/ Marco Sulla

2021-08-16 Thread Nathaniel Smith
Was this post intended to go to python-dev or...? On Mon, Aug 16, 2021 at 9:53 AM Brett Cannon wrote: > > https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/ > > I can't be objective with Marco as I believe we have recorded issues with him >

[Python-Dev] Re: PEP 558, the simplest thing I could come up with

2021-07-29 Thread Nathaniel Smith
On Thu, Jul 29, 2021 at 4:52 PM Nick Coghlan wrote: > > On Fri, 30 Jul 2021, 6:05 am Mark Shannon, wrote: >> >> Hi Nick, >> >> Our discussion on PEP 558 got me thinking >> "What is the simplest thing that would work?". >> >> This is what I came up (in the form of a draft PEP): >>

[Python-Dev] Re: Roundup to GitHub Issues migration

2021-06-22 Thread Nathaniel Smith
On Tue, Jun 22, 2021 at 4:42 AM Sebastian Rittau wrote: > > Am 22.06.21 um 10:00 schrieb Tiziano Zito: > > I think it is important to notice that GitHub actively blocks user > > registration and activity from countries that are sanctioned by the US > > government. At least in 2019 GitHub was

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-20 Thread Nathaniel Smith
On Wed, May 19, 2021 at 7:28 PM Pablo Galindo Salgado wrote: >> >> Excellent point! Do you know how reliable this is in practice, i.e. >> what proportion of bytecode source spans are something you can >> successfully pass to ast.parse? If it works it's obviously nicer, but >> I can't tell how

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-19 Thread Nathaniel Smith
On Tue, May 18, 2021 at 2:49 PM Pablo Galindo Salgado wrote: > * It actually doesn't have more advantages. The current solution in the PEP > can do exactly the same as this solution if you allow reparsing when > displaying tracebacks. This is because with the start line, end line, start >

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Nathaniel Smith
On Mon, May 17, 2021 at 6:18 AM Mark Shannon wrote: > 2. Repeated binary operations on the same line. > > A single location can also be clearer when all the code is on one line. > > i1 + i2 + s1 > > PEP 657: > > i1 + i2 + s1 > > > Using a single location: > > i1 + i2 + s1 >

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Nathaniel Smith
On Fri, May 7, 2021 at 8:14 PM Neil Schemenauer wrote: > > On 2021-05-07, Pablo Galindo Salgado wrote: > > Technically the main concern may be the size of the unmarshalled > > pyc files in memory, more than the storage size of disk. > > It would be cool if we could mmap the pyc files and have the

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-05-06 Thread Nathaniel Smith
On Thu, May 6, 2021 at 2:17 AM Nathaniel Smith wrote: > > On Thu, Apr 29, 2021 at 9:14 AM Yury Selivanov > wrote: > > Nathaniel, at this point it's clear that this thread somehow does not > > help us understand what you want. Could you please just write your own > >

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-05-06 Thread Nathaniel Smith
On Thu, Apr 29, 2021 at 9:14 AM Yury Selivanov wrote: > Nathaniel, at this point it's clear that this thread somehow does not > help us understand what you want. Could you please just write your own > PEP clearly outlining your proposal, its upsides and downsides? > Without a PEP from you this

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-28 Thread Nathaniel Smith
On Fri, Apr 23, 2021 at 4:08 AM Irit Katriel wrote: > > On Fri, Apr 23, 2021 at 9:22 AM Nathaniel Smith wrote: >> I'm not trying to filibuster here -- I really want some form of EGs to >> land. > > I'm very glad to hear that. It's been hard to know where you stand, be

[Python-Dev] Re: Existing asyncio features scheduled for removal in Python 3.9 and 3.10

2021-04-26 Thread Nathaniel Smith
@asyncio.coroutine and @types.coroutine are different beasts. @asyncio.coroutine is the deprecated one; @types.coroutine is lower-level and not deprecated. On Mon, Apr 26, 2021 at 2:48 PM Luciano Ramalho wrote: > > I don't understand how it's possible to "Deprecate @coroutine for sake > of async

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-25 Thread Nathaniel Smith
On Fri, Apr 23, 2021 at 2:45 AM Chris Angelico wrote: > > On Fri, Apr 23, 2021 at 6:25 PM Nathaniel Smith wrote: > > The main possibility that I don't think we've examined fully is to > > make 'except' blocks fire multiple times when there are multiple > > exceptions. &

[Python-Dev] Anyone else gotten bizarre personal replies to mailing list posts?

2021-04-23 Thread Nathaniel Smith
? -- Forwarded message - From: Hoi lam Poon Date: Fri, Apr 23, 2021, 02:01 Subject: Re: [Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST] To: Nathaniel Smith Stop pretending, I can definitely get the key control file, your working group, all past actions and instructions

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-23 Thread Nathaniel Smith
On Wed, Apr 21, 2021 at 4:50 PM Guido van Rossum wrote: > On Wed, Apr 21, 2021 at 3:26 PM Nathaniel Smith wrote: >> Sure. This was in my list of reasons why the backwards compatibility >> tradeoffs are forcing us into awkward compromises. I only elaborated >> on it b/c i

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-21 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 3:15 AM Irit Katriel wrote: > On Tue, Apr 20, 2021 at 2:48 AM Nathaniel Smith wrote: >> >> >> The problem is that most of the time, even if you're using concurrency >> internally so multiple things *could* go wrong at once, only one thing >

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-21 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 2:15 PM srku...@mail.de wrote: > > So, forgive me my relatively simple mental model about ExceptionGroup. I > still try to create one for daily use. > > As noted in the discussion, an EG provides a way to collect exceptions from > different sources and raise them as a

[Python-Dev] Re: Keeping Python a Duck Typed Language.

2021-04-20 Thread Nathaniel Smith
On Tue, Apr 20, 2021 at 10:07 AM Mark Shannon wrote: > > Hi everyone, > > Once upon a time Python was a purely duck typed language. > > Then came along abstract based classes, and some nominal typing starting > to creep into the language. > > If you guarded your code with `isinstance(foo,

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-19 Thread Nathaniel Smith
On Mon, Apr 5, 2021 at 9:48 AM Irit Katriel wrote: > On Mon, Apr 5, 2021 at 11:01 AM Nathaniel Smith wrote: >> - I'm uncomfortable with how in some contexts we treat EG's as placeholders >> for the contained exceptions, and other places we treat them like a single >> f

[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-04-05 Thread Nathaniel Smith
OK, better late than never... here's a much-delayed review of the PEP. Thank you Irit and Guido for carrying this forward while I've been AWOL! It's fantastic to see my old design sketches turned into something like, actually real. == Overall feelings == Honestly, I have somewhat mixed feelings

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Nathaniel Smith
On Fri, Feb 26, 2021 at 5:05 AM Irit Katriel wrote: > I'm not sure it's safe to assume that it is necessarily a programming error, > and that the interpreter can essentially break the program in this case. > Is this not allowed? > > try: > try: > obj.func()# function that raises

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith
On Thu, Feb 25, 2021 at 10:23 PM Glenn Linderman wrote: > So then why do you need except* at all? Only to catch unwrapped > ExceptionGroup before it gets wrapped? > > So why not write except ExceptionGroup, and let it catch unwrapped > ExceptionGroup? > > That "CUTE BIT" could be done only

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith
On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum wrote: > > So is "fail-fast if you forget to handle an ExceptionGroup" really a feature? > (Do we call this out in the PEP?) > > We may believe that "except Exception" is an abuse, but it is too common to > dismiss out of hand. I think if some

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-01-28 Thread Nathaniel Smith
On Thu, Jan 28, 2021 at 9:03 PM Gregory P. Smith wrote: > > On Thu, Jan 28, 2021 at 10:52 AM Charalampos Stratakis > wrote: >> >> >> >> - Original Message - >> > From: "Mark Shannon" >> > To: "Python Dev" >> > Sent: Thursday, January 28, 2021 5:26:37 PM >> > Subject: [Python-Dev] Why

[Python-Dev] Re: Speeding up CPython

2020-10-22 Thread Nathaniel Smith
Hi Mark, This sounds really cool. Can you give us more details? Some questions that occurred to me while reading: - You're suggesting that the contractor would only be paid if the desired 50% speedup is achieved, so I guess we'd need some objective Python benchmark that boils down to a single

[Python-Dev] Re: PEP 515: Non-empty return statement

2020-08-24 Thread Nathaniel Smith
It was decided to leave out 'yield from' support for async generators, at least for now, due to the implementation complexity. And non-empty returns in generators are only intended for use with 'yield from', so they got left out as well. On Mon, Aug 24, 2020 at 4:48 PM Paul Bryan wrote: > > Per

[Python-Dev] Re: [Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

2020-06-29 Thread Nathaniel Smith
On Mon, Jun 29, 2020 at 5:04 AM Paul Sokolovsky wrote: > > Hello, > > On Mon, 29 Jun 2020 14:35:08 +0300 > "Jim F.Hilliard" wrote: > > > I believe I'm not the only one with this question but, how is Strunk & > > White connected with white supremacy? > > I wouldn't be surprised if the only

[Python-Dev] Re: [Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

2020-06-29 Thread Nathaniel Smith
On Mon, Jun 29, 2020 at 2:31 AM Steve Holden wrote: > The commit message used, however, reveals implementation details of the > change which are irrelevant to the stated aim, which is making the > documentation clear and concise. Use of such language is certainly > regrettable, since it

[Python-Dev] Re: Can we stop adding to the C API, please?

2020-06-03 Thread Nathaniel Smith
On Wed, Jun 3, 2020 at 2:10 PM Victor Stinner wrote: > For the short term, my plan is to make structure opaque in the limited > C API, before breaking more stuff in the public C API :-) But you're also breaking the public C API: https://github.com/MagicStack/immutables/issues/46

[Python-Dev] Re: Map errno==ETIME to TimeoutError

2020-05-25 Thread Nathaniel Smith
On Mon, May 25, 2020 at 1:25 AM Serhiy Storchaka wrote: > > 24.05.20 17:48, Eric V. Smith пише: > > Does anyone have an opinion on https://bugs.python.org/issue39673? It > > maps ETIME to TimeoutError, in addition to the already existing ETIMEDOUT. > > > >

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-08 Thread Nathaniel Smith
On Fri, May 8, 2020 at 12:30 AM Sebastian Krause wrote: > > Guido van Rossum wrote: > > Is there some kind of optimized communication possible yet between > > subinterpreters? (Otherwise I still worry that it's no better than > > subprocesses -- and it could be worse because when one > >

[Python-Dev] Re: Issues with import_fresh_module

2020-05-06 Thread Nathaniel Smith
On Wed, May 6, 2020 at 2:34 PM Paul Ganssle wrote: > I think I tried something similar for tests that involved an environment > variable and found that it doesn't play nicely with coverage.py at all. This is a solvable problem: https://coverage.readthedocs.io/en/coverage-5.1/subprocess.html

[Python-Dev] Re: Issues with import_fresh_module

2020-05-06 Thread Nathaniel Smith
On Wed, May 6, 2020 at 7:52 AM Paul Ganssle wrote: > > As part of PEP 399, an idiom for testing both C and pure Python versions of a > library is suggested making use if import_fresh_module. > > Unfortunately, I'm finding that this is not amazingly robust. We have this > issue:

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-06 Thread Nathaniel Smith
On Wed, May 6, 2020 at 10:03 AM Antoine Pitrou wrote: > > On Tue, 5 May 2020 18:59:34 -0700 > Nathaniel Smith wrote: > > On Tue, May 5, 2020 at 3:47 PM Guido van Rossum wrote: > > > > > > This sounds like a significant milestone! > > > > >

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-06 Thread Nathaniel Smith
On Wed, May 6, 2020 at 5:41 AM Victor Stinner wrote: > > Hi Nathaniel, > > Le mer. 6 mai 2020 à 04:00, Nathaniel Smith a écrit : > > As far as I understand it, the subinterpreter folks have given up on > > optimized passing of objects, and are only hoping to do optimized

[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround

2020-05-05 Thread Nathaniel Smith
On Tue, May 5, 2020 at 3:47 PM Guido van Rossum wrote: > > This sounds like a significant milestone! > > Is there some kind of optimized communication possible yet between > subinterpreters? (Otherwise I still worry that it's no better than > subprocesses -- and it could be worse because when

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

2020-04-28 Thread Nathaniel Smith
On Mon, Apr 20, 2020 at 6:21 PM Eric Snow wrote: > > Nathaniel, > > Your tone and approach to this conversation concern me. I appreciate > that you have strong feelings here and readily recognize I have my own > biases, but it's becoming increasingly hard to draw any constructive > insight from

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

2020-04-20 Thread Nathaniel Smith
On Mon, Apr 20, 2020 at 5:36 PM Edwin Zimmerman wrote: > > On 4/20/2020 7:33 PM, Nathaniel Smith wrote: > > On Mon, Apr 20, 2020 at 4:26 PM Edwin Zimmerman > > wrote: > >> On 4/20/2020 6:30 PM, Nathaniel Smith wrote: > >>> We already have robus

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

2020-04-20 Thread Nathaniel Smith
On Mon, Apr 20, 2020 at 4:26 PM Edwin Zimmerman wrote: > > On 4/20/2020 6:30 PM, Nathaniel Smith wrote: > > We already have robust support for threads for low-isolation and > > subprocesses for high-isolation. Can you name some use cases where > > neither of these are app

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

2020-04-20 Thread Nathaniel Smith
On Fri, Apr 17, 2020 at 3:57 PM Eric Snow wrote: > > On Fri, Apr 17, 2020 at 2:59 PM Nathaniel Smith wrote: > > I think some perspective might be useful here :-). > > > > The last time we merged a new concurrency model in the stdlib, it was > > asyncio. >

[Python-Dev] Re: PEP 554 for 3.9 or 3.10?

2020-04-17 Thread Nathaniel Smith
On Fri, Apr 17, 2020 at 11:50 AM Eric Snow wrote: > Dilemma > > > Many folks have conflated PEP 554 with having a per-interpreter GIL. > In fact, I was careful to avoid any mention of parallelism or the GIL > in the PEP. Nonetheless some are expecting that when PEP 554 lands we >

[Python-Dev] Re: Improvement to SimpleNamespace

2020-04-15 Thread Nathaniel Smith
On Wed, Apr 15, 2020 at 2:59 PM Ivan Pozdeev via Python-Dev wrote: > "Glom syntax" still excludes the delimiter, whatever it is, from use in keys. > So it's still a further limitation compared to the JSON spec. Glom does let you be specific about the exact lookup keys if you want, to handle

[Python-Dev] Re: Improvement to SimpleNamespace

2020-04-14 Thread Nathaniel Smith
On Tue, Apr 14, 2020 at 9:26 PM David Mertz wrote: > > I've written AttributeDict a fair number of times. Each time I write it from > scratch, which is only a few lines. And I only make a silly wore about 50% of > the time when I do so. I've also written it a number of times, and never found a

[Python-Dev] Re: PEP 617: New PEG parser for CPython

2020-04-02 Thread Nathaniel Smith
On Thu, Apr 2, 2020 at 2:48 PM Pablo Galindo Salgado wrote: > > > About the migration, can I ask who is going to (help to) fix projects > which rely on the AST? > > I think you misunderstood: The AST is exactly the same as the old and the new > parser. The only > the thing that the new parser

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-21 Thread Nathaniel Smith
On Sat, Mar 21, 2020 at 11:35 AM Steven D'Aprano wrote: > > On Fri, Mar 20, 2020 at 06:18:20PM -0700, Nathaniel Smith wrote: > > On Fri, Mar 20, 2020 at 11:54 AM Dennis Sweeney > > wrote: > > > This is a proposal to add two new methods, ``cutprefix`` and >

[Python-Dev] Re: Proliferation of tstate arguments.

2020-03-21 Thread Nathaniel Smith
On Fri, Mar 20, 2020 at 11:27 AM Victor Stinner wrote: > I would prefer to continue to experiment passing tstate explicitly in > internal C APIs until most blocker issues will be fixed. Once early > work on running two subinterpreters in parallel will start working > (one "GIL" per interpreter),

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

2020-03-20 Thread Nathaniel Smith
On Fri, Mar 20, 2020 at 11:54 AM Dennis Sweeney wrote: > This is a proposal to add two new methods, ``cutprefix`` and > ``cutsuffix``, to the APIs of Python's various string objects. The names should use "start" and "end" instead of "prefix" and "suffix", to reduce the jargon factor and for

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

2019-12-09 Thread Nathaniel Smith
> > On 09/12/2019 2:15 pm, Chris Angelico wrote: > You: "We should limit things. Stuff will be faster." > Others: "Really? Because bit masking is work. It'll be slower." > You: "Maybe we limit it somewhere else, whatever. It'll be faster." > Others: "Will it? How much faster?" > You: "It'll be

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

2019-12-06 Thread Nathaniel Smith
On Thu, Dec 5, 2019 at 5:38 AM Mark Shannon wrote: > From my limited googling, linux has a hard limit of about 600k file > descriptors across all processes. So, 1M is well past any reasonable > per-process limit. My impression is that the limits are lower on > Windows, is that right? Linux does

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

2019-12-03 Thread Nathaniel Smith
On Tue, Dec 3, 2019 at 8:20 AM Mark Shannon wrote: > The Python language does not specify limits for many of its features. > Not having any limit to these values seems to enhance programmer freedom, > at least superficially, but in practice the CPython VM and other Python > virtual > machines

[Python-Dev] Re: Exposing Tools/parser/unparse.py in the stdlib?

2019-11-19 Thread Nathaniel Smith
On Mon, Nov 18, 2019 at 4:41 PM Pablo Galindo Salgado wrote: > > Hi, > > What do people feel about exposing Tools/parser/unparse.py in the standard > library? Here is my initial rationale: > > * The tool already needs to be maintained and updated as is tested as part of > the test suite. > * I

[Python-Dev] Re: Pass the Python thread state to internal C functions

2019-11-17 Thread Nathaniel Smith
On Sun, Nov 17, 2019 at 1:58 PM Nick Coghlan wrote: > On Sat., 16 Nov. 2019, 8:26 am Nathaniel Smith, wrote: >> >> As you know, I'm skeptical that PEP 554 will produce benefits that are >> worth the effort, but let's assume for the moment that it is, and >> we're all 1

[Python-Dev] Re: Pass the Python thread state to internal C functions

2019-11-15 Thread Nathaniel Smith
As you know, I'm skeptical that PEP 554 will produce benefits that are worth the effort, but let's assume for the moment that it is, and we're all 100% committed to moving all globals into the threadstate. Even given that, the motivation for this change seems a bit unclear to me. I guess the

[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?

2019-09-23 Thread Nathaniel Smith
On Mon, Sep 23, 2019 at 1:30 PM Vinay Sajip via Python-Dev wrote: > > OK - but that's just one I picked at random. There are others like it - what > would be the process for deciding which ones need to be made private and > moved? Should an issue be raised to track this? There are really two

[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?

2019-09-23 Thread Nathaniel Smith
On Mon, Sep 23, 2019, 08:28 Vinay Sajip via Python-Dev < python-dev@python.org> wrote: > > requires some newer tools like -fvisibility=hidden that work > > differently across different platforms, and so far no-one's done the > > work to sort out the details. > > I've started looking at this, but

[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?

2019-09-20 Thread Nathaniel Smith
On Fri, Sep 20, 2019 at 2:58 PM Vinay Sajip via Python-Dev wrote: > > > > Right, I'm pretty sure that right now Python doesn't have any way to > > share symbols between .c files without also exposing them in the C > > API. > > On other C projects I've worked on, the public API is expressed in one

[Python-Dev] Re: static variables in CPython - duplicated _Py_IDENTIFIERs?

2019-09-20 Thread Nathaniel Smith
On Fri, Sep 20, 2019 at 1:00 PM Andrew Svetlov wrote: > This target is very important for keeping public API as small as possible. Right, I'm pretty sure that right now Python doesn't have any way to share symbols between .c files without also exposing them in the C API. This is fixable using

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Nathaniel Smith
On Fri, Aug 9, 2019 at 12:07 PM wrote: > > Eric V. Smith wrote: > > Hopefully the warnings in 3.9 would be more visible that what we saw in > > 3.7, so that library authors can take notice and do something about it > > before 3.10 rolls around. > > Eric > > Apologies for the ~double-post on the

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-06 Thread Nathaniel Smith
On Tue, Aug 6, 2019 at 3:44 PM Brett Cannon wrote: > I think this is a good example of how the community is not running tests with > warnings on and making sure that their code is warnings-free. This warning > has existed for at least one full release and fixing it doesn't require some > crazy

Re: [Python-Dev] PEP 595: Improving bugs.python.org

2019-05-31 Thread Nathaniel Smith
On Fri, May 31, 2019 at 11:39 AM Barry Warsaw wrote: > > On May 31, 2019, at 01:22, Antoine Pitrou wrote: > > > I second this. > > > > There are currently ~7000 bugs open on bugs.python.org. The Web UI > > makes a good job of actually being able to navigate through these bugs, > > search

Re: [Python-Dev] Should I postpone PEP 558 (locals() semantics) to Python 3.9?

2019-05-31 Thread Nathaniel Smith
I wouldn't mind having a little more breathing room. It's frustrating to miss the train, but these bugs are several decades old so I guess nothing terrible will happen if their fixes get delayed to 3.9. On Thu, May 30, 2019 at 4:23 PM Nick Coghlan wrote: > > Hi folks, > > The reference

Re: [Python-Dev] [SPAM?] Re: PEP 558: Defined semantics for locals()

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 5:24 PM Greg Ewing wrote: > > Terry Reedy wrote: > > I believe that the situation is or can be thought of as this: there is > > exactly 1 function locals dict. Initially, it is empty and inaccessible > > (unusable) from code. Each locals() call updates the dict to a

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 6:48 PM Greg Ewing wrote: > > Nathaniel Smith wrote: > > - [proxy]: Simply return the .f_locals object, so in all contexts > > locals() returns a live mutable view of the actual environment: > > > > def locals(): > > return get_

Re: [Python-Dev] [SPAM?] Re: PEP 558: Defined semantics for locals()

2019-05-28 Thread Nathaniel Smith
On Tue, May 28, 2019 at 6:02 PM Guido van Rossum wrote: > > On Tue, May 28, 2019 at 5:25 PM Greg Ewing > wrote: >> >> Terry Reedy wrote: >> > I believe that the situation is or can be thought of as this: there is >> > exactly 1 function locals dict. Initially, it is empty and inaccessible >> >

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
On Mon, May 27, 2019 at 9:18 PM Guido van Rossum wrote: > > Note that the weird, Action At A Distance behavior is also visible for > locals() called at module scope (since there, locals() is globals(), which > returns the actual dict that's the module's __dict__, i.e. the Source Of > Truth. So

Re: [Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
On Mon, May 27, 2019 at 9:16 AM Guido van Rossum wrote: > > I re-ran your examples and found that some of them fail. > > On Mon, May 27, 2019 at 8:17 AM Nathaniel Smith wrote: [...] >> The interaction between f_locals and and locals() is also subtle: >> >> def f(

[Python-Dev] [PEP 558] thinking through locals() semantics

2019-05-27 Thread Nathaniel Smith
First, I want to say: I'm very happy with PEP 558's changes to f_locals. It solves the weird threading bugs, and exposes the fundamental operations you need for debugging in a simple and clean way, while leaving a lot of implementation flexibility for future Python VMs. It's a huge improvement

Re: [Python-Dev] PEP 558: Defined semantics for locals()

2019-05-25 Thread Nathaniel Smith
On Sat, May 25, 2019, 07:38 Guido van Rossum wrote: > This looks great. > > I only have two nits with the text. > > First, why is the snapshot called a "dynamic snapshot"? What exactly is > dynamic about it? > It's dynamic in that it can spontaneously change when certain other events happen.

Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done (was: PEP 594: Removing dead batteries from the standard library)

2019-05-24 Thread Nathaniel Smith
On Thu, May 23, 2019 at 2:18 PM Brett Cannon wrote: > I'm personally viewing it as a first step in addressing the maintenance > burden we have with such a large stdlib. Christian started this work over a > year ago and I think it's worth seeing through. After that we should probably > have a

Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done

2019-05-24 Thread Nathaniel Smith
On Fri, May 24, 2019, 08:08 Ben Cail wrote: > > Why not have the PSF hire someone (or multiple people) to be paid to > work on the maintenance burden? This could be similar to the Django > fellows: > https://www.djangoproject.com/fundraising/#who-is-the-django-fellow. It > seems like a good

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-22 Thread Nathaniel Smith
On Wed, May 22, 2019, 04:32 Christian Heimes wrote: > On 22/05/2019 12.19, Steven D'Aprano wrote: > > I don't think this PEP should become a document about "Why you should > > use PAM". I appreciate that from your perspective as a Red Hat security > > guy, you want everyone to use best practices

Re: [Python-Dev] PEP 594: update 1

2019-05-22 Thread Nathaniel Smith
On Wed, May 22, 2019, 12:14 Sean Wallitsch wrote: > Dear python-dev, > > I'm writing to provide some feedback on PEP-594, primarily the proposed > deprecation and reason for the removal of the aifc and audioop libraries. > > The post production film industry continues to make heavy use of AIFFs,

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Nathaniel Smith
On Tue, May 21, 2019 at 4:25 AM Victor Stinner wrote: > > Le mar. 21 mai 2019 à 13:18, André Malo a écrit : > > There's software in production using both. (It doesn't mean it's on pypi or > > even free software). > > > > What would be the maintenance burden of those modules anyway? (at least for

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

2019-05-21 Thread Nathaniel Smith
On Tue, May 21, 2019 at 10:43 AM Glenn Linderman wrote: > After maintaining my own version of http.server to fix or workaround some of > its deficiencies for some years, I discovered bottle.py. It has far more > capability, is far better documented, and is just as quick to deploy. While I >

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 1:23 PM Victor Stinner wrote: > > Le jeu. 16 mai 2019 à 20:58, Petr Viktorin a écrit : > > I always thought the classic (exc_type, exc_value, exc_tb) triple is a > > holdover from older Python versions, and all the information is now in > > the exception instance. > > Is

Re: [Python-Dev] Parser module in the stdlib

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 2:13 PM Pablo Galindo Salgado wrote: > I propose to remove finally the parser module as it has been "deprecated" for > a long time, is almost clear that nobody uses it and has very limited > usability and replace it (maybe with a different name) > with pgen2 (maybe with

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019 at 2:17 PM Steve Dower wrote: > You go on to say "pass an error message" and "keep repr(obj) if you > want", but how is this different from creating an exception that > contains the custom message, the repr of the object, and chains the > exception that triggered it? A

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Nathaniel Smith
On Thu, May 16, 2019, 09:07 Steve Dower wrote: > > Actually, if the default implementation prints the exception message, > how is this different from sys.excepthook? Specifically, from the point > of customizing the hooks. > sys.excepthook means the program has fully unwound and is about to

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-15 Thread Nathaniel Smith
On Wed, May 15, 2019 at 6:25 PM Victor Stinner wrote: > I proposed a different approach: add a new sys.unraisablehook hook > which is called to handle an "unraisable exception". To handle them > differently, replace the hook. For example, I wrote a custom hook to > log these exceptions into a

Re: [Python-Dev] deprecation of abstractstaticmethod and abstractclassmethod

2019-05-15 Thread Nathaniel Smith
I don't care about the deprecation either way. But can we fix the individual decorators so both orders work? Even if it requires a special case in the code, it seems worthwhile to remove a subtle user-visible footgun. On Wed, May 15, 2019, 12:39 Ethan Furman wrote: > In issue 11610*

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-29 Thread Nathaniel Smith
On Mon, Apr 29, 2019 at 5:01 PM Neil Schemenauer wrote: > As far as I understand, we have a similar problem already for > gc.get_objects() because those static type objects don't have a > PyGC_Head. My 2-cent proposal for fixing things in the long term > would be to introduce a function like

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-27 Thread Nathaniel Smith
On Sat, Apr 27, 2019, 04:27 Armin Rigo wrote: > Hi Neil, > > On Wed, 24 Apr 2019 at 21:17, Neil Schemenauer > wrote: > > Regarding the Py_TRACE_REFS fields, I think we can't do them without > > breaking the ABI because of the following. For GC objects, they are > > always allocated by

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-25 Thread Nathaniel Smith
You don't necessarily need rpath actually. The Linux loader has a bug/feature where once it has successfully loaded a library with a given soname, then any future requests for that soname within the same process will automatically return that same library, regardless of rpath settings etc. So as

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

2019-04-23 Thread Nathaniel Smith
On Tue, Apr 23, 2019, 05:09 Andrew Svetlov wrote: > I agree that `from typing import TYPE_CHECKING` is not desirable from > the import time reduction perspective. > > From my understanding code completion *can* be based on type hinting > to avoid actual code execution. > That's why I've

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

2019-04-20 Thread Nathaniel Smith
On Sat, Apr 20, 2019 at 2:11 PM Inada Naoki wrote: > > "import typing" is slow too. Many static analysis tools will also accept: TYPE_CHECKING = False if TYPE_CHECKING: ... At least mypy and pylint both treat all variables named TYPE_CHECKING as true, regardless of where they came from.

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-16 Thread Nathaniel Smith
On Mon, Apr 15, 2019 at 8:58 PM Michael Sullivan wrote: > > On Mon, Apr 15, 2019 at 4:06 PM Nathaniel Smith wrote: >> >> On Mon, Apr 15, 2019, 15:27 Michael Sullivan wrote: >>> >>> > The main question is if anyone ever used Py_TRACE_REFS

Re: [Python-Dev] PEP 591 discussion (final qualifier) happening at typing-sig@

2019-04-15 Thread Nathaniel Smith
On Mon, Apr 15, 2019 at 5:00 PM Michael Sullivan wrote: > > I've submitted PEP 591 (Adding a final qualifier to typing) for discussion to > typing-sig [1]. I'm not on typing-sig [1] so I'm replying here. > Here's the abstract: > This PEP proposes a "final" qualifier to be added to the

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-15 Thread Nathaniel Smith
On Mon, Apr 15, 2019, 15:27 Michael Sullivan wrote: > > The main question is if anyone ever used Py_TRACE_REFS? Does someone > > use sys.getobjects() or PYTHONDUMPREFS environment variable? > > I used sys.getobjects() today to track down a memory leak in the > mypyc-compiled version of mypy. > >

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Fri, Apr 12, 2019 at 5:05 PM Steve Dower wrote: > > On 12Apr.2019 1643, Nathaniel Smith wrote: > > On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > >> > >> On 10Apr2019 1917, Nathaniel Smith wrote: > > I don't know how many people use Py_TRACE_R

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Nathaniel Smith
On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: > > On 10Apr2019 1917, Nathaniel Smith wrote: > > It sounds like --with-pydebug has accumulated a big grab bag of > > unrelated features, mostly stuff that was useful at some point for > > some CPython dev trying to de

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-11 Thread Nathaniel Smith
On Thu, Apr 11, 2019 at 8:32 AM Serhiy Storchaka wrote: > On other hand, since using the debug allocator doesn't cause problems > with compatibility, it may be possible to use similar technique for the > objects double list. Although this is not easy because of objects placed > at static memory.

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Nathaniel Smith
On Wed, Apr 10, 2019 at 1:50 PM Steve Dower wrote: > > On 10Apr2019 1227, Nathaniel Smith wrote: > > On Wed, Apr 10, 2019, 04:04 Victor Stinner > <mailto:vstin...@redhat.com>> wrote: > > I don't think that I ever used sys.getobjects(), whereas many projects >

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Nathaniel Smith
On Wed, Apr 10, 2019, 04:04 Victor Stinner wrote: > Le mar. 9 avr. 2019 à 22:16, Steve Dower a écrit > : > > What are the other changes that would be required? > > I don't know. > > > And is there another > > way to get the same functionality without ABI modifications? > > Py_TRACE_REFS is a

  1   2   3   4   5   >