[Python-Dev] Re: Improve CPython tracing performance
On Fri, 30 Oct 2020 at 23:34, Victor Stinner wrote: > > Le ven. 30 oct. 2020 à 11:02, Nick Coghlan a écrit : > > > Ok, I've created https://bugs.python.org/issue42197 to track it. > > > > Please also have a look at PEP 558 and its draft reference > > implementation at https://github.com/python/cpython/pull/3640 > > I don't think that the PEP 558 and bpo-42197 are incompatible. > > Debuggers and profilers usually only care of specific frames or > function calls (ex: 10% of function calls or even a single function > call in a whole application). The problem is how to make them as > efficient as possible for "no operation" calls, when they don't care > about the current frame. Avoiding PyFrame_FastToLocalsWithError() to > enter the debugger/profile and avoiding PyFrame_LocalsToFast() on exit > sounds a simple and practical solution. Aye, I agree. I just don't think we can remove those implicit calls without preparing a replacement API first. > By the way, I created https://bugs.python.org/issue40421 to prepare > the C API to make the PyFrameObject structure opaque. Once it will be > opaque, we will have more freedom on the API exposed to inspect frame > locals. > > IMO it's a good idea to require function calls to inspect frame > locals, and not let developers think that PyFrameObject.f_locals is > always up-to-date and can always be modified. Aye, if direct access to the PyFrameObject.f_locals struct field goes away, then the cache refresh can be added to the accessor APIs, just as PEP 558 proposes doing for Python code. > The good news is that PyFrame_GetLocals() and PyFrame_SetLocals() can > easily be reimplemented in Python 3.9 for my compatibility header > file: > https://github.com/pythoncapi/pythoncapi_compat > > Such API avoids a complex proxy and simply reuses a regular dict > object (exiting PyFrameObject.f_locals). Unfortunately, cell variables mean that there's no way to make snapshot-with-writeback logic consistently correct in the presence of generators and coroutines (see https://www.python.org/dev/peps/pep-0558/#resolving-the-issues-with-tracing-mode-behaviour for the gist of that problem). The early iterations of PEP 558 worked that way (like you, I wanted to avoid having to actually implement the write-through proxy), but Nathaniel kept finding holes where bugs like bpo-30744 could still happen (hence the acknowledgement in the PEP), so I eventually conceded defeat and accepted that the proxy was the only approach that could ever be truly correct. The core of the write-through proxy implementation isn't actually too bad: https://github.com/python/cpython/blob/ed6e53be7794ec94924b69cd46d5b009633c6307/Objects/frameobject.c#L1315 The annoying parts are the outright copy-and-paste of odict code at the end of the file, and the few remaining mutable mapping methods that still raise NotImplementedError rather than doing what you'd expect. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/5OXVNEOSD5AZBSSDU23Y4UI45VOVRUHT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
Hi folks, This is a mailing list repost of the Discourse thread at https://discuss.python.org/t/pep-642-constraint-pattern-syntax-for-structural-pattern-matching/5614 The rendered version of the PEP can be found here: https://www.python.org/dev/peps/pep-0642/ The full text is also quoted in the Discourse thread. The remainder of this email is the same introduction that I posted on Discourse. I’m largely a fan of the Structural Pattern Matching proposal in PEP 634, but there’s one specific piece of the syntax proposal that I strongly dislike: the idea of basing the distinction between capture patterns and value patterns purely on whether they use a simple name or a dotted name. Thus PEP 642, which retains most of PEP 634 unchanged, but adjusts value checks to use an explicit prefix syntax (either `?EXPR` for equality constraints, or `?is EXPR` for identity constraints), rather than relying on users learning that literals and attribute lookups in a capture pattern mean a value lookup check, while simple names mean a capture pattern (unlike both normal expressions, where all three mean a value lookup, and assignment targets, where both simple and dotted names bind a new reference). The PEP itself has a lot of words explaining why I’ve made the design decisions I have, as well as the immediate and potential future benefits offered by using an explicit prefix syntax for value constraints, but the super short form goes like this: * if you don’t like match statements at all, or wish we were working on designing a C-style switch statement instead, then PEP 642 isn’t going to appeal to you any more than PEP 634 does * if, like me, you don’t like the idea of breaking the existing property of Python that caching the result of a value lookup subexpression in a local variable and then using that variable in place of the original subexpression should “just work”, then PEP 642’s explicit constraint prefix syntax may be more to your liking * however, if the idea of the `?` symbol becoming part of Python’s syntax doesn’t appeal to you, then you may consider any improved clarity of intent that PEP 642 might offer to not be worth that cost Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/WT3ZZ42XJ4G6Y3H26RWUFN5GCUIFLMQ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Thoughts on PEP 634 (Structural Pattern Matching)
On Sat, 31 Oct 2020 at 00:44, Mark Shannon wrote: > Should match be an expression, or a statement? > -- > > Do we want a fancy switch statement, or a powerful expression? > Expressions have the advantage of not leaking (like comprehensions in > Python 3), but statements are easier to work with. PEP 635 already includes a good rationale for using a statement in https://www.python.org/dev/peps/pep-0635/#the-match-statement > Can pattern matching make it clear what is assigned? > > > Embedding the variables to be assigned into a pattern, makes the pattern > concise, but requires discarding normal Python syntax and inventing a > new sub-language. Could we make patterns fit Python better? > > Is it possible to make assignment to variables clear, and unambiguous, > and allow the use of symbolic constants at the same time? > I think it is, but PEP 634 fails to do this. This concern I agree with, and just posted PEP 642 as a proposal to address it by requiring an explicit syntactic prefix on any value lookups that occur in a match pattern. > How should pattern matching be integrated with the object model? > > > What special method(s) should be added? How and when should they be called? > PEP 634 largely disregards the object model, meaning it has many special > cases, and is inefficient. PEP 642 eliminates at least some of the special cases. For the special cases that remain, I think we'll be OK to tackle generalisation as a "later" concern (similar to the way we lived without a Path-like protocol for a long time to allow objects other than strings in filesystem APIs). (As Brandt noted, the early iterations of PEP 622 did include a match customisation protocol, and the authors were persuaded to drop it and adopt a "Let's wait and see how far we get with just `__match_args__`, `__eq__`, and `ABC.register()` before adding anything else" approach instead) > The semantics must be well defined. > --- > > Language extensions PEPs should define the semantics of those > extensions. For example, PEP 343 and PEP 380 both did. > https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement > https://www.python.org/dev/peps/pep-0380/#formal-semantics > > PEP 634 just waves its hands and talks about undefined behavior, which > horrifies me. Is this in relation to the evaluation-order caveat, and the impact of using non-pure operations inside patterns? Even for with statements, we've tinkered with the exact timing of the __enter__ and __exit__ method lookups over the years in order to tweak exactly which exception gets thrown when both are missing (the original implementation followed the PEP in looking up "__exit__" first, as it used multiple CPython bytecodes, and wanted to be able to use "__enter__" immediately after loading it on to the stack. The current implementation instead looks up "__enter__" first, since that's the error users expect to see when both are missing, and we now have a dedicated SETUP_WITH opcode that made the change simple to implement). The problem with providing that level of specificity in PEP 634 is that fully mandating a specific evaluation order for all subpatterns and guard subexpressions means that potentially interesting optimisation opportunities for match statement evaluation could be ruled out. Even if the initial pattern matching implementation uses a straightforward top to bottom, left to right approach where each pattern is checked for a structural match, re-evaluating repeated subexpressions each time they appear, then its associated guard expression evaluated, it makes sense to allow for more clever implementations that aggressively cache repeated subexpressions in the structural matching patterns (which are supposed to be side-effect free, even though we have no way to enforce that restriction), and inspect the guard expressions for subclauses that can be evaluated early and potentially skip the full structural match if the guard clause becomes necessarily false (while abiding by the requirement that the guard expressions still follow normal left-to-right evaluation rules - the subexpression evaluations are just allowed to be interleaved with the evaluation of subpatterns in the structural matching part by a sufficiently capable compiler). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/EW3GQ5GLUXZKYCU672BXS46IYT57NAHJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636
Hello,
On Sat, 31 Oct 2020 02:37:14 +0100
Tin Tvrtković wrote:
[]
> async def ws_proxy(client: WebSocket):
> await client.accept()
> async with ClientSession() as session:
> async with session.ws_connect("wss://echo.websocket.org") as
> s: c = starlette_websocket_iterator(client)
> async for r in wait_on(c, s):
> match r:
> case (src, None):
> print(f"{src} closed the connection")
> break
> case (src, msg) if src is c:
> print(f"CLIENT: {msg}")
> await s.send_str(msg)
> case (src, msg) if src is s:
> print(f"SERVER: {msg}")
> await client.send_text(msg.data)
>
> So yeah, the helper yields tuples of the source and message, using
> None as a sentinel for closure. Guards are used to match on the
> source, using iterator identity.
> My first version just used `case (s, msg):` hoping to match on the
> identity of s, but that doesn't work since s is not a literal.
>
> ..., I'd say
> this style of programming is quite readable and understandable.
It baffled me how the first sentence contradicts the second.
So, with Mark Shannon's recent (re)criticism of pattern matching PEP,
in particular "Can pattern matching make it clear what is assigned?"
anyone wants to go 2nd round on "sigils aren't that bad,
especially comparing to what we're offered otherwise"?
E.g. would it really be that bad to have:
match r:
case (@src, None):
case (c, @msg):
case (s, @msg):
Alternative syntax, arguably, even newbie can pick it up what it does:
match r:
case (>src, None):
case (c, >msg):
case (s, >msg):
Alternatively, can add "inline guard sigils" for the most common guard
conditions, which would be equality, period:
match r:
case (src, None):
case (==c, msg):
case (==s, msg):
[]
--
Best regards,
Paul mailto:[email protected]
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/FDSJOO6KLGSW7SAGBR4W2KHKHBVKVETB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636
On Sat, 31 Oct 2020 at 17:49, Paul Sokolovsky wrote: > Alternatively, can add "inline guard sigils" for the most common guard > conditions, which would be equality, period: > > match r: > case (src, None): > case (==c, msg): > case (==s, msg): `==` interacts poorly with other aspects of the pattern syntax (most notably keywords in class patterns), but PEP 642's identity constraint expressions would allow: match r: case (src, ?is None): ... # Any src, msg is None case (?is c, msg): ... # Any msg, src is c case (?is s, msg): ... # Any msg, src is s Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/MVJY42LZM7ITRMZ3TSHC23GOMKIS6FCJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python
On 10/30/2020 5:08 PM, Garrett D'Amore via Python-Dev wrote: I’m not on this list. But I have offered to help - if there are tasks that need to be done to help this I can help put the weight of a commercial entity behind it whether that involves assigning our developers to work on this, helping pay for external developers to do so, or assisting with access to machine resources. What’s more likely is that some group of developers aren’t interested in supporting stuff they don’t actively use. [snip] I think it is more of a matter of pride in putting out a quality product and not making claims that might be false. Anyway, here is what I posted on the issue. https://bugs.python.org/issue42173 """ Dear Solaris Python fans (about 20 so far here): Here is the situation. There are 1000s of open issues on this tracker and at least 100s of open cpython PRs, and only 20-30 core developers, mostly volunteers, actively (depending on definition) merging PRs and maybe another 10 triagers helping to manage issues. You all can help on issues by checking whether reported bugs still exist with current Python (3.8+) on current 'Solaris' (which includes what?). Searching open issues for 'Solaris' in 'All text' just returned 114 hits. About half were last touched over two years ago, and sometimes the last touch was inconsequential (a version or nosy change). I suspect many of the 114 are obsolete. For example, the last comment, five years ago, on #1471934, says the problem was fixed in Solaris 11.2. Does this mean the issue should be closed? The bottleneck for merging PRs is reviewing PRs. We coredevs cannot do enough reviews ourselves. But any competent user can help. Reviewing has two components. First, does the patch fix the problem? Testing this requires a Github account and a local clone and ability to build a test binary. See devguide.python.org for more. Second, does the patch meet our standards of code quality. Solaris-specific patches likely change the C part of cpython, so C competence and understanding of PEP 7 is needed here. """ Of the 114 issues mentioned above, not all are Solaris specific, and at least one says not a problem on Solaris. Any at least some are obsolete. Anyway, 67 are marked as having a patch. Of those, about 25 have a github PR, the others should have an uploaded .patch or .diff that could be made into a PR. Slicing another way, 20 and 15 are marked as type 'behavior' and 'compile error'. A few errors are possible. Another 11 have no type selected. So I guess there might be say 30 bug issues that affect Solaris and that have a patch to review and improve. So one or more of your people could select issues of interest and help get a patch to where they think is it ready to merge. Then ask if some coredev that can do so will do a final review and possible merge. In message msg380008 yesterday, Victor listed steps toward a buildbot, starting with a census of failing tests to discover the current state of CPython 3.10 on Solaris. -- Terry Jan Reedy ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/IMNECV4FMXN274QZHLESIDF5URG7ZVAB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636
Hello, On Sat, 31 Oct 2020 18:01:16 +1000 Nick Coghlan wrote: > On Sat, 31 Oct 2020 at 17:49, Paul Sokolovsky > wrote: > > Alternatively, can add "inline guard sigils" for the most common > > guard conditions, which would be equality, period: > > > > match r: > > case (src, None): > > case (==c, msg): > > case (==s, msg): > > `==` interacts poorly with other aspects of the pattern syntax (most > notably keywords in class patterns), but PEP 642's identity constraint > expressions would allow: Nice to hear that there're (high-hierarchy) people who want to do 2nd round on intent-explicitizing sigils, thanks. I blame it on my Saturday's hazy mind, but also for the interest of other readers, could you elaborate on "`==` interacts poorly with other aspects of the pattern syntax"? To elaborate from my side, I'm talking about introducing unary prefix operator "==". That should be pretty disambiguated on its own. (Of course, such an operator would be allowed only in the expression syntax for the "case" clause. The biggest risk my hazy mind sees is founding fathers' fixation with doing any grammar disambiguation on the level of the (flat) grammar, which leads to atrocious grammars. Whereas a practical person would just go for a level-above check). > > match r: > case (src, ?is None): > ... # Any src, msg is None > case (?is c, msg): > ... # Any msg, src is c > case (?is s, msg): > ... # Any msg, src is s So, you're talking about introducing unary prefix operators "?" and "?is". And while "?is" is clear (though not pretty), I don't see how the syntax of "?" translates to the meaning you assign to it, "lookup value of". (Btw, just thought, unary "+" could be used for "value lookup" syntax. At least, with such usage, it will be "mathematically understandable". Actually, would anything need to be done at all to use unary "+" in this meaning? Hmm, someone must have thought about that before...) > Cheers, > Nick. -- Best regards, Paul mailto:[email protected] ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/QPYBAPOSMXK7XDETO5XB5GNFITI6JPTN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636
On Sat, 31 Oct 2020 at 18:20, Paul Sokolovsky wrote:
> I blame it on my Saturday's hazy mind, but also for the interest of
> other readers, could you elaborate on "`==` interacts poorly with other
> aspects of the pattern syntax"?
Class patterns use the keyword argument syntax to match by attribute name:
case SomeClass(some_attr=):
...
Mapping patterns use dictionary syntax:
case {some.key:}:
...
If match patterns can start with "=", those both end up looking pretty
weird (and that's assuming the tokenizer can even figure out what is
happening in the first place - if you omitted the spaces before "=="
in the examples below, it would come up with "== =" for "===" and ":=
=" for ":=="):
case SomeClass(some_attr= ==EXPR):
...
case {some.key: ==EXPR}:
...
Most ad hoc examples seem to lean towards sequence matching (I guess
because they're shorter), but it's accounting for the readability of
class and mapping patterns that places the most constraints on the
choice of prefix sigil.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/UY4MXJG46RFBBM25XOLHOVOZQ7HJKZI3/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
Hi Nick and Everyone, We had actually considered a similar idea (i.e. load sigils) during the design phase of pattern matching. In the interest of having a rule that is as simple as possible, we had proposed to use a leading dot as a universal marker. Tin's example would thus have been written as:: match r: case (src, None): ... case (.c, msg): ... case (.s, msg): ... However, this idea was met with some resistance. After briefly looking at various alternatives again, we eventually decided to defer this discussion entirely, allowing for the community to perhaps gain some experience with the basic pattern matching infrastructure and have a more in-depth discussion later on. Paul also wrote [1]: Nice to hear that there're (high-hierarchy) people who want to do 2nd round on intent-explicitizing sigils, thanks. While we from the PEP-622/634/635/636 team are quite adamant that stores should *not* be marked, having a second round of discussion about load sigils is quite exactly what we aimed for! However, we should consider this to be a discussion about an *extension* of the existing PEPs (634-636), rather than about modifying them: *The introduction of a load sigil (be it the dot or a question mark or anything else) can actually be discussed quite independently of the rest of pattern matching.* You might have noticed that the original PEP 622 contained a lot more than the current PEPs 634-636. This is intentional: with the current pattern matching PEPs, we boiled down the entire concept to the basic infrastructure that we need in order to get it going; a basic "starter kit" if you will. There are a lot of ideas around for extending this basic pattern matching and make it much more powerful and versatile, including load sigils as proposed by PEP 642. But let us perhaps just start with pattern matching---hopefully in 3.10 :)---and then gradually build on that. Otherwise, I am afraid we will just keep running in circles and never get it to lift off. Cheers, Tobias [[1]] https://mail.python.org/archives/list/[email protected]/message/QPYBAPOSMXK7XDETO5XB5GNFITI6JPTN/ ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/FYGTU4RMZGKTQUANDSTRUSTC5GTZ5WUY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
Thank you for the well-written PEP, although I don't agree with it. My
response below is quite long. Here is my opinionated TL;DR:
(1) Just get over the use of `_` for the wildcard pattern.
another identifier. Now that the parser will support soft keywords, we
should expect more cases that something that is an identifier is one
context will be a keyword in another.
(2) The most common uses of patterns should not require sigils.
(3) None is special, and we should insist on `is` comparisons by
default. True and False are a little more problematic.
(4) Using sigils to over-ride the default is okay. That includes turning
what would otherwise be a capture pattern into a comparison.
Details below.
On Sat, Oct 31, 2020 at 05:16:59PM +1000, Nick Coghlan wrote:
> The rendered version of the PEP can be found here:
> https://www.python.org/dev/peps/pep-0642/
Quoting from the PEP:
"Wildcard patterns change their syntactic marker from _ to ?"
Yuck. Sorry, I find `?` in that role very aesthetically and
visually unappealing :-(
I really don't get why so many people are hung up over this minuscule
issue of giving `_` special meaning inside match statements. IMO,
consistency with other languages' pattern matching is more useful than
the ability to capture using `_` as a variable name.
Now that the PEG parser makes it easy to have soft keywords, there will
probably be more cases in the future where something that is
syntactically an identifier is a regular name in one context and special
syntax in another. This has happened before (e.g. "as") and it will
happen again.
We have a very strong convention that `_` is used as a write-only "don't
care" variable. (The two exceptions are the magic underscore in the
REPL, and `_()` in i18n.) In idiomatic Python code, if we bind a value
to `_` and then use it later, we are Doing It Wrong.
Is there such a shortage of local variable names that the inability to
misuse `_` is a problem in practice? Just use another identifier.
But if we really *must* break that convention and bind to `_`, we can
still do it inside a match statement:
case a:
_ = a
print(_)
The fact that you have to use a temporary variable to break the rules
is, in my opinion, a good thing -- it reminds you that what you are
doing is weird.
Quoting code from the PEP:
```
# Literal patterns
match number:
case ?0:
print("Nothing")
case ?1:
print("Just one")
```
I think this is an example of what Larry Wall talked about when he
discussed the mistakes of Perl's original regex syntax:
"Poor Huffman coding"
https://www.perl.com/pub/2002/06/04/apo5.html/
Wall regrets that many common patterns are longer and harder to write
than rarer patterns.
Why do we need a `?` sigil to match a literal? `case 1` cannot possibly
be interpreted as a capture pattern. It would be wrong to compare it
with `is`. What else could it mean other than equality comparison? The
question mark is pure noise.
So here's a counter suggestion:
(1) Literals still match by equality, because that is what want 99% of
the time. No sigil required.
You mention this in the "Rejected ideas" section, but I reject your
rejection :-)
The PEP rejects this because:
"they have the same syntax sensitivity problem as value patterns do,
where attempting to move the literal pattern out to a local variable for
naming clarity would turn the value checking literal pattern into a name
binding capture pattern"
but that's based on a really simple-minded refactoring. Sure, the naive
user who knows little about pattern matching might try to refactor like
this:
# Before.
match record:
case (42, x): ...
# After.
ANSWER_TO_LIFE = 42
match record:
# It's a Trap!
case (ANSWER_TO_LIFE, x): ...
and I am sympathetic to your desire to avoid that.
But this is the sort of error that:
- only applies in a comparatively unusual circumstances
(naively refactoring a literal in a case statement);
- is easily avoided by automated refactoring tools;
- linters will warn about (assignment to a CONSTANT);
- is easily spotted if you have unit tests;
- is obvious to those with more experience in pattern matching.
So I don't see this is as a large problem. I expect few people will
be bitten by this more than once, if that. I think that your
preventative solution, forcing all literal patterns to require a
sigil, is worse than the problem it is solving.
Bottom line: let's not hamstring pattern matching with poor Hoffman
coding right from day one.
(2) While literals usually compare by equality, the exception is three
special keywords, and one symbol, that compare by identity:
case None | True | False | ... :
# Compares by identity.
I can't think of any other literal where identity tests would be useful
and guaranteed by the language (no relying on implementation-specific
details, such as small int caching or string interning)
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat, Oct 31, 2020 at 10:22:09PM +1100, Steven D'Aprano wrote: > (1) Just get over the use of `_` for the wildcard pattern. > another identifier. Now that the parser will support soft keywords, we > should expect more cases that something that is an identifier is one > context will be a keyword in another. Oops, I lost a word. That should say "use another identifier". All other typos and misspellings are intentional :-) -- Steve ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/BLKDXXQPAID2MY75YOIHZ63MTTQ3SG23/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat, 31 Oct 2020 at 11:25, Steven D'Aprano wrote: > > Thank you for the well-written PEP, although I don't agree with it. My > response below is quite long. Here is my opinionated TL;DR: For what it's worth, I find your rebuttal of PEP 642 convincing, and in line with my thoughts on the matter. -1 from me on PEP 642. Paul ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/BMD63XIUQGFPPFCCIE3WN35JU4CEJL5O/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
Hello, On Sat, 31 Oct 2020 12:16:09 + Paul Moore wrote: > On Sat, 31 Oct 2020 at 11:25, Steven D'Aprano > wrote: > > > > Thank you for the well-written PEP, although I don't agree with it. > > My response below is quite long. Here is my opinionated TL;DR: > > For what it's worth, I find your rebuttal of PEP 642 convincing, and > in line with my thoughts on the matter. > > -1 from me on PEP 642. Given that this was a direct reply to Steven's mail, and he explicitly said: > (4) Using sigils to over-ride the default is okay. That includes > turning what would otherwise be a capture pattern into a comparison. And that's also the stated goal of PEP 642, quoting: > This PEP takes the view that not requiring a marker prefix on value > lookups in match patterns results in a cure that is worse than the > disease: Python's first ever syntax-sensitive value lookup where you > can't transparently replace an attribute lookup with a local variable > lookup So, both PEP 642 and Steven agree that the problem exists, and explicit marker is a suitable means to address it. Then, deriving "rebuttal" and "-1" to PEP 642 from Steven's mail sounds a bit confusing. -- Best regards, Paul mailto:[email protected] ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/4GXMELBO6HCXCYWKPQAXOSSC6ZD7Y2ZV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636
Hello,
On Sat, 31 Oct 2020 18:53:57 +1000
Nick Coghlan wrote:
> On Sat, 31 Oct 2020 at 18:20, Paul Sokolovsky
> wrote:
> > I blame it on my Saturday's hazy mind, but also for the interest of
> > other readers, could you elaborate on "`==` interacts poorly with
> > other aspects of the pattern syntax"?
>
[]
> case SomeClass(some_attr=):
[]
>
> If match patterns can start with "=", those both end up looking pretty
> weird (and that's assuming the tokenizer can even figure out what is
> happening in the first place - if you omitted the spaces before "=="
> in the examples below, it would come up with "== =" for "===" and ":=
> =" for ":=="):
Of course, there would need to be a space between "=" and "==", so
those could be recognized as exactly those tokens, and not as another
token, "===".
If you hint that it would be the first case of that in Python, then ...
yeah, seems to be so. But then, pattern matching is the first case when
a seemingly r-value looking expression, like 'Cls(a, b)' can actually
be l-value (i.e. bind/assign to variables). I'm afraid, we won't go
thru this journey without expanding perceptual horizons...
>
> case SomeClass(some_attr= ==EXPR):
I agree this looks ugly.
> ...
> case {some.key: ==EXPR}:
IMHO, this looks less unpretty than above, almost Ok.
> Most ad hoc examples seem to lean towards sequence matching (I guess
> because they're shorter), but it's accounting for the readability of
> class and mapping patterns that places the most constraints on the
> choice of prefix sigil.
Right, because many people are familiar with pattern matching from
languages where it's applied to algebraic data types, a clean,
simple (bordering on primitive) concept. Whereas problem of Python is
that it tries to "embrace unembraceable". And as it tries to do so,
complex things will necessarily look cumbersome, and would require
special treatment.
We already use "def foo(kw1: ann = 1, k2: ann2 = 2)" syntax, and so can
use:
> case SomeClass(some_attr = ==EXPR):
Which no longer looks as ugly as before (just unpretty).
But anyway, the talk is about using an explicit sigil, which one is
hopefully still being considered. So, "==" as a sigil has issues, but
then "?" too. There was a (seemingly) novel idea of using unary plus,
"+" as a sigil, with a clear meaning which can be derived from its
usual Python semantics: "get value of an expression (variable as a
particular case)". What about it?
"By abuse of notation" it could be used as a wildcard match too ;-).
(But as-value sigil and wildcard syntax are 2 orthogonal questions,
while PEP 642 goes a long way to conflate them, and presents that
conflation as a selling point.)
> Cheers,
> Nick.
--
Best regards,
Paul mailto:[email protected]
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/IUFWUVFM4JTYZC2W5UQ6C5HZNI3QLGWY/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat., 31 Oct. 2020, 9:29 pm Steven D'Aprano, wrote:
>
>
>
> (3) Overriding the default comparison with an explicit sigil is
> allowed:
>
>
> case ==True:
> print("True, or 1, or 1.0, or 1+0j, etc")
>
> case ==None:
> print("None, or something weird that equals None")
>
> case is 1943.63:
> print("if you see this, the interpreter is caching floats")
>
Where is this override allowed? It isn't covered under the syntax for value
patterns or literal patterns:
* https://www.python.org/dev/peps/pep-0634/#value-patterns
* https://www.python.org/dev/peps/pep-0634/#literal-patterns
and there aren't any other pattern types that make comparisons.
It also isn't in the draft reference implementation.
If PEP 634 allowed the exact comparison operator to be specified for
patterns (with at least "==" and "is" allowed), and patterns with such
explicit operators allowed arbitrary primary expressions as PEP 642
proposes, that would indeed address the bulk of my concerns:
* literal patterns would be an unambiguous shorthand for a comparison
pattern (always equality - see discussion below)
* attribute patterns would be an unambiguous shorthand for a comparison
pattern (always equality)
* the implementation would have no need to reinvent a subset of expression
compilation specifically for literal and attribute patterns, it could just
use the parser to control the conversion of the restricted syntactic
shorthand to the more general comparison pattern at the AST level
* the deferred ideas in PEP 642 (negated comparisons, containment checks)
would all be just as applicable as deferred ideas for an updated PEP 634
that included comparison patterns (with the question mark free spellings
"!=", "is not", "in" and "not in")
(To a first approximation, the code needed to implement this feature for
PEP 634 is the code I already wrote to implement "?" and "?is" for PEP 642,
and the code deletion notes in my branch would also generally apply)
>
> I don't think that there will be any ambiguity between the unary "=="
> pattern modifier and the real `==` operator. But if I am wrong, then we
> can change the spelling:
>
>
> case ?None:
> print("None, or something weird that equals None")
>
> case ?is 1943.63:
> print("if you see this, the interpreter is caching floats")
>
>
> (I don't love the question mark here, but I don't hate it either.)
>
> The important thing here is that the cases with no sigil are the common
> operations; the sigil is only needed for the uncommon case.
>
The tokeniser does struggle with "==" appearing after "=" or ":" in class
patterns and mapping patterns, so you have to make sure to help it out with
whitespace or parentheses.
That's why I didn't use it for PEP 642, but the whitespace sensitivity
would be more tolerable if the explicit symbol was left out most of the
time.
>
> (4) Patterns which could conceivably be interpreted as assignment
> targets default to capture patterns, because that's what is normally
> wanted in pattern matching:
>
>
> case [1, spam, eggs]:
> # captures spam and eggs
>
>
> If you don't want to capture a named value, but just match on it,
> override it with an explicit `==` or `is`:
>
>
> case [1, ==spam, eggs]:
> # matches `spam` by equality, captures on eggs
>
As noted above, the current PEP 634 spec doesn't allow this, but if it did,
then I agree it would adress most of the concerns that prompted me to write
PEP 642.
If the 634 PEP authors are amenable, I'd be happy to prepare a PR against
the PEP that made this change so you could see what it would look like at
the grammar level.
>
> Quoting the PEP:
>
> "nobody litters their if-elif chains with x is True or x is False
> expressions, they write x and not x, both of which compare by value, not
> identity."
>
> That's incorrect. `if x` doesn't *compare* at all, not by value and not
> with equality, it duck-types truthiness:
>
Aye, I considered going back and rewording that part to be more technically
precise, but never actually did it (whether by type coercion or equality
comparison, the ultimate effect is being more permissive than the strict
identity check suggested for literal patterns).
```
> >>> class Demo:
> ... def __bool__(self):
> ... return True
> ... def __eq__(self, other):
> ... return False
> ...
> >>> x = Demo()
> >>> x == True
> False
> >>> if x: print("truthy")
> ...
> truthy
> ```
>
> There's a reasonable argument to make that (unless overridden by an
> explicit sigil) the `True` and `False` patterns should match by
> truthiness, not equality or identity, but I'm not going to make that
> argument.
>
While I'd consider duck typing True & False less objectionable than
comparing them by identity (as it would follow PEP 8), it wouldn't fix the
key problem with special casing literals in the compiler: you lose that
special casing if the literal value is replaced by a symbolic reference to
t
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat, Oct 31, 2020 at 6:30 PM Nick Coghlan wrote:
> On Sat., 31 Oct. 2020, 9:29 pm Steven D'Aprano,
> wrote:
>
>>
>>
>>
>> (3) Overriding the default comparison with an explicit sigil is
>> allowed:
>>
>>
>> case ==True:
>> print("True, or 1, or 1.0, or 1+0j, etc")
>>
>> case ==None:
>> print("None, or something weird that equals None")
>>
>> case is 1943.63:
>> print("if you see this, the interpreter is caching floats")
>>
>
> Where is this override allowed? [...]
>
You're quoting from Steven's counter-proposal, which he prefaced with:
> So here's a counter suggestion:
> If PEP 634 allowed the exact comparison operator to be specified for
> patterns (with at least "==" and "is" allowed), and patterns with such
> explicit operators allowed arbitrary primary expressions as PEP 642
> proposes, that would indeed address the bulk of my concerns:
>
> * literal patterns would be an unambiguous shorthand for a comparison
> pattern (always equality - see discussion below)
> * attribute patterns would be an unambiguous shorthand for a comparison
> pattern (always equality)
> * the implementation would have no need to reinvent a subset of expression
> compilation specifically for literal and attribute patterns, it could just
> use the parser to control the conversion of the restricted syntactic
> shorthand to the more general comparison pattern at the AST level
> * the deferred ideas in PEP 642 (negated comparisons, containment checks)
> would all be just as applicable as deferred ideas for an updated PEP 634
> that included comparison patterns (with the question mark free spellings
> "!=", "is not", "in" and "not in")
>
> (To a first approximation, the code needed to implement this feature for
> PEP 634 is the code I already wrote to implement "?" and "?is" for PEP 642,
> and the code deletion notes in my branch would also generally apply)
>
I think this over-stresses the notion that users might want to override the
comparison operator to be used. We only have two operators that make sense
in this context, 'is' and '==', and really, for almost everything you want
to do, '==' is the appropriate operator. (There is a small trickle of bugs
caused by people inappropriately using e.g. `if x is 1` instead of `if x ==
1`, suggesting that if anything, there is too much freedom here.) The big
exception is `None`, where you basically always want to use `is`, which is
what PEP 634 does.
In PEP 622, we didn't do this, and we felt uncomfortable about it, so we
changed it in PEP 634.
We also changed it for True and False, because we realized that since 1 ==
1.0 == True, people writing
```
case True:
```
would expect this to match only Booleans. The main use case here is
situations (like JSON) where Booleans are *not* to be considered equivalent
to 0 and 1, which using PEP 622 would have to be written as
```
case bool(True):
```
which is hard to discover and not that easy to grasp when reading either.
There's not really ever a reason to write
```
case ==True: # Using Steven's notation
```
since that's just an odd and misleading way to write
```
case 1:
```
I don't ever want to be having conversations about why "case True:" doesn't
> behave the same way as "case some.attr.referring.to.true:".
>
And you won't, because why would people define their own names for True and
False? For sure people will define constants with Boolean values (e.g.
`DEBUG = True`) but these aren't good candidates for use in patterns. I
could imagine seeing
```
match DEBUG_NETWORK, DEBUG_LOGIC:
case False, False: pass
case False, True: print("We're debugging logic only")
case True, False: print("Debugging network only")
case True, True: print("Debugging network and logging")
```
but I would be surprised by
```
match x:
case DEBUG: ...
```
just like I'd be surprised seeing
```
if x == DEBUG: ...
```
PS. Using `...` as a literal pattern is also Steven's invention, this isn't
in PEP 634. People would probably think it had some special meaning as a
pattern rather than understanding it was meant as the literal value
`Ellipsis`.
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat, Oct 31, 2020 at 9:37 PM Guido van Rossum wrote: > > I think this over-stresses the notion that users might want to override > the comparison operator to be used. We only have two operators that make > sense in this context, 'is' and '==', and really, for almost everything you > want to do, '==' is the appropriate operator. (There is a small trickle of > bugs caused by people inappropriately using e.g. `if x is 1` instead of `if > x == 1`, suggesting that if anything, there is too much freedom here.) The > big exception is `None`, where you basically always want to use `is`, which > is what PEP 634 does. > FWIW, there's an additional exception: sentinel = object() if var is sentinel: I use this idiom from time to time - instead of None. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/VVOITSG54OTVBDTZGGRW7NTTAAG4EIQU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching
On Sat, Oct 31, 2020 at 21:48 Dan Stromberg wrote: > > On Sat, Oct 31, 2020 at 9:37 PM Guido van Rossum wrote: > >> >> I think this over-stresses the notion that users might want to override >> the comparison operator to be used. We only have two operators that make >> sense in this context, 'is' and '==', and really, for almost everything you >> want to do, '==' is the appropriate operator. (There is a small trickle of >> bugs caused by people inappropriately using e.g. `if x is 1` instead of `if >> x == 1`, suggesting that if anything, there is too much freedom here.) The >> big exception is `None`, where you basically always want to use `is`, which >> is what PEP 634 does. >> > FWIW, there's an additional exception: > sentinel = object() > > if var is sentinel: > > I use this idiom from time to time - instead of None. > You can just write ‘case sentinel’, since object’s == operator uses identity anyway. > -- --Guido (mobile) ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/OTVDPG34HRQM2NNWUUUB5KRZQR7O5ORE/ Code of Conduct: http://python.org/psf/codeofconduct/
