[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching

2021-02-20 Thread Tobias Kohn

 Hi Oscar

Quoting Oscar Benjamin :


On Fri, 19 Feb 2021 at 15:41, Tobias Kohn  wrote: [...]

It's not easy now to look back over the history of all of this. My
recollection of the original version of PEP 622 (if that's the right
one...) was that it had an overcomplicated protocol for __match__. It
needed to be simplified but in the end it was dropped.



Without further context, I would agree with you: it is difficult to  
look back over the entire history of the pattern matching PEPs.   
However, on the one hand PEP 622 is directly linked in the current PEP  
634.  On the other hand, for those who actively particiated in the  
discussion of both PEP 622 as well as the DLS paper, I find it a bit  
too easy and 'convenient' to call those resources now 'hard to find'.


That being said, I think it is part of your homework to first research  
about the history and what others have done before proposing your  
innovation.  Granted, this is hard and very laborious work that often  
takes a long time and can be frustrating.  But if we want to make  
progress and move forward, we have to stop running in circles.—To be  
absolutely clear here: I do not think that Mark's proposal is running  
in circles and I think it is fair enough to bring up these ideas.  But  
I equally think it is well possible to acknowledge that one part of it  
is a discussion of existing ideas, and to have a look at why these  
ideas have not made it into PEP 634.

 


The question now is if it will be straight-forward to retrofit a
protocol to PEP 634 after it is released and when backward
compatibility constraints kick in. PEP 653 (as discussed here) is
precisely an attempt to retrofit a protocol to PEP 634. I think the
difficulties involved in achieving that will become harder rather than
easier in future.

--
Oscar


There are actually two protocols in PEP 653.  One is about changing  
the fundamental design of pattern matching as outlined in PEP 634.   
This is a part that I reject for reasons presented in one of my last  
posts.  The second one is the customisation part using  
`__deconstruct__` or `__match__`.  This is something that I like a lot  
and would certainly like to see in pattern matching—alas, just being  
in favour of something is not a strong enough argument for it.


Similar to my votum above: if we are going to discuss this, I think we  
need new arguments or evidence, something to show why the previous  
decision to drop it (for now) was wrong.  For that it might be worth  
reading the respective section in deferred ideas of PEP 622.  Some  
part of our decision was based on the notion that adding such a custom  
protocol later one will be relatively painless, but if we were wrong,  
please speak up and show us what we have overlooked.  If you can  
present, for instance, some real-world use cases where this feature  
would enormously benefit sympy (or any other popular package), that  
would be helpful and a good starting point.


 Kind regards,
Tobias
 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7EK4K36F6NKD67VTBS2XWRXAN4E737AD/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching

2021-02-19 Thread Tobias Kohn

 Hi Mark,

Quoting Mark Shannon :


Hi Tobias,

[...]

But they are not referenced in PEP 634. I shouldn't have to trawl  
the internet to find the rejected ideas section.



https://dl.acm.org/doi/10.1145/3426422.3426983


That paper describes a `__match__` method, which is absent from PEP 634.
Why?

Cheers,
Mark.



You are right: you should not have to trawl the internet to get these  
information.  However, given the history of the Pattern Matching PEPs  
and that PEP 622 is linked directly from PEP 634, I would think it is  
not that far a journey—even though there have been so many discussions  
that it has definitely become unwieldy to retain an overview...


Anyway, the answer to both your questions lies in that the Pattern  
Matching feature itself is rather complex and the PEPs ended up being  
huge and hard to read and understand as it is.  We therefore refrained  
from long lists of rejected ideas in PEP 634, since that has already  
been done in PEP 622.  Moreover, the `__match__` method and protocol  
were removed from the PEPs to focus on a core infrastructure and keep  
it as simple as possible.


I think such a customisable protocol will eventually add great value  
to pattern matching.  However, one of the main arguments was to wait  
until we have more experience with pattern matching in Python and can  
give specific use cases for this extended protocol, which allow us to  
guide the design of the protocol.


At the end of the day, I guess we quite agree that we would like to  
have `__match__`, `__deconstruct__` or whatever you want to name it.   
But the slight variations in design demonstrate that it might also be  
a good idea to carefully lay it out first, before adding it to the  
pattern matching engine.  Hence, separating this from the core  
implementation seems like a good idea that I still fully support  
(despite also looking forward to having it some day in the future ^_^).


Cheers,
Tobias
 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T6Y7DCKIXFYV3UQY7BHTARUBPUQ6BI2C/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching

2021-02-19 Thread Tobias Kohn

 Hi Mark,

Quoting Mark Shannon :


[...]

If you had published these "more complicated, powerful protocols",  
you might be able to claim that this is a "rehash".

But you didn't.
 


I would say that these ideas have been quite prominently published:
https://www.python.org/dev/peps/pep-0622/#custom-matching-protocol

https://dl.acm.org/doi/10.1145/3426422.3426983

Moreover, much of the discussion is publicly available, for instance here:
https://github.com/gvanrossum/patma/issues/8

Cheers,
Tobias
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WS2DFBMBRTF5YXZS6IG42QNJNDJHPBYN/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching

2021-02-19 Thread Tobias Kohn

 Hi Oscar,

Quoting Oscar Benjamin :

Yes, thanks Mark. I'm not sure I've fully understood the PEP yet but  
I can see some parts that I definitely like. [...]


As I have just noted in my response to Mark, the aspect with the  
"deconstructor" (or `__match__` protocol as we called it) is  
definitely something that I like, too.  Moreover, I think that  
packages like sympy might make a strong argument for it.


That being said, perhaps we would have to start thinking about  
concrete use cases first and then consider how to best provide for  
those with an extended match protocol.  Even though I still like the  
idea of a flexible match method, there might be aspects that I/we  
overlook without knowing exactly where we want to go with this.


I'm not entirely sure but I think that with PEP 653 you can  
implement this like:


   def __deconstruct__(obj):
       if obj.step != 1:
           return obj.start, obj.stop, obj.step
       elif obj.start != 0:
           return obj.start, obj.stop
       else:
           return obj.stop

I think that would then mean you could use a pattern range(10) to  
unambiguously match range(10) without matching range(10, 20) etc.


This is certainly more of an anti-pattern (no pun intended) than an  
argument for the match protocol.  Shuffling positional arguments  
around like this seems like a rather bad idea, particularly in the  
context of pattern matching.  I'd rather write `case range(_, stop,  
_)` or `case range(0, stop, 1)` and be explicit here.


Kind regards,
Tobias

 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PWJSQIQ7XHJRD7ZN7QWT6HF5MGMG47GN/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching

2021-02-19 Thread Tobias Kohn

 Hi Mark,

Thank you for your proposal to try and have more precise semantics for  
pattern matching.  Of course, the proposal primarily introduces a new  
and extended protocol for pattern matching, upon which the 'semantics'  
is then based.  I think it is important to recognise and discuss your  
work on this basis that it proposes a different protocol of how  
pattern matching works, before we consier whether your semantics  
really is 'more precise'.


As I understand it, your PEP proposes two major differences: control  
over the matching process moves from patterns to the objects and a  
custom protocol is introduced (via the `__deconstruct__` method).   
There is also the matter of the `__match_kind__` with the intention of  
improving performance, which I find of minor importance here.


If you go back and look at the history and development of PEP 634, you  
will find that we started off with a fully customisable method that we  
called `__match__` instead of `__deconstruct__`.  The idea of more  
flexible and customisable pattern matching is thus certainly in the  
spirit of our proposal as well.  However, we finally removed this part  
from our proposal due to strong concerns from the community that the  
match protocol is too complex, particularly for an initial version of  
pattern matching.  Should the need arise, it would still be possible  
to later add such a customisation protocol in an upcoming version of  
Python.


Given these concerns with respect to complexity and our concession to  
remove the `__match__` method, I am wondering: do you have strong  
arguments that would speak for inclusion of this feature in the first  
version of pattern matching after all?  I would love to have this  
feature and am truly interested in any arguments in its favour.


When pattern matching meets OOP, there is indeed some discussion that  
can be had on where the matching itself is to happen.  Simply put:  
should an object recognise that it is an instance of a given class, or  
should the class recognise that an object is an instance of it?


In the spirit of Python's `instancecheck` method, we opted for  
classes/patterns recognising whether an object matches, where the  
object itself is 'passive'.  In other words, `case C()` would match  
any object that is an instance of a subclass of `C`, in line with  
usual OOP principles (where any instance of a subclass of `C` is still  
a valid instance of `C` itself).


Your design turns this around and has the object take on the  
responsibility of checking whether it wants to match a specific  
pattern.  This has the advantage that the object itself can morph into  
anything it wants to—as long as it is aware of the pattern.  And  
therein lies the problem that IMHO fully negates one of the major  
advantages of pattern matching: that the patterns and the objects that  
are to be matched are de-coupled and fully independent.  In other  
words: in your design it is not she who writes the patterns that  
controls what objects match, but he who designs the class hierarchy of  
the objects used.


One of the great traits of pattern matching is that it works  
out-of-the-box even with objects from ancient or alien code bases.   
Pattern matching according to PEP 634 is isolated, it happens only  
where pattern matching is explicitly used.  When writing classes, you  
do not have to worry about pattern matching at all, unless you want  
your classes to be used as patterns.  And should we decide some time  
in the future that we want to introduce new kinds of patterns, we  
still do not have to change a single object because the matching  
happens in the patterns and not in the objects, anyway.  That his  
would not be the case with your design is already obvious with the  
list of `__match_kind__` for common builtin classes, for instance.


Although there are some other programming languages that chose to  
follow this route, I think it goes against the spirit of what is  
already there in Python, it violates the principle of separated  
concerns and is thus a rather ill-advised modification of the original  
protocol.


Kind regards,
Tobias
 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7UIKPFTOU6APJGHW35A3P4JV4N6PL7K7/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Questions about about the DLS 2020

2020-11-26 Thread Tobias Kohn

 Hi Koos,

Yes, our three PEPs (634/635/636) are up to date.  PEP 634 is the  
technical specification, so you probably want to start with the  
tutorial (PEP 636) or the rationale (PEP 635).


https://www.python.org/dev/peps/pep-0636/
https://www.python.org/dev/peps/pep-0635/

Perhaps the following link to a discussion and vote might be a good  
starting point to get an overview of the other proposals that are  
around:


https://discuss.python.org/t/gauging-sentiment-on-pattern-matching/5770

Kind regards,
Tobias

Quoting Koos Zevenhoven :

I've had some things going on, and I'm still trying to catch up with  
the discussions here. Can someone tell me what would be the best  
place to look at the most recent proposal? Is one of the PEPs up to  
date?


   On Mon, Nov 16, 2020 at 7:02 PM Tobias Kohn  
 wrote:



_Hi Mark,

Thank you for your interest and the questions.

1.  This really comes down to how you look at it, or how you define  
pattern matching.  The issue here is that the concept of pattern  
matching has grown into a large and somewhat diverse flock of  
interpretations and implementations (as a side note: interestingly  
enough, some of the only universally agreed-upon standards are to  
use `_` as a wildcard and not to mark names that capture/bind  
values---which are quite exactly the points most fiercely debatted  
here).


Anyway, the paper presents the pattern matching structure we are  
proposing as one of three major variants of pattern matching:

(a)  Matching arguments to parameters in a function call,
(b)  Matching elements to elements in iterable unpacking,
(c)  Matching tree-like data to general patterns in a conditional  
pattern matching structure.


The last one is the subject of the PEP and the paper.  Nonetheless,  
in the first two cases (a) and (b), we find that indeed the  
computer will validate that the data matched the pattern and raise  
an exception if this fails.  This is where this way of looking at  
it comes from.


2.  Yes, that is indeed a deliberate simplification.  The idea is  
to abstract away from the details of how exactly Python implements  
abstract syntax trees (which I honestly believe are irrelevant for  
the sake of the entire narrative).  Moreover, using strings here  
allows us to exemplify the literal patterns, rather only showcasing  
only the constructor/class pattern.


Essentially, this is a question of making the most out of the  
little space available.


Since you have addressed this email to me directly, I would like to  
take this opportunity and briefly stress that this paper really  
grew out of a team effort.  While I might have been the one pushing  
for an academic publication, the DLS'20 paper represents the input  
and ideas of all the authors, as well as the long discussions we  
had.  Of course, I am happy to answer any questions about the  
paper, but it would be wrong to see me as the one person behind it.


Cheers,
Tobias

Quoting Mark Shannon :_


_Hi Tobias,

A couple of questions about the DLS 2020 paper.

1. Why do you use the term "validate" rather than "test" for the  
process of selecting a match?


It seems to me, that this is a test, not a validation, as no  
exception is raised if a case doesn't match.


2. Is the error in the ast matching example, an intentional  
"simplification" or just an oversight?


The example:

```
def simplify(node):
   match node:
       case BinOp(Num(left), '+', Num(right)):
           return Num(left + right)
       case BinOp(left, '+' | '-', Num(0)):
           return simplify(left)
       case UnaryOp('-', UnaryOp('-', item)):
           return simplify(item)
       case _:
           return node

```

is wrong.

The correct version is

```
def simplify(node):
   match node:
       case BinOp(Num(left), Add(), Num(right)):
           return Num(left + right)
       case BinOp(left, Add() | Sub(), Num(0)):
           return simplify(left)
       case UnaryOp(USub(), UnaryOp(USub(), item)):
           return simplify(item)
       case _:
           return node
```

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at  
https://mail.python.org/archives/list/python-dev@python.org/message/ETZGYRCF4DR6RJXTHGXIRZXINXJ76J2D/Code of Conduct:  
http://python.org/psf/codeofconduct/_



 


Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at  
https://mail.python.org/archives/list/python-dev@python.org/message/CUNO57W7KSIM2WRROC5R43ZT7HUQZCZ6/

Code of Conduct: http://python.org/psf/codeofconduct/_


    
___
Python-Dev mailing

[Python-Dev] Re: The semantics of pattern matching for Python

2020-11-21 Thread Tobias Kohn

 Hi David and Steve,

There is hardly anything that needs to be added to your comments, of  
course.  However, I consider these explicitly named attributes in the  
class pattern to be one of the most difficult aspects of our pattern  
matching proposal, which is something I just want to briefly  
acknowledge.  Although there are good reasons for using the syntax as  
proposed, understanding that ``Spam(a=x)`` assigns the attribute ``a``  
to the variable ``x`` is not quite as intuitive and straight forward.


Using the curly braces for that purpose might help in that we  
instantly think differently and more in line with dictionaries. This  
small clue could potentially have quite an impact on readability.   
This idea has thus a huge advantage over the square brackets not  
because of PEP 637, but because it might change the way we look at the  
code at hand.


The reason why I would not favour this specific syntax is because we  
expect that the predominant case would be to just write ``Spam(1, 2)``  
without the attribute names.  It is even possible to mix and match the  
two, which would then lead to something like ``Spam{1, b=2}``.  In  
that case, the advantage of the dictionary-like notation might just  
evaporate I assume, leaving behind a notation that turns out to be  
unusual and somewhat annoying in the overwhelming majority of cases.


There are a few other concepts and ideas behind it all, which I do not  
want to go into unless there is demand for it, but part of it is the  
idea that we want to be as much as possible in line with the syntax  
already used in iterable unpacking, say.


Kind regards,
Tobias

Quoting David Mertz :


On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano  wrote:


Clearly Spam(a=1, b=2) does not necessarily result in an instance with
attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
equivalent to (roughly):

    if (instance(obj, Spam)
        and getattr(obj, a) == 1
        and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to
the constructor `Spam(a=1, b=2)`, or even that this call would be
possible.


 
I think this explanation makes me not worry about the fact that  
`Spam(a=1, b=2)` in a pattern looks a lot like a constructor.  Like  
some other commenters, I was vaguely bothered that the identical  
spelling might have these different meanings in different contexts.   
But I think a match case just clearly enough IS a different context  
that using slightly different intuitions is no real conceptual  
stretch for remembering or teaching it.

 
As a strawman, we could use different syntax for "match the  
thing of class Spam that has attributes with these values:

 
match eggs:
    case Spam[a=1, b=2]: ...
 
Or:
 
match eggs:
    case Spam{a=1, b=2}: ...
 
Well, the square brackets COULD mean something different if PEP  
637 is adopted.  But even supposing the curly braces could be fit  
into the grammar.  Yes, it sort of suggests the connection between  
dictionaries and Spam.__dict__.  But it still reads as "this is  
something special that I have to think about a little differently."

 
Even where there are capture variables, I think I'd be  
completely comfortable thinking about the different context for:

 
match eggs:
    case Spam(a=x, b=2): ...
 

--
   The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J4GVSQN4UPOCN3EXZASQOIWN7MO723A4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: The semantics of pattern matching for Python

2020-11-21 Thread Tobias Kohn

 Hi Steve,

Thank you very much for your comments here.  This is certainly not the  
first time I feel that you not only have an excellent insight into a  
topic, but also manage to make your points very clearly and  
succinctly.  Your car example highlights the background of the  
proposed syntax very nicely, indeed, and the for-in example strikes me  
as quite astute.


Kind regards,
Tobias

Quoting Steven D'Aprano :


On Fri, Nov 20, 2020 at 02:23:45PM +, Mark Shannon wrote:


Why force pattern matching onto library code that was not designed for
pattern matching? It seems risky.


Can you give a concrete example of how this will "force" pattern
matching onto library code? I don't think that anyone has suggested that
we go around to third-party libraries and insert pattern matching in
them, so I'm having trouble understanding your fear here.


Fishing arbitrary attributes out of an object and assuming that the
values returned by attribute lookup are equivalent to the internal
structure breaks abstraction and data-hiding.


Again, can we have a concrete example of what you fear?

Python is not really big on data-hiding. It's pretty close to impossible
to hide data in anything written in pure Python.


An object's API may consist of methods only. Pulling arbitrary
attributes out of that object may have all sorts of unintended side-effects.


Well, sure, but merely calling print() on an object might have all sorts
of unintended side-effects. I think that almost the only operation
guaranteed to be provably side-effect free in Python is the `is`
operator. So I'm not sure what you fear here?

If I have a case like:

   match obj:
       case Spam(eggs=x):

I presume any sensible implementation is going to short-cut the
attempted pattern match for x if obj is *not* an instance of Spam. So
it's not going to be attempting to pull out arbitrary attributes of
arbitrary objects, but only specific attributes of Spam objects.

To the degree that your objection here has any validity at all, surely
it has been true since Python 1.5 or older that we can pull arbitrary
attributes out of unknown objects? That's what duck-typing does, whether
you guard it with a LBYL call to hasattr or an EAFP try...except block.

   if hasattr(obj, 'eggs'):
       result = obj.eggs + 1

Not only could obj.eggs have side-effects, but so could the call to
hasattr. Can you explain how pattern matching is worse than what we
already do?


PEP 634 and the DLS paper assert that deconstruction, by accessing
attributes of an object, is the opposite of construction.
This assertion seems false in OOP.


Okay. Does it matter?

Clearly Spam(a=1, b=2) does not necessarily result in an instance with
attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
equivalent to (roughly):

   if (instance(obj, Spam)
       and getattr(obj, a) == 1
       and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to
the constructor `Spam(a=1, b=2)`, or even that this call would be
possible.

I think that it will certainly be true that for many objects, there is a
very close (possibly even exact) correspondence between the constructor
parameters and the instance attributes, i.e. deconstruction via
attribute access is the opposite of construction.

But for the exceptions, why does it matter that they are exceptions?

Let me be concrete for the sake of those who may not be following these
abstract arguments. Suppose I have a class:

   class Car:
       def __init__(self, brand, model):
           self.brand = brand
           self.model = model

and an instance:

   obj = Car("Suzuki", "Swift")

For this class, deconstruction by attribute access is exactly the
opposite of construction, and I can match any Suzuki like this:

   match obj:
       case Car(brand="Suzuki", model)

which is roughly equivalent to:

   if isinstance(obj, Car) and getattr(obj, "brand") == "Suzuki":
       model = getattr(obj, "model")

It's not actually asserting that the instance *was* constructed with a
call to `Car(brand="Suzuki", model="Swift")`, only that for the purposes
of deconstruction it might as well have been.

If the constructor changes, leaving the internal structure the same:

   class Car:
       def __init__(self, manufacturer, variant):
           self.brand = manufacturer
           self.model = variant

the case statement need not change.

Remember that non-underscore attributes are public in Python, so a
change to the internal structure:

   class Car:
       def __init__(self, brand, model):
           self.brand_name = brand
           self.model_id = model

is already a breaking change, whether we have pattern matching or not.


When we added the "with" statement, there was no attempt to force
existing code to support it. We made the standard library support it,
and let the community add support as and when it suited them.

We should do the same with pattern matching.


That's a terrible analogy. Pattern matching is 

[Python-Dev] Re: Questions about about the DLS 2020

2020-11-20 Thread Tobias Kohn

 Hi Brett,

Thanks for your replies.

_> But you can write `123 .bit_length()`.  That's a parser limitation  
more than human understanding._


Touché.  I took this ambiguity of the dot so much for granted that I  
would not have thought of trying that.


_> Yep, but PEP 634 not only changes the convention, it special-cases  
"_" to be different in how it can be used anywhere else in Python.  
[...]_


I feel this is a bit of a misunderstanding, though: the underscore is  
really only special as a pattern.  Nothing in the PEP is changing  
anything outside of pattern matching.  I think it is quite important  
that such a complex feature is kind of separate from the rest of the  
language.


_> Welcome to worrying about every little detail in language design. _

Cheers ^_^.  To be fair, I learned a lot in this regard during the  
past months and I feel extremely lucky to have benefitted so much from  
working with these very smart and highly experienced team mates.  At  
the same token, we researched and approached the topic from so many  
different angles that I feel quite comfortable with the result and  
think this turned out to be a solid proposal.


_> [...]  Now obviously "practicality beats purity" as well, but the  
argument "other languages do it this way" doesn't hold well for a  
language that doesn't use curly braces for scoping delineation. _


I completely get that I fully agree with the intent of it.  We  
obviously have to concentrate much more on OCaml, Pascal, and Basic,  
which do not use curly braces for scoping delineation, either ;-)  
;-P.  Honestly, though, I guess there is a certain grey area here.  I  
am thinking, for instance, of things like a for-loop, which indicates  
iteration in about every language known(*), although there is research  
indicating that the keyword ``for`` itself is a rather bad choice.  At  
some point, it just becomes so much a standard that it transcendents  
into some kind of "meta-language".  I would argue that in this case,  
the burden of proof kind of reverses: you then need a very good  
argument why /_not_/ to stick to a universal standard.  And somehow,  
"Python only sticks superficially and by convention to the standard  
but we want to be able to break the rules even in this case whenever  
we fancy" strikes me as particularly convincing... :)


_> My question about adding pattern matching later was more to comment  
on the fact that the languages that use "_" for a wildcard pattern did  
it from early on, not later on; it had nothing to do with the proposal  
proposing pattern matching late in Python's history._


Hmm, yes I understand.  Still, it is probably hard to make a solid  
argument either way, because only few languages really added pattern  
matching later, with most of them being statically typed, functional,  
and so on.  So, there is not really much to go for I guess.  It is  
probably Python's burden now to embark on adventures to seek out new  
worlds, new possibilities and boldly go where no snake has gone  
before... :)


Kind regards,
Tobias

(*)  And yes, I am fully aware of assembly, purely functional  
languages, brainf*, etc.


Quoting Brett Cannon :


 

   On Tue, Nov 17, 2020 at 1:16 PM Tobias Kohn  
 wrote:



_Hi Brett,

Without having really looked at the history of all the languages we  
mention in the PEPs, I have a hunch that most of them had pattern  
matching from quite the beginning or an early stage on, indeed.   
That being said, I think the question itself does not really make  
much sense, though.  Programming languages are rarely if ever just  
invented out of the blue these days, but evolve from other  
languages that have gone before them.  So, sure C++ and Objective-C  
both had objects and classes from the very beginning on, but if we  
take into consideration that they strongly build on C, we could  
argue just as well that OOP was a later addition to C.  It really  
depends on your point of view.


But since we are talking about different languages here, there is  
one example I could bring up.  In C#, the underscore is a perfectly  
legal variable name.  And yet, it is also used as a 'discard' [1]  
in newer versions.  Moreover, F#, which certainly uses the  
underscore as a wildcard, runs on the same platform as C# and thus  
has to deal with this, too.  Somehow, Microsoft and its developers  
seem able to cope with it.


If I may widen the topic here a bit and seize the opportunity to go  
beyond just answering your email: I must admit that I find this  
entire discussion about the wildcard pattern rather mind-boggling.   
We seem all to accept the reality that ``match`` and ``case`` are  
context-sensitive keywords (i.e. they are both keywords and legal  
names)_


 
_Yes, because that's new syntax that's unambiguously new even if  
you have been programming in Python for decades._

_ _

_and that yo

[Python-Dev] Re: The semantics of pattern matching for Python

2020-11-20 Thread Tobias Kohn

 Hi Daniel and Mark,

Sorry for being slightly late to the party, but please let me add a  
few remarks of my own to the discussion here.


1. MUST HAVE PRECISELY DEFINED SEMANTICS

Yes, there are some aspects that we left open intentionally.  Most  
prominently the question of how often the pattern matching engine will  
check whether the subject is an instance of a particular class.  Take  
the following trivial example::


  match some_data:
  case Pair(12, 34):
  ...
  case Triple(12, 34, z):
  ...
  case Pair(12, y):
  ...
  case Pair(x, y):
  ...

In a perfect world, the compiler discovers that it must check whether  
``some_data`` is an instance of ``Pair`` exactly once and not three  
times.  This, of course, plays right into Mark's second point on  
efficiency and seems obvious enough.  Yet, as soon as we are  
considering nested patterns, it turns much less obvious whether the  
compiler is supposed to cache repeated isinstance-checks.  Can we  
really expect that the compiler must discover that in both case  
clauses the first element is checked against the same class?  Or would  
it make more sense to simply expect the compiler to potentially  
perform this ``Num`` instance check twice::


  match some_data:
  case [ Num(), 12 ]:
  ...
  case [ Num(), y, *z ]:
  ...

It is easy to think of cases where we accidentally end up calling an  
isinstance check more than once because the compiler could not prove  
that they are equal.  Still, whenever possible we want to give the  
compiler the freedom to optimise the pattern matching statement by  
caching.


In a static language, all of this would not be an issue at all, of  
course.  In Python, however, we end up being caught between its  
dynamic features and the desire to make pattern matching reasonably  
efficient.  So, we ended up leaving the question open as how often the  
pattern matching engine is allowed or supposed to check instances.   
Naturally, if you go and write some isinstance-check on a class with  
side-effects, you can break it.


2. USERS SHOULD NOT HAVE TO PAY AN UNNECESSARY PERFORMANCE PENALTY TO  
USE PATTERN MATCHING


To quote Mark [1] here:

/> Users should not have to pay an unnecessary performance penalty to  
use pattern matching./


Alright, what does this even mean?  What is an unnecessary performance  
penalty?  How should that be measured or compared?


Pattern matching is not just fancy syntax for an if-elif-statement,  
but a new way of writing and expressing structure.  There is currently  
nothing in Python that fully compares to pattern matching (which is  
obviously why we propose to add in the first place).  So, do you want  
to compare a pattern matching structure to an if-elif-chain or rather  
an implementation using reflection and/or the visitor pattern?  When  
implementing pattern matching, would we be allowed to trade off a  
little speed handling the first pattern for moving faster to patterns  
further down?


Do our PEPs really read to you like we went out of our ways to make it  
slow or inefficient?  Sure, we said let's start with an implementation  
that is correct and worry about optimising it later.  But I thought  
this is 101 of software engineering, anyway, and am thus rather  
surprised to find this item on the list.


3. FAILED MATCHES SHOULD NOT POLLUTE THE ENCLOSING NAMESPACE

 This is a slightly wider issue that has obviously sparked an entire  
discussion on this mailing list on scopes.


If there is a good solution that only assigns variables once the  
entire pattern matched, I would be very happy with that.  However, I  
think that variables should be assigned in full before evaluating any  
guards---even at the risk of the guard failing and variables being  
assigned that are not used later on.  Anything else would obviously  
introduce a mini-scope and lead to shadowing, which hardly improves  
anything with respect to legibility.


4. OBJECTS SHOULD BE ABLE DETERMINE WHICH PATTERNS THEY MATCH

Short version: no!

Class patterns are an extension of instance checks.  Leaving out the  
meta-classes at this point, it is basically the class that is  
responsible for determining if an object is an instance of it.   
Pattern matching follows the same logic, whereas Mark suggests to put  
that upside-down.  Since you certainly do not want to define the  
machinery in each instance, you end up delegating the entire thing to  
the class, anyway.


I find this suggestion also somewhat strange in light of the history  
of our PEPs.  We started with a more complex protocol that would allow  
for customised patterns, which was then ditched because it was felt as  
being too complicated.  There is still a possibility to add it later  
on, of course.  But here we are with Mark proposing to introduce a  
complex protocol again.  It would obviously also mean that we could  
not rely as much on Python's existing 

[Python-Dev] Re: Questions about about the DLS 2020

2020-11-17 Thread Tobias Kohn

 Hi Brett,

Without having really looked at the history of all the languages we  
mention in the PEPs, I have a hunch that most of them had pattern  
matching from quite the beginning or an early stage on, indeed.  That  
being said, I think the question itself does not really make much  
sense, though.  Programming languages are rarely if ever just invented  
out of the blue these days, but evolve from other languages that have  
gone before them.  So, sure C++ and Objective-C both had objects and  
classes from the very beginning on, but if we take into consideration  
that they strongly build on C, we could argue just as well that OOP  
was a later addition to C.  It really depends on your point of view.


But since we are talking about different languages here, there is one  
example I could bring up.  In C#, the underscore is a perfectly legal  
variable name.  And yet, it is also used as a 'discard' [1] in newer  
versions.  Moreover, F#, which certainly uses the underscore as a  
wildcard, runs on the same platform as C# and thus has to deal with  
this, too.  Somehow, Microsoft and its developers seem able to cope  
with it.


If I may widen the topic here a bit and seize the opportunity to go  
beyond just answering your email: I must admit that I find this entire  
discussion about the wildcard pattern rather mind-boggling.  We seem  
all to accept the reality that ``match`` and ``case`` are  
context-sensitive keywords (i.e. they are both keywords and legal  
names) and that you cannot write ``123.bit_length()`` because the dot  
has several different meanings.  But when it comes to the underscore  
and the idea that *as a pattern* it might simply not store a value, it  
is considered an insurmountable obstacle to learning and understanding  
pattern matching?  Particularly because, after all, the underscore is  
in Python only a wildcard *by convention*?


I think pattern matching has a lot in common with 'recursion': if you  
see it for the first time, it can be very hard to wrap you head around  
it and understand what is going on or why anyone would want to do such  
a thing.  But once you understood it, it can be an extremely elegant  
solution to many problems.  So, I really can fully understand a lot of  
the reservations brought forward about this feature and various  
aspects of it.  And there were enough good and valid points brought up  
by the community, which we then integrated into and respected in our  
overhauled design---but that the wildcard pattern causes so much grief  
really is entirely beyond me...


So, yes, most languages had pattern matching quite from the outset on  
(or not at all).  But then, again, Python is about the only language I  
know to have introduced 'type' annotations so late in life rather than  
from its very beginning.  It came at the cost of using the colon for  
something different than compound statements or dictionaries.  If we  
consider how much annotations have been expanded in more recent  
versions of Python and how many tools make good use of it, I would  
call that a pretty successful move, though---even though all other  
languages might have had type annotations from their very birth.


Kind regards,
Tobias

P.S. Sorry, I guess this turned out to be not so much a reply to your  
comment alone, as much more a reply to many a message and discussion  
that has been posted here over time.


[1]  https://docs.microsoft.com/en-us/dotnet/csharp/discards

Quoting Brett Cannon :


 

   On Mon, Nov 16, 2020 at 9:03 AM Tobias Kohn  
 wrote:



_Hi Mark,

Thank you for your interest and the questions.

1.  This really comes down to how you look at it, or how you define  
pattern matching.  The issue here is that the concept of pattern  
matching has grown into a large and somewhat diverse flock of  
interpretations and implementations (as a side note: interestingly  
enough, some of the only universally agreed-upon standards are to  
use `_` as a wildcard and not to mark names that capture/bind  
values---which are quite exactly the points most fiercely debatted  
here)._


_How many of those languages added pattern matching /later/ and  
not at the earliest stages of the language (if not from the  
beginning)? And for those that added it later, how many of those  
didn't already have a convention surrounding "_"? My suspicion is  
"not many" and "not many". _

_ _
_-Brett_
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KP2TVEAPDT553VXL4QEUOTYBHHMDUXSK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Questions about about the DLS 2020

2020-11-16 Thread Tobias Kohn

 Hi Mark,

Thank you for your interest and the questions.

1.  This really comes down to how you look at it, or how you define  
pattern matching.  The issue here is that the concept of pattern  
matching has grown into a large and somewhat diverse flock of  
interpretations and implementations (as a side note: interestingly  
enough, some of the only universally agreed-upon standards are to use  
`_` as a wildcard and not to mark names that capture/bind  
values---which are quite exactly the points most fiercely debatted  
here).


Anyway, the paper presents the pattern matching structure we are  
proposing as one of three major variants of pattern matching:

(a)  Matching arguments to parameters in a function call,
(b)  Matching elements to elements in iterable unpacking,
(c)  Matching tree-like data to general patterns in a conditional  
pattern matching structure.


The last one is the subject of the PEP and the paper.  Nonetheless, in  
the first two cases (a) and (b), we find that indeed the computer will  
validate that the data matched the pattern and raise an exception if  
this fails.  This is where this way of looking at it comes from.


2.  Yes, that is indeed a deliberate simplification.  The idea is to  
abstract away from the details of how exactly Python implements  
abstract syntax trees (which I honestly believe are irrelevant for the  
sake of the entire narrative).  Moreover, using strings here allows us  
to exemplify the literal patterns, rather only showcasing only the  
constructor/class pattern.


Essentially, this is a question of making the most out of the little  
space available.


Since you have addressed this email to me directly, I would like to  
take this opportunity and briefly stress that this paper really grew  
out of a team effort.  While I might have been the one pushing for an  
academic publication, the DLS'20 paper represents the input and ideas  
of all the authors, as well as the long discussions we had.  Of  
course, I am happy to answer any questions about the paper, but it  
would be wrong to see me as the one person behind it.


Cheers,
Tobias

Quoting Mark Shannon :


Hi Tobias,

A couple of questions about the DLS 2020 paper.

1. Why do you use the term "validate" rather than "test" for the  
process of selecting a match?


It seems to me, that this is a test, not a validation, as no  
exception is raised if a case doesn't match.


2. Is the error in the ast matching example, an intentional  
"simplification" or just an oversight?


The example:

```
def simplify(node):
   match node:
       case BinOp(Num(left), '+', Num(right)):
           return Num(left + right)
       case BinOp(left, '+' | '-', Num(0)):
           return simplify(left)
       case UnaryOp('-', UnaryOp('-', item)):
           return simplify(item)
       case _:
           return node

```

is wrong.

The correct version is

```
def simplify(node):
   match node:
       case BinOp(Num(left), Add(), Num(right)):
           return Num(left + right)
       case BinOp(left, Add() | Sub(), Num(0)):
           return simplify(left)
       case UnaryOp(USub(), UnaryOp(USub(), item)):
           return simplify(item)
       case _:
           return node
```

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at  
https://mail.python.org/archives/list/python-dev@python.org/message/ETZGYRCF4DR6RJXTHGXIRZXINXJ76J2D/Code of Conduct:  
http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CUNO57W7KSIM2WRROC5R43ZT7HUQZCZ6/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Pattern Matching controversy: Don't read PEP 635, read DLS'20 paper instead

2020-11-15 Thread Tobias Kohn

 Hi Paul,

Thank you for your comments on the DLS'20 paper.  I am glad to hear  
that it helps paint a clear(er) picture of pattern matching in  
Python.  However, please let me set the record straight in a few  
regards.


First, there is no 'shoehorning' or 'deception' in our pattern  
matching PEPs.  This notion of our team standing against the Python  
community or trying to outsmart it is quite absurd and unnecessary,  
really.  At the end of the day, you might find that the PEPs are a  
boiled-down version of months of intense work and discussions (as  
Brandt has already pointed out elsewhere) and thus necessarily terse  
in some parts.  Pattern matching is a complex subject and the PEPs  
have been written with a sincere effort to convey the big picture and  
give a tour of this feature in an accessible manner.  Our entire  
discussion is all openly available to anyone interested.


Second, our work on pattern matching is based on a very simple  
premise/idea: *bring pattern matching to Python*.  In particular, it  
is *not* to 'bring Python to pattern matching'.  We want to introduce  
pattern matching as a new feature to Python as Python stands right  
now, without the need to first modify Python and make it more static,  
C-like, functional or whatever.  As I have mentioned before, pattern  
matching is an opt-in feature, something with absolutely no influence  
on the behaviour of any Python program you have ever written before.   
Not introducing constants or a completely new kind of scopes to Python  
first is certainly not a lack of academic rigour, oversight on our  
part because we are not smart enough, or trick to coerce people into  
accepting pattern matching.  It just means that we propose to  
introduce pattern matching to Python as it is right now.


Third, the academic paper has a different scope than the PEPs.  In  
case you missed it: after the first version of PEP 622, we listened to  
the reactions and suggestions from the community and went back to  
overhaul the entire design and build as much as possible on the raised  
concerns and cummulative experience of the community.  One major  
outcome of this was to reduce the overall proposal to the absolute  
core features of what we need for pattern matching (another was to put  
more effort into writing an accessible account of what we propose and  
why we think the various design features are a good choice, leading to  
three PEPs with different aims).  The academic paper outlines a much  
larger scope of possibilities than the PEPs, whereas the PEPs are more  
grounded in the pragmatic and taking one step at a time.  In short,  
the academic paper lays out an entire journey, a vision, whereas the  
PEPs propose a first step forward.


Finally, the reason why the academic paper has not been referred to  
before is simply that it will officially be published coming (!)  
week.  Since the process of peer review is now complete, we can  
already provide a preprint.  It was clear from the outset that the  
PEPs will contain a link to the paper as soon as it becomes publicly  
available.


Of course, we encourage anyone interested in it to read the academic  
paper.  Since its focus is somewhat complementary to the PEPs, it  
might highlight some ideas behind our design that might not be made  
clear enough in the PEPs.  But while doing so, please keep in mind  
that the paper is a vision of a bigger picture and as such distinct  
from the PEPs!


Kind regards,
Tobias

Quoting Paul Sokolovsky :


Hello,

As was mentioned many times on the list, PEP634-PEP636 are thoroughly
prepared and good materials, many thanks to their authors. PEP635
"Motivation and Rationale" (https://www.python.org/dev/peps/pep-0635/)
stands out among the 3 however: while reading it, chances that you'll
get a feeling of "residue", accumulating a section over section. By the
end of reading, you may get a well-formed feeling that you've read a
half-technical, half-marketing material, which is intended to "sell" a
particular idea among many other very viable ideas, by shoehorning some
concepts, downplaying other ideas, and at the same time, light-heartedly
presenting drawbacks of its subject one.

Just to give one example, literally at the very beginning, at the
"Pattern Matching and OO" section (3rd heading) it says:


Pattern matching is complimentary to the object-oriented paradigm.


It's not until the very end of document, in the "History and Context" it
tells the whole truth:


With its emphasis on abstraction and encapsulation, object-oriented
programming posed a serious challenge to pattern matching.


You may wonder how "complimentary" and "posed a serious challenge"
relate to each other. While they're definitely not contradictory,
starting the document with light-hearted "complimentary" can be seen as
trying to set the stage where readers don't pay enough attention to the
problem. And it kinda worked: only now [1] wider community discovers the

[Python-Dev] Re: My thoughts on Pattern Matching.

2020-11-09 Thread Tobias Kohn

Hi Thomas,

Thank you very much for your carefully worded and thoughtful email.  I  
feel, however, that many of your concerns are based on an idealised  
picture of a future Python language that will never actually  
materialise.


As I understand it your main point is that the concept of patterns  
might---or even should---be retro-fitted to general assignments.  Just  
as we have borrowed from and expanded on the idea of iterable  
unpacking in assignments, so should assignments then pick up the  
concepts introduced in pattern matching.  Indeed, assignments of the  
form ``Viking(spam, ham) = ...`` are not only appealing but can be  
found in various languages with pattern matching.  So, why would we  
not want to have consistent concepts and symmetric (orthogonal)  
structures across all of Python?


Unfortunately, such consistency or symmetry comes at a high cost---too  
high as far as I am concerned.


One of the simplest patterns is without doubt the literal pattern that  
essential only matches itself (e.g., ``case 123:`` or ``case  
'abc':``).  Any future unification of patterns and assignments would  
therefore necessarily allow for statement such as::


    1 = x
    
This is essentially equivalent to ``assert(x == 1)``.  Indeed,  
_Elixir_ [1] uses such syntax for exactly this semantics.  However,  
knowing how much novice programmers struggle with understanding that  
assignments are right-to-left (i.e. ``x = 1`` and not ``1 = x``),  
including such syntax would immediately raise the learning curve  
significantly.  In short, a very common students' mistake will take  
them to error messages and concepts they could not possibly understand  
without a basic comprehension of pattern matching.


And this is probably where it becomes most obvious how our views of  
pattern matching differ.  The pattern matching as we propose it in  
PEPs 634/635/636 is guarded by a keyword needed to activate these  
features.  Unless you start your statement with ``match my_data:``,  
you can easily pretend as if pattern matching did not exist: it will  
not, cannot affect your code.  This encapsulation is intentional and  
by design.


As far as I am aware, those languages that support syntax like  
``Viking(spam, ham) = ...`` only allow this in combination with  
variable _declaration_, i.e. you actually have to write ``let  
Viking(spam, ham) = `` or ``var Viking(spam, ham) = ...`` or similar.   
Without such a marker, this syntax quickly descends into unreadable  
gibberish.  As noted in the original section of rejected ideas in PEP  
622, we had originally considered adding 'one-off pattern matching':  
pattern matching with only a single case that must succeed, very much  
like normal assignments do.  But our approach was always guarded by a  
keyword, be that ``case`` or ``if``---in line with the ``var`` or  
``let`` found in other languages.  Even in that case, patterns would  
not leak into the language outside pattern matching.


Finally, there is already a necessary inconsistency between iterable  
unpacking and pattern matching.  By their very nature, patterns  
express a _possible_ structure, whereas iterable unpacking imposes a  
_necessary_ structure.  So, when dealing with iterators, it is safe to  
'unpack' the iterator in iterable unpacking.  If the expected and  
actual structures differ, it is an error, anyway.  In pattern  
matching, however, we have to be more conservative and careful,  
exploring options rather than certanties.  Hence, even if all other  
concerns were wiped away, the closest we could come to an entirely  
symmetric and consistent language is one with some subtle differences  
and thus prone for bugs and errors.


PEPs 634/635/636 are the result of a long and careful design process  
where we sought to appeal to the intuition of the Python programmer as  
much as possible, without betraying the differences and new concepts  
that pattern matching necessarily introduces.  Readability was always  
one of our main concerns and we believe that having a clear context  
where patterns occur greatly helps writing readable and consistent code.


So, long story short, I am afraid I would question the very premise  
upon which your concerns are founded: that it would ever be a good  
idea to expand patterns to assignments in general.  Although such a  
unification was in principle possible, it would rob Python of one of  
its greatest and strongest assets: its simplicity and learnability.


Kind regards,
Tobias

[1]  https://elixir-lang.org/getting-started/pattern-matching.html

 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KOYU2FAY3DZ4EHYWR3FXBQMTDP72EEQQ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 642: Constraint Pattern Syntax for Structural Pattern Matching

2020-10-31 Thread Tobias Kohn

Hi Nick and Everyone,

We had actually considered a similar idea (i.e. load sigils) during  
the design phase of pattern matching.  In the interest of having a  
rule that is as simple as possible, we had proposed to use a leading  
dot as a universal marker.  Tin's example would thus have been written  
as::


  match r:
  case (src, None): ...
  case (.c, msg): ...
  case (.s, msg): ...

However, this idea was met with some resistance.  After briefly  
looking at various alternatives again, we eventually decided to defer  
this discussion entirely, allowing for the community to perhaps gain  
some experience with the basic pattern matching infrastructure and  
have a more in-depth discussion later on.



Paul also wrote [1]:

Nice to hear that there're (high-hierarchy) people who want to do  
2nd round on intent-explicitizing sigils, thanks.



While we from the PEP-622/634/635/636 team are quite adamant that  
stores should *not* be marked, having a second round of discussion  
about load sigils is quite exactly what we aimed for!  However, we  
should consider this to be a discussion about an *extension* of the  
existing PEPs (634-636), rather than about modifying them:


*The introduction of a load sigil (be it the dot or a question mark or  
anything else) can actually be discussed quite independently of the  
rest of pattern matching.*



You might have noticed that the original PEP 622 contained a lot more  
than the current PEPs 634-636.  This is intentional: with the current  
pattern matching PEPs, we boiled down the entire concept to the basic  
infrastructure that we need in order to get it going; a basic "starter  
kit" if you will.  There are a lot of ideas around for extending this  
basic pattern matching and make it much more powerful and versatile,  
including load sigils as proposed by PEP 642.  But let us perhaps just  
start with pattern matching---hopefully in 3.10 :)---and then  
gradually build on that.  Otherwise, I am afraid we will just keep  
running in circles and never get it to lift off.


Cheers,
Tobias

 
[[1]]   
https://mail.python.org/archives/list/python-dev@python.org/message/QPYBAPOSMXK7XDETO5XB5GNFITI6JPTN/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FYGTU4RMZGKTQUANDSTRUSTC5GTZ5WUY/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

2020-08-04 Thread Tobias Kohn

 Hi Larry,

You are right that just dismissing intuition is wrong.  I should have  
been more careful with my wording or explain them better, and I would  
like to apologise if my response came across as too strong in this  
regard.


The actual problem that I see is that we have different  
cultures/intuitions fundamentally clashing here.  In particular, so  
many programmers welcome pattern matching as an "extended switch  
statement" and find it therefore strange that names are binding and  
not expressions for comparison.  Others argue that it is at odds with  
current assignment statements, say, and question why dotted names are  
_/not/_ binding.  What all groups seem to have in common, though, is  
that they refer to _/their/_ understanding and interpretation of the  
new match statement as 'consistent' or 'intuitive'---naturally  
pointing out where we as PEP authors went wrong with our design.


But here is the catch: at least in the Python world, pattern matching  
as proposed by this PEP is an unprecedented and new way of approaching  
a common problem.  It is not simply an extension of something already  
there.  Even worse: while designing the PEP we found that no matter  
from which angle you approach it, you will run into issues of seeming  
'inconsistencies' (which is to say that pattern matching cannot be  
reduced to a 'linear' extension of existing features in a meaningful  
way): there is always something that goes fundamentally beyond what is  
already there in Python.  That's why I argue that arguments based on  
what is 'intuitive' or 'consistent' just do not make sense _/in this  
case/_.  I think the discussion on this mailing list with the often  
contradictory views, proposals, and counter-proposals more than makes  
my point.


As for your argument that it looks like calling a function or creating  
an object: I tried to explain a little while ago that you'd be well  
advised to rather approach it as something similar to a function  
_/definition/_.  After all, the part after `def` in `def foo(a, b):`  
also looks like a function call!  But nobody seems to mind this  
similarity in syntax there!  And the target in `(a, b) = c` looks like  
a tuple constructor, although it actually is the exact opposite.


Finally, I completely agree that intuition is informed by experience  
and serving us very well.  The first part of this, however, is also to  
say that intuition is malleable thing!  And experience from other  
programming languages who took the leap to having pattern matching  
shows that it quickly becomes a quite intuitive and easy to use feature.


Cheers,
Tobias

P.S. Please excuse my late reply; I am currently on vacation.

Quoting Larry Hastings :


 
  On 7/31/20 12:36 AM, Tobias Kohn wrote:


And since pattern matching is really
a new feature to be introduced to Python, a feature that can
be seen in different lights, there is no 'Python-Programmer
intuition' that would apply in this case.


  It's not fair to say "intuition doesn't apply because it's new  
syntax".  There are plenty of examples of intuition serving a Python  
programmer well when encountering new syntax.  A Python programmer's  
intuition is informed by existing syntax and conventions in the  
language.  When they see a new construct, its similarity to existing  
constructs can make understanding the new syntax quite intuitive  
indeed.


  Take for example list comprehensions.  Python 1 programmers hadn't seen


a = [x for x in y]


  But they knew what square brackets meant in that context, it meant  
"creates a new list".  And they knew what "for x in y" meant, that  
meant iteration.  Understanding those separate two concepts, a  
Python 1 programmer would be well on their way to guessing what the  
new syntax meant--and they'd likely be right.  And once they  
understood list comprehensions, the first time they saw generator  
expressions and set and dict comprehensions they'd surely intuit  
what those did immediately.


  The non-intuitiveness of PEP 622, as I see it, is that it  
repurposes what looks like existing Python syntax but frequently has  
wholly different semantics.  For example, a "class pattern" looks  
like it's calling a function--perhaps instantiating an object?--but  
the actual semantics and behavior is very different.  Similarly, a  
"mapping pattern" looks like it's instantiating a dict, but it does  
something very different, and has unfamiliar and seemingly arbitrary  
rules about what is permitted, e.g. you can't use full expressions  
or undotted-identifiers when defining a key.  Add the "capture  
pattern" to both of these, and a Python programmer's intuition about  
what this syntax traditionally does will be of little help when  
encountering a PEP 622 match statement for the first time.


  Cheers,

   

  //arry/
___
Python-Dev mailing list 

[Python-Dev] Re: PEP 622 version 2 (Structural Pattern Matching)

2020-07-31 Thread Tobias Kohn

 Hi Caleb,

I will only answer to the second part, as the wildcard issue has
been brought up and discussed time and again, and the `np`
analogue is quite a stretch and far-fetched, really.

 One thing that stood out a bit to me as I feel to have seen it a
couple of times is the question of intuition, so I will add a few
more general thoughts to that...

 > [...] but it  seems quite unintuitive to me [...]


[...] don't necessarily make it intuitively clear [...]


  

 Intuition (or lack thereof) has already been brought forward
as an argument a couple of times.  I would just like to briefly
point out that there is no such thing as universal intuition in
the field of programming.  We all have different training,
skills, preferences and experiences, which make up what
we call 'intuition'.  But what is intuitive is usually something
completely different to C-programmer than to a Haskell- or
Lisp-Programmer, say.  And since pattern matching is really
a new feature to be introduced to Python, a feature that can
be seen in different lights, there is no 'Python-Programmer
intuition' that would apply in this case.

As for beginners, virtually every part of programming is
unintuitive at first.  Even something innocuous-looking like
assignment is often reason for confusion because `3 + 4 = x`
would probably be more 'intuitive'.  But there is good reason
with regards to the bigger picture to stick to `x = 3 + 4`.

A Python-programmer (at any level) not familiar with pattern
matching will most likely not understand all subtlety of the
syntax---but this is alos true of features like `async` or the
`/` in parameters, say.  I would argue, though, that the clear
choice of keywords allow anyone to quickly look pattern
matching up and get informed on what it does.  So, we do
not need to come with something that is entirely 'intuitive'
and 'self-evident'.  But by sticking to common convention
like `_` as wildcard, we can help quickly build the correct
intuition.

In your examples, for instance, it is perfectly obvious to me
that you cannot directly assign to attributes and it would in
fact look very weird to my eyes if you could.  Your use case
is quite similar to initialisers and you are arguing that you
would like being able to write:
```
CLASS Point:
    DEF __init__(self, self.x, self.y):
    PASS
```
rather than the more verbose:
```
CLASS Point:
    DEF __init__(self, x, y):
    self.x, self.y = x, y
```
I do not think that this would be a good idea for either
parameters or patterns.  After all, pattern matching is
*/not/* assignment, even though it is related to it, of
course.

Kind regards,
Tobias

Quoting Caleb Donovick :

Adding this feature would be a giant quality of life improvement for  
me and I really hope it succeeds.  So I have been trying to keep up  
on the debate in this and related thread.


While I do want this feature,  I agree with a lot of the issues  
people are raising.


First I agree that _ should not be the wildcard symbol.  Or rather  
the hobgoblins in my mind think that if _ is to be the wildcard  
symbol it would be more consistent with assignment if it was bound  
to the last value it was matched with (as should other repeated  
identifiers) e.g.,

   match pt:case (x, _, _):assert _ == pt[2]  
I understand the authors rationalize the decision based on  
conventions with the gettext module.  I find these arguments very  
unconvincing.  It's like saying the identifier np should be  
forbidden from appearing in cases because it's frequently used as  
the name of numpy.  If there is to be a wildcard symbol (that does  
not bind and is repeatable) it should not be a valid identifier. 


Second,  the distinction between a capture and constant value  
pattern should be more explicit.  I don't have any great insight  
into the color of the shed beyond the numerous suggestions already  
made (name=, ?name, etc...), but it  seems quite unintuitive to me  
that I can't capture into a namespace nor match a constant without a  
namespace.  It is also unclear to me why it would be so terrible to  
add a new token or abuse an existing one.


 > Honestly, it doesn't help the case for `?` that it's been  
proposed as a mark for both capture patterns and value patterns (by  
different people, obviously :-).


 I agree that most of the proposed sheds don't necessarily make  
it intuitively clear what is a capture variable vs what is a  
constant.  However they do give the programmer the ability to choose.


For example if I want to modify the motivating example from the PEP  
slightly to copy attributes from one point to another I can't  
express it concisely:
  def update_point_3d(p: Point3d, pt):match pt: case  
(x, y): p.x, p.y = x, y case Point2d(x, y):   
   p.x, p.y = x, y ... 



(Okay I could have just called the original make_point_3d and  
unpacked the results but it would require the creation of an  

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

2020-07-26 Thread Tobias Kohn

 Hi Rob,

 You are right: the grammar should probably read `suite` rather
than `block` (i.e. the `pass` is necessary).  Thanks for catching
this!

 As for the second question, I assume there might be a slight
oversight on your part.  The last line in the example replaces the
string `"_"` rather than the variable `_`.  The not-binding of `_`
thus has no influence on the last line.

 I think I will leave it for Mark himself to name the two bugs rather
than start a guessing game.  However, in an earlier version we had
left out the `if value` for the first case, accidentally translating
the `len(value) > 1` as a `len(value) >= 1` instead.

 Kind regards,
Tobias

Quoting Rob Cliffe via Python-Dev :


[...]

_First question_: Sometimes no action is needed after a case  
clause.  If the Django example had been written
if ( isinstance(value, (list, tuple)) and len(value) > 1 and  
isinstance(value[-1], (Promise, str)) ): *value, label =  
value else: label = key.replace('_', ' ').title()  the  
replacement code would/could be  match value: case [*value,  
label := (Promise() | str())] if value: pass case _:  
label = key.replace('_', ' ').title()  AFAICS the PEP does not  
*explicitly* state that the 'pass' line is necessary (is it?), i.e.  
that the block following `case` cannot (or can?) be empty. The term  
`block` is not defined in the PEP, or in  
https://docs.python.org/3/reference/grammar.html. But an empty block  
following a line ending in `:` would AFAIK be unprecedented in  
Python.  I think it is worth clarifiying this.  _Second question_:  
in the above example replacement, if `case _:` does not bind to `_`,  
does that mean that the following line will not work? Is this one of  
the "two bugs" that Mark Shannon alluded to?  (I have read every  
message in the threads and I don't remember them being spelt out.)  
And I'm curious what the other one is (is it binding to a variable  
`v`?).  Best wishes Rob Cliffe
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4MNGTBKIXNMMVAIFOLR2W62SLK637OY5/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Another take on PEP 622

2020-07-19 Thread Tobias Kohn

 Hi Terry,

 Thank you: I really like your wave/particules analogy.  I think that  
pattern matching is indeed uniting different concepts to build a  
stronger, more versatile structure.


 I also like your concept of a general "binding structure" with  
different forms, such as assignment, parameter passing, and (according  
to PEP 622) patterns.  I feel that this categorisation puts pattern  
matching quite in the correct place.  Of course, there is also the  
second aspect of "bind these variables---/if you can/" or "analyse and  
compare the structure", which is the other part of this particule/wave  
duality.


 Concerning the guards (optional conditions), I also think that you  
summarised very nearly.  I tend to think of the patterns a bit like  
the grammar of a programming language.  Something that is supposed to  
be static and declarative (as far as this is possible in Python).   
Yet, some constraints in programming languages cannot be expressed by  
a (context-free) grammar in a meaningful way.  For instance, the  
grammar itself does not keep you from having two parameters in a  
function sharing the same name.  This is a dynamic aspect, a  
relationship between two otherwise independent parts of the overall  
structure.  And that is best caught by the guards.  So, yes: as I see  
it, the guards really add something that goes beyond the declarative  
side of patterns.  Hence, I completely agree with your characterisation.


 Kind regards,
Tobias

Quoting Terry Reedy :


On 7/16/2020 9:51 PM, Tobias Kohn wrote:

Hi Everyone,

I feel there are still quite a few misconceptions around concerning  
PEP 622 and the new pattern matching feature it proposes.  Please  
allow me therefore to take another attempt at explaining the ideas  
behind PEP 622 with a different approach.  Bear in mind that I  
naturally cannot cover everything, though, and that some parts  
presented here are still slightly simplified.


Thank you Tobias.  I am a fan of looking at things from multiple  
viewpoint.  For 200 years, physicists argued about whether light is  
waves or particles, before discovering that 'neither' and 'both'  
were more correct.



1. Function Overloading
2. Visitor Pattern and Dispatch > 3. Shape and Structure


[snip, snip, snip]

In an assignment statement, the code to the left of '=' is a 'target  
list' of 'target's, with some idiosyncratic rules.  Even though it  
might, misleadingly, be called a 'target expression', it is not an  
expression to be evaluated.  Similarly, the code between parentheses  
in a def statement is a 'parameter list' of 'defparameter' and  
special symbols, with other idiosyncratic rules.  Both could be  
called 'binding lists' or more generally, 'binding structures'.


To me, the important point of your point is that 'case' is somewhat  
analogous to 'def', and that the stuff between 'case' and ':' is a  
binding structure.  We should call this structure a 'match  
structure'. It is misleading and confusing to call it a 'match  
expression'.  A match structure consists of a 'match list' of 'match  
items' or 'matcher's and an optional "'if' ".


Matchers have a 3rd, new, and larger set of idiosyncratic rules.   
The optional condition is an escape hatch for when expressing the  
intended match constraint and corresponding match set is difficult  
or worse using the match rules.


As with target and parameter items, untagged simple name matchers  
are (and I now see, should be) binding targets.  (The parameter list  
tags are ':' for types and '=' for default values.)  Unlike  
assignment targets, dotted names and subscriptings are not binding  
targets.  Like parameter lists, match lists include literals.  Match  
lists also include syntactic structure not seen in the other binding  
structures.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at  
https://mail.python.org/archives/list/python-dev@python.org/message/NNZ67OMAV2CDV7GSX64SOLUAERJSF5HP/Code of Conduct:  
http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6XZXGOHDBRN5ZXREZWB74DYHUGIEPLYI/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 622 aspects

2020-07-19 Thread Tobias Kohn

 Hi Koos,

 Let me try and address some of the concerns and questions you are  
rising.  I am replying here to two emails of yours so as to keep  
traffic down.


 Quoting Koos Zevenhoven :

 > (1) Class pattern that does isinstance and nothing else.


If I understand the proposed semantics correctly, `Class()` is  
equivalent to checking `isinstance(obj, Class)`, also when  
`__match_args__` is not present. However, if a future match protocol  
is allowed to override this behavior to mean something else, for  
example `Class() == obj`, then the plain isinstance checks won't  
work anymore! I do find `Class() == obj` to be a more intuitive and  
consistent meaning for `Class()` than plain `isinstance` is.


Instead, the plain isinstance check would seem to be well described  
by a pattern like `Class(...)`. This would allow isinstance checks  
for any class, and there is even a workaround if you really want to  
refer to the Ellipsis object. This is also related to the following  
point.


(2) The meaning of e.g. `Class(x=1, y=_)` versus `Class(x=1)`

In the proposed semantics, cases like this are equivalent. I can see  
why that is desirable in many cases, although Class(x=1, ...)` would  
make it more clear. A possible improvement might be to add an  
optional element to  `__match_args__` that separates optional  
arguments from required ones (although "optional" is not the same as  
"don't care").



Please let me answer these two questions in reverse order, as I think  
it makes more sense to tackle the second one first.


 **2. ATTRIBUTES**

 There actually is an important difference between `Class(x=1, y=_)`  
and `Class(x=1)` and it won't do to just write `Class(x=1,...)`  
instead.  The form `Class(x=1, y=_)` ensures that the object has an  
attribute `y`.  In a way, this is where the "duck typing" is coming in.


 The class of an object and its actual shape (i.e. the set of  
attributes it has) are rather loosely coupled in Python: there is  
usually nothing in the class itself that specifies what attributes an  
object has (other than the good sense to add these attributes in  
`__init__`).  Conceptually, it therefore makes sense to not only  
support `isinstance` but also `hasattr`/`getattr` as a means to  
specify the shape/structure of an object.


 Let me give a very simple example from Python's `AST` module.  We  
know that compound statements have a field `body` (for the suite) and  
possibly even a field `orelse` (for the `else` part).  But there is no  
common superclass for compound statements.  Hence, although it is  
shared by several objects, you cannot detect this structure through  
`isinstance` alone.  By allowing you to explicitly specify attributes  
in patterns, you can still use pattern matching notwithstanding:

```
MATCH node:
    CASE ast.stmt(body=suite, orelse=else_suite) if else_suite:
    # a statement with a non-empty else-part
    ...
    CASE ast.stmt(body=suite):
    # a compound statement without else-part
    ...
    CASE ast.stmt():
    # a simple statement
    ...
```

 The very basic form of class patterns could be described as  
`C(a_1=P_1, a_2=P_2, ...)`, where `C` is a class to be checked through  
`isinstance`, and the `a_i` are attribute names to be extracted by  
means of `getattr` to then be matched against the subpatterns `P_i`.   
In short: you specify the structure not only by class, but also by its  
actual structure in form of required attributes.


 Particularly for very simple objects, it becomes annoying to specify  
the attribute names each time.  Take, for instance, the  
`Num`-expression from the AST.  It has just a single field `n` to hold  
the actual number.  But the AST objects also contain an attribute  
`_fields = ('n',)` that not only lists the *relevant* attributes, but  
also specifies an order.  It thus makes sense to introduce a  
convention that in `Num(x)` without argument name, the `x` corresponds  
to the first field `n`.  Likewise, you write `UnarOp('+', item)`  
without the attribute names because `_fields=('op', 'operand')`  
already tells you what attributes are meant.  That is essentially the  
principle we adopted through introduction of `__match_args__`.




**1. MATCH PROTOCOL**

 I am not entirely sure what you mean by `C() == obj`.  In most cases  
you could not actually create an instance of `C` without some  
meaningful arguments for the constructor.


 The idea of the match-protocol is very similar to how you can  
already override the behaviour of `isinstance`.  It is not meant to  
completely change the semantics of what is already there, but to allow  
you to customise it (in some exciting ways ^_^).  Of course, as with  
everything customisable, you could go off and do something funny with  
it, but if it then breaks, that's quite on you.


 On the caveat that this is **NOT PART OF THIS PEP (!)**, let me try  
and explain why we would consider a match protocol in the first  
place.  The 

[Python-Dev] Another take on PEP 622

2020-07-16 Thread Tobias Kohn

Hi Everyone,

I feel there are still quite a few misconceptions around concerning  
PEP 622 and the new pattern matching feature it proposes.  Please  
allow me therefore to take another attempt at explaining the ideas  
behind PEP 622 with a different approach.  Bear in mind that I  
naturally cannot cover everything, though, and that some parts  
presented here are still slightly simplified.



Let's start with perhaps the most crucial part:

PEP 622 does **NOT** propose to introduce a `switch`-statement!

Indeed, pattern matching is much more closely related to concepts like  
regular expressions, sequence unpacking, visitor patterns, or function  
overloading.  In particular, the patterns themselves share more  
similarities with formal parameters than with expressions.


So, to start with I would like to invite you to think of PEP 622 not  
so much as proposing a new control structure in the sense of  
`if`-`elif`-`else`, `try`-`except`-`finally`, or even `switch`, but  
much rather as addressing the question: what would function  
overloading look like in Python?  Of course, this does not fully  
capture pattern matching or do it justice, either, but it might offer  
a better basis to start with.




1. Function Overloading
---

In a statically typed language, you might define a function `product`  
that either accepts two numbers or an iterable something like the  
following (I am sticking to Python syntax for simplicity):

```
def product(a: float, b: float) -> float:
    return a * b

def product(it: Iterable[float]) -> float:
    result = 1.0
    for x in it: result *= x
    return result
```
In Python, however, this needs to be done differently and the dispatch  
logic has to be put inside the function itself:

```
def product(*args):
    if len(args) == 2 and isinstance(args[0], float) and  
isinstance(args[1], float):

    return args[0] * args[1]
    elif len(args) == 1 and isinstance(args[0], Iterable):
    result = 1.0
    for x in args[0]: result *= x
    return result
```
It is this use case to begin with that pattern matching is addressing  
by introducing a more declarative way.  Each `case` represents one  
possibility of what the parameter list might look like:

```
def product(*args):
    match args:
    case (float(a), float(b)):
    return a * b
    case (Iterable(it),):
    result = 1.0
    for x in it: result *= x
    return result
```
And if you squint a little, you might even see that these parameter  
lists could almost be written in C: `(float a, float b)`.


In the context of more functional languages, you might also have seen  
some wilder stuff, where function overloading allows you to include  
literals.  One of the most prevalent examples is the factorial  
function, usually defined a bit like this:

```
def fact(0):
    return 1
def fact(n):
    return n * fact(n-1)
```
Again, when doing the same thing in Python, we put the dispatch logic  
inside the function:

```
def fact(n):
    if n == 0:
    return 1
    else:
    return n * fact(n - 1)
```
It is only natural to also allow pattern matching to express this use case:
```
def fact(arg):
    match arg:
    case 0:
    return 1
    case n:
    return n * fact(n - 1)
```
And this here is where you probably start to see good old `switch`  
coming in.  Indeed, pattern matching is powerful and versatile enough  
to act as a `switch` replacement in many cases.  But given how we  
arrived here, you might start to understand that this is a happy  
accident and not by design of the structure itself.


There is one big elephant in the room, of course: why do we need the  
`match`-line in the first place?  If all we want is to somehow express  
function overloading, couldn't we just put the cases directly into the  
function body?


In principle, yes, we could do that.  But!  As it turns out, there is  
something to be gained by a clear separation.  When actually using  
pattern matching, you might discover that you do not always want to  
define a function.  Sometimes, it is quite convenient being able to  
have a pattern like this, for instance:

```
def foo(*args):
    for arg in args:
    match arg:
    case ...
```

In all of these examples, it looks as if pattern matching is replacing  
an `if`-`elif`-`else` chain.  However, this is not entirely accurate.   
What we really want to express is the function overloading in the  
first place.  The `if`s are only needed to express the same idea in  
Python for the time being.  In other words: the individual cases here  
express independent implementations and we leave it up to the Python  
interpreter to choose the right one.




2. Visitor Pattern and Dispatch
---

If you wanted to implement something like function overloading based  
on the type of the argument, you might use the _visitor pattern_ for  
that purpose.  Thanks to 

[Python-Dev] Re: PEP 622 constant value syntax idea

2020-07-16 Thread Tobias Kohn

 Hi Mohammad,

In addition to what Rhodri James has already well pointed out, here  
are two additional thoughts on this.


At the moment, the keyword `global` is a marker to say that the  
respective variable is /modified/ by a function.  Your suggestion  
would invert that meaning and might therefore add more confusion than  
what it would solve.


Moreover, to provide yet another alternative, you could also define  
something along the lines of:

   this = sys.modules[__name__]
and then write:
   MATCH value:
       CASE this.x:
   ...
Of course, this is a bit of a hack and not that much different to  
using SimpleNamespace.  But it demonstrates that any such keyword  
would not really add that much power or save any keystrokes.


In the end, I can well imagine that a future version of Python might  
add something like a "read and compare" marker to patten matching.   
But it is probably a good idea to start with a set of fully working  
but rather minimalistic core features, and then let it grow and evolve  
with time, guided by real use cases and issues that we run into.


Kind regards,
Tobias
 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MJGSHTSPUXYCWEYCF5S6QGIMNAFP6ZSY/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 622 and variadic positional-only args

2020-07-16 Thread Tobias Kohn

 Hi Oscar
 On Wed, Jul 15, 2020 at 4:41 PM Oscar Benjamin  
 wrote:
I've taken a look through PEP 622 and I've been thinking about how  
it could be used with sympy.
 Thank you very much for taking the time to carefully elaborate an  
interesting possible use case.  I find this very helpful and a great  
basis for further/future discussions on the design of pattern matching.


A deliberate part of the current design was to address the  
/structure/shape/ of objects rather than the constructor directly  
(which could be an arbitrary complex function after all).  Writing  
`Add(args=(Mul(...), Mul(...))` for instance is therefore consistent  
as it reflects the /actual structure/ of your objects.  The  
`__match_args__` is primarily intended for rather simple object  
shapes, where it is quite obvious what attributes constitute the  
object and in which order (similar to the `_fields` attribute in AST  
nodes).


From this perspective, your use case makes an argument for something  
I would call '/variadic shape/' (in lack of a better word).   
Structurally, your objects behave like sequences or tuples, adorned  
with a specific type/class---which, again, is currently expressed as  
the "class(tuple)" pattern such as `Add((Mul(), Mul()))`.


There are two possibilities to approach this issue.  We could  
introduce patterns that extract "sequential elements" via  
`__getitem__` rather than attributes.  Or we could have a special  
method `__match__` that might return a representation of the object's  
data in sequential form.


The `__getitem__` approach turned out to come with quite a few  
complications.  In short: it is very hard to assess an object's  
possibly sequential structure in a non-destructive way.  Because of  
the multiple cases in the new pattern matching structure, we cannot  
just use an iterator as in unpacking.  And `__getitem__` itself is  
extremely versatile, being used, e.g., for both sequences as well as  
mappings.  We therefore ended up supporting only built-in structures  
like tuples, list, and dicts for now, for which the interpreter can  
easily determine how to handle `__getitem__`.


The `__match__` protocol, on the other hand, is something that we  
deferred so that we can make sure it really is powerful and well  
designed enough to handle a wide range of use cases.  One of the more  
interesting use cases, e.g., I had in mind was to destructure data  
that comes as byte strings, say (something that Rhodri James [1] has  
brought up, too).  And I think you have just added another very  
interesting use case to take into consideration.  But probably the  
best course of action is really to gain some experience and collect  
some additional use cases.


Kind regards,
Tobias

[1]   
https://mail.python.org/archives/list/python-dev@python.org/message/WD2E3K5TWR4E6PZBM4TKGHTJ7VDERTDG/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PDZZGP6TY5G2SJBY2SII4W5GCO3B64PQ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)

2020-07-14 Thread Tobias Kohn

Hi Mark,

Thank you for your message.  I might be able to answer some of the  
questions and also address some issues with the underlying assumptions  
in your email---after all, we would most certainly want to avoid  
discussing and reasoning about straw men, as you yourself have  
repeatedly pointed out.



Why the use of "shape" in "scare quotes"?


Because the `shape' of an object is not something that is a  
well-defined term, but rather addresses the intuitive understanding to  
explain what we are talking about.  It is usually the intention of the  
abstract to give a rough overview, before delving into the details and  
more formal descriptions.


Using a contrived example like this seems like a straw man.  It  
feels like it is constructed to favour the PEP, whilst being unlike  
any real code.


It is a trait of many (good) tutorials to present a new structure with  
an example that highlights its feature in an approachable way rather  
than coming up with `real code'---particularly given that real code is  
always surrounded by a larger context, which rather clouds any  
understanding.  And it would only be a straw man had we constructed a  
contrived example of unrealistic code that we then showed to be  
simplified by pattern matching.  That's not the case: it really is  
just a simple educational example to present the idea.


BTW: as a matter of course, it is constructed in favour of the PEP!


There seem to be three things you want to enhance here:
Unpacking; to avoid calls to `len`
Type checking; to avoid calls to `isinstance`.
To avoiding nesting by using complex lvalues.


Actually, no.  Pattern matching is not about _avoiding_ anything, it  
is about a more concise syntax.  The suggestion that we wanted to  
avoid calls to `len` or `isinstance` is similar to the idea of using a  
lambda to avoid a function.  The objective of pattern matching is to  
introduce a more succinct and readable way to express structure.  It's  
all still there but we want to focus on the structure rather than  
`isinstance` calls.


Pattern matching already exists in some limited way in Python.  Even  
today you can write:

  `a, b = value`

instead of:
  `a = value[0]
   b = value[1]`
   
The idea of this is, of course, not so much about avoiding item  
access, but about having a more concise syntax for it.  Moreover,  
saying this only saves a single line and is therefore rather useless  
is quite beside the point.  It is about a better representation of  
what the code is supposed to do and not about saving lines.



Saves two lines of code, but introduces two bugs!
(Assuming that the original behavior should be preserved)


Thank you for pointing out the bug in our example.  This highlights,  
however, rather the difficulty of refactoring (did I already mention  
the issue of the 'context' that real code is embedded in?), than any  
shortcoming of pattern matching as a tool.  And by the way:  
constructive critisism would perhaps include explicitly naming the two  
bugs.


For example, by adding a simple "matches" method to Node and Leaf,  
`is_tuple` can be rewritten as something like:


This assumes that you have full control over the entire code.  But if  
you are using some third-party library, you cannot "simply add a  
`matches` method" (without some substantial trickery).  Moreover,  
there is quite some work needed to define such a `matches` function as  
you propose it (including support for the ellipsis as wildcard).  In  
effect, you would have to implement large parts of the pattern  
matching for just this simple example.  Whereas we believe that it has  
merit enough to have the compiler do it for a wide range of possible  
use cases.



I really don't see you how you can claim that
`case 406:`
is more readable than
`elif response == HTTP_UPGRADE_REQUIRED:`
?
Preventing the use of symbolic constants or other complex rvalues is  
a impairment to usability.


First, let me quote what the PEP has to say on that exact example:

Although this will work, it's not necessarily what the proposal is  
focused on.


Moreover, it is usually a good idea to put constants into a separate  
namespace that further describes their meaning and intended use.  And  
that is fully supported by the syntax as proposed in the PEP.

```
match response.status:
    case HTTP_RESPONSE.UPGRADE_REQUIRED:
    ...
```


For a PEP to succeed it needs to show two things.

1. Exactly what problem is being solved, or need is to be fulfilled,  
and that is a sufficiently large problem, or need, to merit the  
proposed change.


2. That the proposed change is the best known solution for the  
problem being addressed.


IMO, PEP 622 fails on both counts.


This seems fair enough as far as the two issues are concerned.  It  
might seem indeed as if pattern matching does not add anything that  
could not be done already in Python.  Part of the problem is that  
pattern matching starts to really shine when the objects and data