Re: [Python-ideas] Why operators are useful

[Greg Ewing]

Antoine Pitrou wrote:

You are being idealistic here.  MyPy relies on typing hints being
available, and sufficiently precise.


Yes, but it doesn't require type hints for *everything*. Given
enough starting points, it can figure out the rest.

Mathematicians rely heavily on their readers being able to do the
same thing.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Antoine Pitrou]

On Tue, 19 Mar 2019 10:49:41 +1300
Greg Ewing  wrote:
> Rémi Lapeyre wrote:
> 
> > You can make "inferences from the way things are used". But the
> > comparison with maths stops here, you don’t make such inferences because 
> > your
> > object must be well defined before you start using it.  
> 
> In maths texts it's very common to see things like 'Let y = f(x)...'
> where it's been made clear beforehand (either explicitly or implicitly)
> what type f returns.

It's made clear because, when f was defined, it's explicitly spelled
out what are the source and destination domains (not sure that's the
right terminology).  That's part of how you define a function in maths.

There's no such thing in Python, unless you enforce typing hints and/or
comprehensive docstrings.

> No, you don't, because most lines in most programs allow types to
> be inferred. The reason that things like MyPy are possible and
> useful is that Python programs in practice are usually well-typed.

You are being idealistic here.  MyPy relies on typing hints being
available, and sufficiently precise.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Dan Sommers]

On 3/18/19 7:12 PM, Steven D'Aprano wrote:
> On Mon, Mar 18, 2019 at 06:34:48PM -0500, Dan Sommers wrote:
>
>> So how many of you got tired of those three statements and
>> added something like the following function to your private
>> collection of useful functions:
>>
>> def merged_mappings(mapping, other):
>> temp = mapping.copy()
>> temp.update(other)
>> return temp # no need to del temp here!
>>
>> turning two or three statements into a simple expression?
>>
>> I sure didn't.
>
> I did, only I called it "updated()".
>
> As tends to happen, what started as a three line function quickly
> became more complex. E.g. docstrings, doctests, taking an arbitrary
> number of dicts to merge, keyword arguments. The latest version of my
> updated() function is 12 lines of code and a 13 line docstring, plus
> blank lines.

Yeah, that happens.  :-)

> And then I found I could never guarantee that my private toolbox was
> available, on account of it being, you know, *private*. So I found
> myself writing:
>
> try:
>  from toolbox import updated
> except ImportError:
>  # Fall back to a basic, no-frills version.
>  def updated(d1, d2):
>  ...
>
> which then means I can't use the extra frills in my private
> version. So why have the private version when I can't use it?

The fact that you went to the trouble of writing (and testing and
documenting and maintaining) that function, if nothing else, says that
you perform this operation enough that repeating those three lines of
code started to bother you.  That's real evidence that merging mappings
is *not* a once-a-year problem.

To get back to Antoine's question, what are the use cases for
your well-honed library function?  Do you use it in every new
project, or only projects of certain kinds (GUIs, daemons, etc.)?

> Unless you are only writing for yourself, never to share your code
> with anyone else, "just put it in your private toolbox" can be
> impractical.

Depending on the nature of the project, my private toolbox is open in
both directions.  Obviously, I'm not for stealing code from elsewhere
and calling it my own, but I've certainly taken outside ideas and
incorporated them into my private toolbox.  I've also "contributed" bits
and pieces from my private toolbox into other projects; in some ways,
that's just passing folklore and tribal knowledge to the next generation
of programmers.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Mon, Mar 18, 2019 at 06:34:48PM -0500, Dan Sommers wrote:

> So how many of you got tired of those three statements and
> added something like the following function to your private
> collection of useful functions:
> 
> def merged_mappings(mapping, other):
> temp = mapping.copy()
> temp.update(other)
> return temp # no need to del temp here!
> 
> turning two or three statements into a simple expression?
> 
> I sure didn't.

I did, only I called it "updated()".

As tends to happen, what started as a three line function quickly became 
more complex. E.g. docstrings, doctests, taking an arbitrary number of 
dicts to merge, keyword arguments. The latest version of my updated() 
function is 12 lines of code and a 13 line docstring, plus blank lines.

And then I found I could never guarantee that my private toolbox was 
available, on account of it being, you know, *private*. So I found 
myself writing:

try:
from toolbox import updated
except ImportError:
# Fall back to a basic, no-frills version.
def updated(d1, d2):
...

which then means I can't use the extra frills in my private version. So 
why have the private version when I can't use it?

Unless you are only writing for yourself, never to share your code with 
anyone else, "just put it in your private toolbox" can be impractical.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Mon, Mar 18, 2019 at 05:51:08AM -0700, Rémi Lapeyre wrote:

> Maths’ typing is explicit so you don’t need to spend brain cycles to
> determine them.

Surely that depends on how formal you are being?

Maths can vary hugely in formality, even at a professional level. It is 
a terrible overgeneralisation to state that maths is always explicitly 
typed, unless your definition of mathematics is defined so narrowly as 
to exclude the majority of maths done in the world.

In my own personal experience, there is a lot of mathematics done using 
implicit typing. I've never seen anyone explicitly declare that the i, j 
or k loop variables in a sum or product is an element of ℤ, they just 
use them:

∞
∑ expression
i=0

Likewise it is very common to assume that n is an integer, x and y are 
Reals, and z is a Complex. Perhaps not in formal papers, but in less 
formal contexts, it is very common to assume specific convections used 
in the field rather than spell them out fully. For example:

https://en.wikipedia.org/wiki/Volume_of_an_n-ball

You might not give Wikipedia much credence, but I trust you won't 
object to John Baez and Terry Tao as examples of actual practicing 
mathematicians:

https://johncarlosbaez.wordpress.com/2019/03/15/algebraic-geometry/

https://terrytao.wordpress.com/2019/02/19/on-the-universality-of-the-incompressible-euler-equation-on-compact-manifolds-ii-non-rigidity-of-euler-flows/


Similarly, I've never seen anyone explicit declare the type of a 
variable used for a change in variable. Even if we've explicitly stated 
that x is a Real, we might write something like:

let u = x^2 + 3x

in order to apply the chain rule, without explicitly stating 
that u is also a Real. Why would you need to?

Its not like mathematics has a compiler which can flag type errors. We 
declare types only when needed. The rest of the time, we can use 
convention, domain-knowledge or inference to determine types.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Dan Sommers]

On 3/18/19 6:08 PM, Steven D'Aprano wrote:

On Mon, Mar 18, 2019 at 03:12:52PM +0100, Antoine Pitrou wrote:


(also, don't forget you can still use the copy() + update() method)


If we had fluent method calls, we could write:

 process(mapping.copy().update(other))

but we don't, so we use a pointless temporary variable:

 temp = mapping.copy()
 temp.update(other)
 process(temp)
 del temp  # don't pollute the global namespace


turning what ought to be a simple expression into an extra two or three
statements.


So how many of you got tired of those three statements and
added something like the following function to your private
collection of useful functions:

def merged_mappings(mapping, other):
temp = mapping.copy()
temp.update(other)
return temp # no need to del temp here!

turning two or three statements into a simple expression?

I sure didn't.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Sat, Mar 16, 2019 at 07:13:04PM -0400, Terry Reedy wrote:

> >     new = a.copy()
> >     new.update(b)
> >     # do something with new
> 
> In my census of the stdlib, already posted and noted as subject to 
> error, this was twice as common as all other non-update-in-place 
> constructions (8 to 4) and about 1/4 as common as update in place (8 to 
> 35).

Thank you Terry for doing a survey of the stdlib.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Mon, Mar 18, 2019 at 04:07:11PM +0100, Jimmy Girardet wrote:

> The syntax {**b,**c} wasn't hard to remember.
[...]
> And easy because at the end it's idiomatic.

It is only idiomatic if moderately experienced Python programmers can 
automatically read it and write it without thinking about what it means.
That's not yet the case.

Since it is new, not many people know it at all, and those who do often 
can't use it because they have to use older versions of Python where the 
syntax is not allowed.

I don't recall the discussion over whether to allow dict unpacking 
literals. Was there a PEP? Did it occur here on Python-Ideas?



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Mon, Mar 18, 2019 at 03:12:52PM +0100, Antoine Pitrou wrote:

> (also, don't forget you can still use the copy() + update() method)

If we had fluent method calls, we could write:

process(mapping.copy().update(other))

but we don't, so we use a pointless temporary variable:

temp = mapping.copy()
temp.update(other)
process(temp)
del temp  # don't pollute the global namespace


turning what ought to be a simple expression into an extra two or three 
statements.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Elazar]

בתאריך יום ג׳, 19 במרץ 2019, 0:41, מאת Greg Ewing ‏<
greg.ew...@canterbury.ac.nz>:

> Rémi Lapeyre wrote:
>
> > You can make "inferences from the way things are used". But the
> > comparison with maths stops here, you don’t make such inferences because
> your
> > object must be well defined before you start using it.
>
> In maths texts it's very common to see things like 'Let y = f(x)...'
> where it's been made clear beforehand (either explicitly or implicitly)
> what type f returns.
>
> That's completely analogous to inferring the type bound to a Python
> name from an assignment statement.
>
>  > You can track types with
> > comments but you need to comment each line.
>
> No, you don't, because most lines in most programs allow types to
> be inferred. The reason that things like MyPy are possible and
> useful is that Python programs in practice are usually well-typed.
>
> > In maths, an given object is not duck because it quacks and walks like
> > a duck, it’s
> > either part of the set of all ducks, or not.
>
> But there's no restriction on how you define the set of all ducks.
> It could be "the set of all things that quack". Duck typing is
> totally possible in mathematics, even common.
>
> For example, in linear algebra the ducks are "anything you can
> apply a linear operator to". That can cover a surprisingly large
> variety of things.
>
> > But the equation is only meaningful in a given context. Asking whether
> > f: x -> 1/x
> > is differentiable is only meaningful if we know whether x is in R, C,
> > [1; +oo[...
>
> That depends on what you want to say. You can say "let f be a
> differentiable function" and then go on to talk about things that
> depend only on the differentiability of f, without knowing
> exactly what types f operates on. Then later you can substitute
> any function you know to be differentiable, and all of those
> thing will be true for it.
>
> Mathematicians abstract things this way all the time. Groups,
> fields, vector spaces, etc. are all very abstract concepts,
> defined only as a set of objects having certain properties.
> If that's not duck typing, I don't know what is.
>

Technically, that's structural typing. Duck typing is only relevant when
there is some kind of control flow, and things need not always have all the
properties in question.

But I don't think this difference is that important in the context.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Greg Ewing]

Rémi Lapeyre wrote:


You can make "inferences from the way things are used". But the
comparison with maths stops here, you don’t make such inferences because your
object must be well defined before you start using it.


In maths texts it's very common to see things like 'Let y = f(x)...'
where it's been made clear beforehand (either explicitly or implicitly)
what type f returns.

That's completely analogous to inferring the type bound to a Python
name from an assignment statement.

> You can track types with

comments but you need to comment each line.


No, you don't, because most lines in most programs allow types to
be inferred. The reason that things like MyPy are possible and
useful is that Python programs in practice are usually well-typed.


In maths, an given object is not duck because it quacks and walks like
a duck, it’s
either part of the set of all ducks, or not.


But there's no restriction on how you define the set of all ducks.
It could be "the set of all things that quack". Duck typing is
totally possible in mathematics, even common.

For example, in linear algebra the ducks are "anything you can
apply a linear operator to". That can cover a surprisingly large
variety of things.


But the equation is only meaningful in a given context. Asking whether
f: x -> 1/x
is differentiable is only meaningful if we know whether x is in R, C,
[1; +oo[...


That depends on what you want to say. You can say "let f be a
differentiable function" and then go on to talk about things that
depend only on the differentiability of f, without knowing
exactly what types f operates on. Then later you can substitute
any function you know to be differentiable, and all of those
thing will be true for it.

Mathematicians abstract things this way all the time. Groups,
fields, vector spaces, etc. are all very abstract concepts,
defined only as a set of objects having certain properties.
If that's not duck typing, I don't know what is.

--
Greg


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Jimmy Girardet]

Hi,

Please let me share my story of non experienced python programmer.

Last year I wanted to merge three dicts  for config stuff.

I found very quickly the answer : a = {**b, **c, **d}

Sadly I was working on python 3.3 and that was nos possible to use this
syntax. I don't remember what I did next : use chain,; ChainMap,  some
comprehension or some a.update() but I was missing the "upacking syntax".

The syntax {**b,**c} wasn't hard to remember. That wasn't something
known by mathematician, experienced programmers or some artist at the
first look maybe. But It's a clear syntax easy to remember. Easy because
two arterisk `**` in python is a well known syntax due to `**kwargs` in
many functions. And easy because at the end it's idiomatic.

Many things are not straightforward in python depending where you come
from :

if __name__ == '__main__':  # Ugly

len(collection) et not collection.len() # Ugly depending your
programming background

item  in collection instead of collection.contains(i) # same thing.

list/dict comprehensions...


At the end, only a few things are straightforward at the beginning, so
d1+d2 fails isn't a big deal since you will easy remember after a quick
initial search the idiom {**d1,***d2}


Jimmy


Le 18/03/2019 à 15:12, Antoine Pitrou a écrit :
> On Mon, 18 Mar 2019 14:06:53 +
> Rhodri James  wrote:
>> On 16/03/2019 12:01, Gustavo Carneiro wrote:
>>> Already been said, but might have been forgotten, but the new proposed
>>> syntax:
>>>
>>>  new = a + b
>>>
>>> has to compete with the already existing syntax:
>>>
>>>  new = {**a, **b}
>>>   
>> That's easy.  Whether it's spelt with "+" or "|" or pretty much anything 
>> else, the operator version is clearer and cleaner.  "{**a, **b}" is a 
>> combination of operators and literal (display) syntax, and following 
>> Guido's reasoning that makes it inherently harder to interpret.  It's 
>> also ugly IMHO, but that's me.
> The question is whether it's too hard or ugly for the use cases.  In
> other words: where are the use cases where it's frequent enough to
> merge dicts that a nicer syntax is required?
>
> (also, don't forget you can still use the copy() + update() method)
>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Antoine Pitrou]

On Mon, 18 Mar 2019 14:06:53 +
Rhodri James  wrote:
> On 16/03/2019 12:01, Gustavo Carneiro wrote:
> > Already been said, but might have been forgotten, but the new proposed
> > syntax:
> > 
> >  new = a + b
> > 
> > has to compete with the already existing syntax:
> > 
> >  new = {**a, **b}
> >   
> 
> That's easy.  Whether it's spelt with "+" or "|" or pretty much anything 
> else, the operator version is clearer and cleaner.  "{**a, **b}" is a 
> combination of operators and literal (display) syntax, and following 
> Guido's reasoning that makes it inherently harder to interpret.  It's 
> also ugly IMHO, but that's me.

The question is whether it's too hard or ugly for the use cases.  In
other words: where are the use cases where it's frequent enough to
merge dicts that a nicer syntax is required?

(also, don't forget you can still use the copy() + update() method)

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rhodri James]

On 16/03/2019 12:01, Gustavo Carneiro wrote:

Already been said, but might have been forgotten, but the new proposed
syntax:

 new = a + b

has to compete with the already existing syntax:

 new = {**a, **b}



That's easy.  Whether it's spelt with "+" or "|" or pretty much anything 
else, the operator version is clearer and cleaner.  "{**a, **b}" is a 
combination of operators and literal (display) syntax, and following 
Guido's reasoning that makes it inherently harder to interpret.  It's 
also ugly IMHO, but that's me.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rémi Lapeyre]

Le 17 mars 2019 à 02:01:51, Greg Ewing
(greg.ew...@canterbury.ac.nz(mailto:greg.ew...@canterbury.ac.nz)) a
écrit:

> Richard Damon wrote:
> > Rémi, I believe, is assuming in their example that by defining the field
> > of mathematics being used, there is at least an implicit definition (if
> > not actually explicit as such a statement would typically be preceded by
> > definitions) definition of the types of the variables.
>
> In Python, we have such implicit definitions in the form
> of comments, and inferences from the way things are used.

Yes, exactly. You can make "inferences from the way things are used". But the
comparison with maths stops here, you don’t make such inferences because your
object must be well defined before you start using it. You can track types with
comments but you need to comment each line. There is also no definitions if no
comment was written.

In maths, an given object is not duck because it quacks and walks like
a duck, it’s
either part of the set of all ducks, or not.

Python’s typing is implicit
Maths’ typing is explicit so you don’t need to spend brain cycles to
determine them.

Python is imperative
Maths is functional

So once you know the type or the value of an object in maths, you
don’t have to check
all the time to make sure they did not change and spend precious brain
cycles tracking it.

I would argue that those two differences are really important when
using an operator,
When doing maths, you are always acutely aware of the context.

> > when looking at a piece of code you
> > don't necessarily know the types of the objects being used
>
> And if you look at an equation from a mathematics text without
> the context in which it appears, you won't always know what
> it means either.

But the equation is only meaningful in a given context. Asking whether
f: x -> 1/x
is differentiable is only meaningful if we know whether x is in R, C,
[1; +oo[...

> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Greg Ewing]

Richard Damon wrote:

Rémi, I believe, is assuming in their example that by defining the field
of mathematics being used, there is at least an implicit definition (if
not actually explicit as such a statement would typically be preceded by
definitions) definition of the types of the variables.


In Python, we have such implicit definitions in the form
of comments, and inferences from the way things are used.


when looking at a piece of code you
don't necessarily know the types of the objects being used


And if you look at an equation from a mathematics text without
the context in which it appears, you won't always know what
it means either.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Terry Reedy]

On 3/16/2019 8:01 AM, Gustavo Carneiro wrote:

On Sat, 16 Mar 2019 at 10:33, Steven D'Aprano 
> wrote:



The question this PEP is trying to answer is not "can we support every
use-case imaginable for a merge operator?" but "can we support the most
typical use-case?", which I believe is a version of:

     new = a.copy()
     new.update(b)
     # do something with new


In my census of the stdlib, already posted and noted as subject to 
error, this was twice as common as all other non-update-in-place 
constructions (8 to 4) and about 1/4 as common as update in place (8 to 
35).


Already been said, but might have been forgotten, but the new proposed 
syntax:

     new = a + b
has to compete with the already existing syntax:
     new = {**a, **b}


Thank you and whoever mentioned it first on this thread.  I will look at 
using this in idlelib.  There is one place where .update is called 3 
times on the same initial dict in multiple lines.


I wonder if this is only a matter of time, and over time programmers 
will become more accustomed to "{**a, **b}"


I never paid this much attention as I did not know of any immediate use 
in my personal work (and there has not been yet) and I did not think 
about looking at idlelib.


--
Terry Jan Reedy


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Gregory P. Smith]

On Sat, Mar 16, 2019 at 5:02 AM Gustavo Carneiro 
wrote:

> On Sat, 16 Mar 2019 at 10:33, Steven D'Aprano  wrote:
>
>> On Fri, Mar 15, 2019 at 10:53:31PM +, MRAB wrote:
>>
>> > There was also the suggestion of having both << and >>.
>> >
>> > Actually, now that dicts are ordered, that would provide a use-case,
>> > because you would then be able to choose which values were overwritten
>> > whilst maintaining the order of the dict on the LHS.
>>
>> Is that common enough that it needs to be built-in to dict itself?
>>
>> If it is uncommon, then the conventional solution is to subclass dict,
>> overriding the merge operator to use first-seen semantics.
>>
>> The question this PEP is trying to answer is not "can we support every
>> use-case imaginable for a merge operator?" but "can we support the most
>> typical use-case?", which I believe is a version of:
>>
>> new = a.copy()
>> new.update(b)
>> # do something with new
>>
>
> Already been said, but might have been forgotten, but the new proposed
> syntax:
>
> new = a + b
>
> has to compete with the already existing syntax:
>
> new = {**a, **b}
>
> The existing syntax is not exactly an operator in the mathematical sense
> (or is it?...), but my intuition is that it already triggers the visual
> processing part of the brain, similarly to operators.
>
> The only argument for "a + b" in detriment of "{**a, **b}" is that "a + b"
> is more easy to discover, while not many programmers are familiar with
> "{**a, **b}".
>
> I wonder if this is only a matter of time, and over time programmers will
> become more accustomed to "{**a, **b}", thereby reducing the relative
> benefit of  "a + b"?  Especially as more and more developers migrate code
> bases from Python 2 to Python 3...
>

FWIW, even as a core developer I had forgotten that the {**a, **b} syntax
existed, thanks for the reminder! :)  But that's more likely because I
rarely write code that needs to update and merge a dict or when i do it's
still 2and3 compatible.

Antoine said:

> If "+" is added to dicts, then we're overloading an already heavily used
operator.  It makes reading code more difficult.

This really resonated with me.  Reading code gives you a feel for what
possible types something could be.  The set of possibilities for + is
admittedly already quite large in Python.  But making an existing core type
start supporting + *reduces the information given to the reader* by that
one line of code.  They now have more possibilities to consider and must
seek hints from more surrounding code.

For type inferencers such as us humans 
or tools like pytype , it means we need
to consider which version Python's dict the code may be running under in
order to infer what it may mean from the code's context.  For tooling,
that's just a flag and a matter of conditionally changing code defining
dict, but for humans they need to carry the possibility of that flag with
them everywhere.

We should just promote the use of {**d1, **d2} syntax for anyone who wants
an inline updated copy.

Why?

(1) It already exists.  (insert zen of python quote here)

(2) Copying via the + operator encourages inefficient code (already true
for bytes/str/list).  A single + is fine.  But the natural human extension
to that when people want to merge a bunch of things is to use a string of
multiple operators, because that is how we're taught math.  No matter what
type we're talking about, in Python this is an efficiency antipattern.

 z = a + b + c + d + e

That's four __add__ calls.  Each of which is a copy+update/extend/append
operation for a dict/list/str respectively.

We already tell people not to do that with bytes and lists, instead using
b''.join(a,b,c,d) or z = []; z.extend(X)... calls or something from
itertools.

Given dict addition, it'd always be more efficient to join the "don't use
tons of  + operators" club (a good lint warning) and write that as
 z = {**a, **b, **c, **d, **e}.

Unless the copy+update concept is an extremely common operation, having
more than one way to do it feels like it'll cause more cognitive harm than
good.

Now (2) could *also* be used as an argument that Python should detect
chains of operators and allow those to be optimized.  That'd be a PEP of
its own and is complicated to do; technically a semantic change given how
dynamic Python is, as we do name lookups at time of use and each __add__
call could potentially have side effects changing the results of future
name lookups (the a+b could change the meaning of c).  Yes, that is
horrible and people writing code that does that deserve very bad things,
but those are the semantics we'd be breaking if we tried to detect and
support a mythical new construct like __chained_add__ being invoked when
the types of all elements being added sequentially are identical (how
identical? do subtypes count? see, complicated).

a + b + c:
   0 LOAD_GLOBAL  0 (a)
   

Re: [Python-ideas] Why operators are useful

[Richard Damon]

On 3/16/19 8:14 AM, Dan Sommers wrote:
> On 3/16/19 6:17 AM, Richard Damon wrote:
>> On 3/16/19 4:39 AM, Greg Ewing wrote:
>>> Rémi Lapeyre wrote:
 I think this omit a very important property of
 mathematic equations thought, maths is a very strongly typed language
 which can be a significant improvement for readability.
>>>
>>> Python is very strongly typed too, so I don't really see how
>>> maths is different.
>>
>> 'Strongly Typed Language' can have slightly different meaning to
>> different people. In Python, an object have a very definite type which
>> strongly defines what you can do with that object, while other languages
>> are less definitive in that aspect. But in Python, names are NOT that
>> strongly typed, as a name can be rebound to any sort of object with a
>> wide variety of types, compared to other languages where before using
>> (or at first use) a variable you need to declare the 'type' that will be
>> stored in it, and that type is all that it can hold.
>
> That's not strong vs. weak typing, that's dynamic vs. static typing.
>
> That said, I agree that different people get this wrong.  :-)

As I said, different meaning to different people, Some consider that
dynamic typing implies not a totally strong typing (since the name
doesn't have a well know type).

-- 
Richard Damon

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Kirill Balunov]

сб, 16 мар. 2019 г. в 16:02, Kirill Balunov :

> Thank you for this very thoughtful message! It reminded me of my first
> experience with the old Fortran code. You probably know that earlier in
> Fortran there were no cryptic shortcuts for writing relational operators:
> instead of `A >= B`, you had to write `A .GE. B`, or as many often wrote
> this without spaces `A.GE.B`. Even after a decent time, I still mentally
> stop and linger on these places. It's amazing that, before your message, I
> never thought about the difference in perception between `>=` and `.GE.`.
> It seems to me that, from the perception point of view, the main difference
> is that `A .GE. B` consists of readable characters and therefore we try
> to read them, while `A >= B` is perceived as a single structure (picture)
> due to unreadable `>=`. And our brain is much more better at pattern
> matching than when reading. The same is true in Python in the difference
> between the operator and method forms: `a >= b` and `a.__ge__(b)`. If we
> draw an analogy for dictionaries between:
>
> a | b   # (my preference) over `a + b`   (1)
>
> and
>
> d = d1.copy()(2)
> d = d.update(d2)
>
>
of course

d = d1.copy()(2)
d.update(d2)

just copy-pasted your example without any thought:)


> The (1) is perceived as a picture, while (2) is perceived as a short
> story. And you have to read it, and spend some extra time, and spend some
> extra energy. English is not my mother tongue, so I'm not sure that my
> words correctly convey the meaning of the analogy.
>
> Offtopic: To be honest, the idea of `+` operator overloading for
> something non numeric still does not fully fit in my numerically oriented
> mind. If I started from the beginning, I would introduce a special dunder
> for concatenation (__concat__) with the corresponding operator, something
> like seq1 .. seq2 or seq1 ~ seq2. But that ship has long sailed.
>
> With kind regards,
> -gdg
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Kirill Balunov]

Thank you for this very thoughtful message! It reminded me of my first
experience with the old Fortran code. You probably know that earlier in
Fortran there were no cryptic shortcuts for writing relational operators:
instead of `A >= B`, you had to write `A .GE. B`, or as many often wrote
this without spaces `A.GE.B`. Even after a decent time, I still mentally
stop and linger on these places. It's amazing that, before your message, I
never thought about the difference in perception between `>=` and `.GE.`.
It seems to me that, from the perception point of view, the main difference
is that `A .GE. B` consists of readable characters and therefore we try to
read them, while `A >= B` is perceived as a single structure (picture) due
to unreadable `>=`. And our brain is much more better at pattern matching
than when reading. The same is true in Python in the difference between the
operator and method forms: `a >= b` and `a.__ge__(b)`. If we draw an
analogy for dictionaries between:

a | b   # (my preference) over `a + b`   (1)

and

d = d1.copy()(2)
d = d.update(d2)

The (1) is perceived as a picture, while (2) is perceived as a short story.
And you have to read it, and spend some extra time, and spend some extra
energy. English is not my mother tongue, so I'm not sure that my words
correctly convey the meaning of the analogy.

Offtopic: To be honest, the idea of `+` operator overloading for something
non numeric still does not fully fit in my numerically oriented mind. If I
started from the beginning, I would introduce a special dunder for
concatenation (__concat__) with the corresponding operator, something like seq1
.. seq2 or seq1 ~ seq2. But that ship has long sailed.

With kind regards,
-gdg




пт, 15 мар. 2019 г. в 20:52, Guido van Rossum :

> There's been a lot of discussion about an operator to merge two dicts. I
> participated in the beginning but quickly felt overwhelmed by the endless
> repetition, so I muted most of the threads.
>
> But I have been thinking about the reason (some) people like operators,
> and a discussion I had with my mentor Lambert Meertens over 30 years ago
> came to mind.
>
> For mathematicians, operators are essential to how they think. Take a
> simple operation like adding two numbers, and try exploring some of its
> behavior.
>
> add(x, y) == add(y, x)(1)
>
> Equation (1) expresses the law that addition is commutative. It's usually
> written using an operator, which makes it more concise:
>
> x + y == y + x(1a)
>
> That feels like a minor gain.
>
> Now consider the associative law:
>
> add(x, add(y, z)) == add(add(x, y), z)(2)
>
> Equation (2) can be rewritten using operators:
>
> x + (y + z) == (x + y) + z(2a)
>
> This is much less confusing than (2), and leads to the observation that
> the parentheses are redundant, so now we can write
>
> x + y + z(3)
>
> without ambiguity (it doesn't matter whether the + operator binds tighter
> to the left or to the right).
>
> Many other laws are also written more easily using operators.  Here's one
> more example, about the identity element of addition:
>
> add(x, 0) == add(0, x) == x(4)
>
> compare to
>
> x + 0 == 0 + x == x(4a)
>
> The general idea here is that once you've learned this simple notation,
> equations written using them are easier to *manipulate* than equations
> written using functional notation -- it is as if our brains grasp the
> operators using different brain machinery, and this is more efficient.
>
> I think that the fact that formulas written using operators are more
> easily processed *visually* has something to do with it: they engage the
> brain's visual processing machinery, which operates largely subconsciously,
> and tells the conscious part what it sees (e.g. "chair" rather than "pieces
> of wood joined together"). The functional notation must take a different
> path through our brain, which is less subconscious (it's related to reading
> and understanding what you read, which is learned/trained at a much later
> age than visual processing).
>
> The power of visual processing really becomes apparent when you combine
> multiple operators. For example, consider the distributive law:
>
> mul(n, add(x, y)) == add(mul(n, x), mul(n, y))  (5)
>
> That was painful to write, and I believe that at first you won't see the
> pattern (or at least you wouldn't have immediately seen it if I hadn't
> mentioned this was the distributive law).
>
> Compare to:
>
> n * (x + y) == n * x + n * y(5a)
>
> Notice how this also uses relative operator priorities. Often
> mathematicians write this even more compact:
>
> n(x+y) == nx + ny(5b)
>
> but alas, that currently goes beyond the capacities of Python's parser.
>
> Another very powerful aspect of operator notation is that it is convenient
> to apply them to objects of different types. For example, laws (1) through
> (5) also work when n, x, y and z are same-size vectors 

Re: [Python-ideas] Why operators are useful

[Rémi Lapeyre]

Le 16 mars 2019 à 10:02:31, Greg Ewing
(greg.ew...@canterbury.ac.nz(mailto:greg.ew...@canterbury.ac.nz)) a
écrit:

> Rémi Lapeyre wrote:
> > I think this omit a very important property of
> > mathematic equations thought, maths is a very strongly typed language
> > which can be a significant improvement for readability.
>
> Python is very strongly typed too, so I don't really see how
> maths is different.

Sorry, this should have read « maths is a statically typed language ».

For example, in Python I can write:

def inverse(x):
return x ** (-1)

But this would never be accepted in maths, I should say one of

   R -> R
f: x -> x ** (-1)


   R+* -> R
f: x   -> x ** (-1)


   [1; +oo[ -> R
f: x        -> x ** (-1)



   GLn(K) -> GLn(K)
f: x      -> x ** (-1)

And in all those examples, ** would have meant something very
different and the resulting objects f are very different.


For example, the third one is Lipschitz continuous but not the
first. On the other hand, I know nothing regarding the inverse
Function in Python.

Knowing nothing about `inverse` means that every time I use it
i must determine what it means in the given context.


> > For example, a
> > mathematician working within the space of linear maps over a vector
> > space will easily recognize the meaning of every symbol in:
> >
> > f(a * x + y) = a * f(x) + f(y)
>
> Yes, but he has to remember what types are associated with
> the variables -- nothing at their point of use indicates that.
> Likewise, the reader of a Python program has to remember what
> type of object each name is expected to be bound to. If he
> can remember that, he will know what all the operators do.

The overhead to track the associated type for a given name in maths
is far lower since it is a functional language. In maths, I can just
make a mental note of it and be done with it; in Python, you can
never be sure the type of the binded object did not change unexpectedly.


> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rémi Lapeyre]

Le 16 mars 2019 à 13:15:37, Dan Sommers
(2qdxy4rzwzuui...@potatochowder.com(mailto:2qdxy4rzwzuui...@potatochowder.com))
a écrit:

> On 3/16/19 6:17 AM, Richard Damon wrote:
> > On 3/16/19 4:39 AM, Greg Ewing wrote:
> >> Rémi Lapeyre wrote:
> >>> I think this omit a very important property of
> >>> mathematic equations thought, maths is a very strongly typed language
> >>> which can be a significant improvement for readability.
> >>
> >> Python is very strongly typed too, so I don't really see how
> >> maths is different.
> >
> > 'Strongly Typed Language' can have slightly different meaning to
> > different people. In Python, an object have a very definite type which
> > strongly defines what you can do with that object, while other languages
> > are less definitive in that aspect. But in Python, names are NOT that
> > strongly typed, as a name can be rebound to any sort of object with a
> > wide variety of types, compared to other languages where before using
> > (or at first use) a variable you need to declare the 'type' that will be
> > stored in it, and that type is all that it can hold.
>
> That's not strong vs. weak typing, that's dynamic vs. static typing.
>
> That said, I agree that different people get this wrong. :-)

Yes, I’m dumb. I should have wrote « maths is a static typed language ».

This together with the fact that it is nearly purely functional means that
the overhead to know what type a given symbol is is much smaller.

If I say « let f an automorphism over E », I can write three pages of equations
and f will still be the same automorphism and E its associated vector space.

I don’t have to very carefully read each intermediate result to make
sure I did not
bind f to something else.

In Python, if I write three pages of code f could be something else so
to know its
type, I must look at all intermediate lines, including the called functions to
know what the operator will refer too.

This means that the overhead to track what a given symbol is in Python and
much larger than it is in math. It’s already the case in a given function, but
it gets worse when some of the names come from arguments, then you have to look
in the caller context which may have been written at completely another time,
by another team, increasing again the overhead.

> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rémi Lapeyre]

Le 16 mars 2019 à 13:15:37, Dan Sommers
(2qdxy4rzwzuui...@potatochowder.com(mailto:2qdxy4rzwzuui...@potatochowder.com))
a écrit:

> On 3/16/19 6:17 AM, Richard Damon wrote:
> > On 3/16/19 4:39 AM, Greg Ewing wrote:
> >> Rémi Lapeyre wrote:
> >>> I think this omit a very important property of
> >>> mathematic equations thought, maths is a very strongly typed language
> >>> which can be a significant improvement for readability.
> >>
> >> Python is very strongly typed too, so I don't really see how
> >> maths is different.
> >
> > 'Strongly Typed Language' can have slightly different meaning to
> > different people. In Python, an object have a very definite type which
> > strongly defines what you can do with that object, while other languages
> > are less definitive in that aspect. But in Python, names are NOT that
> > strongly typed, as a name can be rebound to any sort of object with a
> > wide variety of types, compared to other languages where before using
> > (or at first use) a variable you need to declare the 'type' that will be
> > stored in it, and that type is all that it can hold.
>
> That's not strong vs. weak typing, that's dynamic vs. static typing.
>
> That said, I agree that different people get this wrong. :-)

Yes, I’m dumb. I should have wrote « maths is a static typed language ».

This together with the fact that it is nearly purely functional means that
the overhead to know what type a given symbol is is much smaller.

If I say « let f an automorphism over E », I can write three pages of equations
and f will still be the same automorphism and E its associated vector space.

I don’t have to very carefully read each intermediate result to make
sure I did not
bind f to something else.

In Python, if I write three pages of code f could be something else so
to know its
type, I must look at all intermediate lines, including the called functions to
know what the operator will refer too.

This means that the overhead to track what a given symbol is in Python and
much larger than it is in math. It’s already the case in a given function, but
it gets worse when some of the names come from arguments, then you have to look
in the caller context which may have been written at completely another time,
by another team, increasing again the overhead.

> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Dan Sommers]

On 3/16/19 6:17 AM, Richard Damon wrote:

On 3/16/19 4:39 AM, Greg Ewing wrote:

Rémi Lapeyre wrote:

I think this omit a very important property of
mathematic equations thought, maths is a very strongly typed language
which can be a significant improvement for readability.


Python is very strongly typed too, so I don't really see how
maths is different.


'Strongly Typed Language' can have slightly different meaning to
different people. In Python, an object have a very definite type which
strongly defines what you can do with that object, while other languages
are less definitive in that aspect. But in Python, names are NOT that
strongly typed, as a name can be rebound to any sort of object with a
wide variety of types, compared to other languages where before using
(or at first use) a variable you need to declare the 'type' that will be
stored in it, and that type is all that it can hold.


That's not strong vs. weak typing, that's dynamic vs. static typing.

That said, I agree that different people get this wrong.  :-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Gustavo Carneiro]

On Sat, 16 Mar 2019 at 10:33, Steven D'Aprano  wrote:

> On Fri, Mar 15, 2019 at 10:53:31PM +, MRAB wrote:
>
> > There was also the suggestion of having both << and >>.
> >
> > Actually, now that dicts are ordered, that would provide a use-case,
> > because you would then be able to choose which values were overwritten
> > whilst maintaining the order of the dict on the LHS.
>
> Is that common enough that it needs to be built-in to dict itself?
>
> If it is uncommon, then the conventional solution is to subclass dict,
> overriding the merge operator to use first-seen semantics.
>
> The question this PEP is trying to answer is not "can we support every
> use-case imaginable for a merge operator?" but "can we support the most
> typical use-case?", which I believe is a version of:
>
> new = a.copy()
> new.update(b)
> # do something with new
>

Already been said, but might have been forgotten, but the new proposed
syntax:

new = a + b

has to compete with the already existing syntax:

new = {**a, **b}

The existing syntax is not exactly an operator in the mathematical sense
(or is it?...), but my intuition is that it already triggers the visual
processing part of the brain, similarly to operators.

The only argument for "a + b" in detriment of "{**a, **b}" is that "a + b"
is more easy to discover, while not many programmers are familiar with
"{**a, **b}".

I wonder if this is only a matter of time, and over time programmers will
become more accustomed to "{**a, **b}", thereby reducing the relative
benefit of  "a + b"?  Especially as more and more developers migrate code
bases from Python 2 to Python 3...

-- 
Gustavo J. A. M. Carneiro
Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Richard Damon]

On 3/16/19 4:39 AM, Greg Ewing wrote:
> Rémi Lapeyre wrote:
>> I think this omit a very important property of
>> mathematic equations thought, maths is a very strongly typed language
>> which can be a significant improvement for readability.
>
> Python is very strongly typed too, so I don't really see how
> maths is different.

'Strongly Typed Language' can have slightly different meaning to
different people. In Python, an object have a very definite type which
strongly defines what you can do with that object, while other languages
are less definitive in that aspect. But in Python, names are NOT that
strongly typed, as a name can be rebound to any sort of object with a
wide variety of types, compared to other languages where before using
(or at first use) a variable you need to declare the 'type' that will be
stored in it, and that type is all that it can hold.

Rémi, I believe, is assuming in their example that by defining the field
of mathematics being used, there is at least an implicit definition (if
not actually explicit as such a statement would typically be preceded by
definitions) definition of the types of the variables. This is part of
the rigors of the language of mathematics.  Python on the other hand,
while it allows providing a 'Type Hint' for the type of a variable,
doesn't demand such a thing, so when looking at a piece of code you
don't necessarily know the types of the objects being used (which can
also be a strength).

-- 
Richard Damon

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Sat, Mar 16, 2019 at 06:43:52AM +0400, Abdur-Rahmaan Janhangeer wrote:
> Despite my poor python skills, i don't think i'd ever use this one.
> 
> blocks = blocks + [block]  # Not good for you.

Neither would I. But I would use:


result = process(blocks + [block])


in preference to:

temp = blocks[:]
temp.append(block)
result = process(temp)
del temp  # don't pollute the global namespace


Can I make it clear that the dict addition PEP does not propose 
deprecating or removing the update method? If you need to update a dict 
in place, the update method remains the preferred One Obvious Way to do 
so, just as list.append remains the One Obvious Way to append to a 
list.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Antoine Pitrou]

On Fri, 15 Mar 2019 10:51:11 -0700
Guido van Rossum  wrote:
> Of course, everything comes at a price. You have to learn the operators,
> and you have to learn their properties when applied to different object
> types.

That's not the only price, though.  If "+" is added to dicts, then
we're overloading an already heavily used operator.  It makes reading
code more difficult.  In mathematics, this is not a problem as the
"types" of the "variables" are explicitly given.  Not in (usual) Python.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Steven D'Aprano]

On Fri, Mar 15, 2019 at 10:53:31PM +, MRAB wrote:

> There was also the suggestion of having both << and >>.
> 
> Actually, now that dicts are ordered, that would provide a use-case, 
> because you would then be able to choose which values were overwritten 
> whilst maintaining the order of the dict on the LHS.

Is that common enough that it needs to be built-in to dict itself?

If it is uncommon, then the conventional solution is to subclass dict, 
overriding the merge operator to use first-seen semantics.

The question this PEP is trying to answer is not "can we support every 
use-case imaginable for a merge operator?" but "can we support the most 
typical use-case?", which I believe is a version of:

new = a.copy()
new.update(b)
# do something with new


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Greg Ewing]

Rémi Lapeyre wrote:

I think this omit a very important property of
mathematic equations thought, maths is a very strongly typed language
which can be a significant improvement for readability.


Python is very strongly typed too, so I don't really see how
maths is different.


For example, a
mathematician working within the space of linear maps over a vector
space will easily recognize the meaning of every symbol in:

f(a * x + y) = a * f(x) + f(y)


Yes, but he has to remember what types are associated with
the variables -- nothing at their point of use indicates that.
Likewise, the reader of a Python program has to remember what
type of object each name is expected to be bound to. If he
can remember that, he will know what all the operators do.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Terry Reedy]

I agree with Guido's general comments on operators.  Modern arithmetic 
and algebra really took off with the introduction of operators.  On the 
other hand, I have seen condensed blocks of 'higher math', dense with 
operators, that I could hardly read, and that reminded me of API or Perl.


On 3/15/2019 9:39 PM, Raymond Hettinger wrote:


We don't design Python for ourselves. We design it for everyday users. Telling them that 
they can assume nothing is an anti-pattern. People do rely quite a bit on their 
intuitions. They also rely on implicit patterns already present in the language (i.e. in 
no other place is + idempotent, in no other place is + a destructive rather than 
concatenative or accumulative operator).  As for commutativity, + would be obviously 
commutative for numeric types and obviously noncommutative for sequence concatenation, 
but for dicts the non-commutativity isn't obvious at all. And since the "|" 
operator is already used for mapping views, the + operator for merging would be 
unexpected.


I agree with this argument in favor of '|' over '+'.


What is missing from the discussion is that we flat out don't need an operator 
for this.


I grepped idlelib's 60 modules for '.update('.  Ignoring the tkinter 
.update() calls, there are 3 uses of copy-update, to create a namespace 
for eval or exec, that could use the new operator.  There are 3 other 
used to update-mutate an existing dict, which would not.


If someone took a similar look as stdlib modules, I missed it.  So I 
looked at non-package top-level modules in /lib (no recursion).  The 
following likely has a few mis-classification mistakes, but most were clear.


35 dict mutate updates
7 set updates
8 dict copy-updates that could use '|' (assuming not set updates)
  # I did not think of set possibility until I had seen move of these
4 copy, intervening try or if, update
  # these either could not use '|' or only with code contortion

5 tk widget updates
10 other update methods  (a few 'dict updates might belong here)
10? 'update's in docstrings and comments
--
79 hits

--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Guido van Rossum]

On Fri, Mar 15, 2019 at 9:19 PM Inada Naoki  wrote:

> On Sat, Mar 16, 2019 at 2:51 AM Guido van Rossum  wrote:
> >
> > But I think that the folks who point out "there is already a way to do
> this" are missing the point that it really is easier to grasp the meaning
> of this:
> >
> > d = d1 + d2
> >
> > compared to this:
> >
> > d = d1.copy()
> > d = d1.update(d2)
>

[Note that I made a typo in the last line. It should be `d.update(d2)`, no
assignment.]


> > and it is not just a matter of fewer lines of code: the first form
> allows us to use our visual processing to help us see the meaning quicker
> -- and without distracting other parts of our brain (which might already be
> occupied by keeping track of the meaning of d1 and d2, for example).
>
> It seems this example is bit unfair.  It is not just method vs operator,
> because dict doesn't provide outer place version of update() method.
>

Actually most of my post was exactly about why operators can in some cases
be better than functions (which includes methods).


> In case of set, `s = s1 | s2` can be compared to `s = s1.union(s2)`.
>
> So dict example doesn't explain "why add operator instead of method?"
>

Correct, since most of the post was already explaining it. :-)


> > Of course, everything comes at a price. You have to learn the operators,
> and you have to learn their properties when applied to different object
> types. (This is true in math too -- for numbers, x*y == y*x, but this
> property does not apply to functions or matrices; OTOH x+y == y+x applies
> to all, as does the associative law.)
>
> I think behavior is more important than properties.
> When we learn operator's behavior, its property is obvious.
> So main point of using operator or not is consistency.  Same operator
> should be used for same thing as possible.
>
> I prefer | to + because the behavior of dict.update() looks similar
> set.union()
> rather than list.extend().
>

That's a separate topic and I did not mean to express an opinion on it in
this post. I simply used + because it's the simplest of all operators, and
it makes it easier for everyone to follow the visual argument.


> Another option I like is add + operator to not only dict, but also set.
> In this case, + is used to join containers by the way most natural to the
> container's type.
>
> That's what Kotlin and Scala does.  (Although Scala used ++ instead of +).
> ref:
> https://discuss.python.org/t/pep-584-survey-of-other-languages-operator-overload/977


This probably belongs in another thread (though IIRC it has been argued to
death already).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Inada Naoki]

On Sat, Mar 16, 2019 at 2:51 AM Guido van Rossum  wrote:
>
> But I think that the folks who point out "there is already a way to do this" 
> are missing the point that it really is easier to grasp the meaning of this:
>
> d = d1 + d2
>
> compared to this:
>
> d = d1.copy()
> d = d1.update(d2)
>
> and it is not just a matter of fewer lines of code: the first form allows us 
> to use our visual processing to help us see the meaning quicker -- and 
> without distracting other parts of our brain (which might already be occupied 
> by keeping track of the meaning of d1 and d2, for example).

It seems this example is bit unfair.  It is not just method vs operator,
because dict doesn't provide outer place version of update() method.

In case of set, `s = s1 | s2` can be compared to `s = s1.union(s2)`.

So dict example doesn't explain "why add operator instead of method?"


> Of course, everything comes at a price. You have to learn the operators, and 
> you have to learn their properties when applied to different object types. 
> (This is true in math too -- for numbers, x*y == y*x, but this property does 
> not apply to functions or matrices; OTOH x+y == y+x applies to all, as does 
> the associative law.)

I think behavior is more important than properties.
When we learn operator's behavior, its property is obvious.
So main point of using operator or not is consistency.  Same operator
should be used for same thing as possible.

I prefer | to + because the behavior of dict.update() looks similar set.union()
rather than list.extend().

Another option I like is add + operator to not only dict, but also set.
In this case, + is used to join containers by the way most natural to the
container's type.

That's what Kotlin and Scala does.  (Although Scala used ++ instead of +).
ref: 
https://discuss.python.org/t/pep-584-survey-of-other-languages-operator-overload/977

Regards,
-- 
Inada Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Abdur-Rahmaan Janhangeer]

Despite my poor python skills, i don't think i'd ever use this one.

blocks = blocks + [block]  # Not good for you.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Chris Angelico]

On Sat, Mar 16, 2019 at 1:27 PM Raymond Hettinger
 wrote:
>
> > On Mar 15, 2019, at 6:49 PM, Chris Angelico  wrote:
> >
> > On Sat, Mar 16, 2019 at 12:40 PM Raymond Hettinger
> >  wrote:
> >> Also, it seems like the efficiency concerns were dismissed with 
> >> hand-waving. But usually, coping and updating aren't the desired behavior. 
> >> When teaching Python, I like to talk about how the design of the language 
> >> nudges you towards fast, clear, correct code.  The principle is that 
> >> things that are good for you are put within easy reach. Things that 
> >> require more thought are placed a little further away.  That is the usual 
> >> justification for copy() and deepcopy() having to be imported rather than 
> >> being builtins.  Copying is an obvious thing to do; it is also not usually 
> >> good for you; so, we have you do one extra step to get to it.
> >>
> >
> > I'm not sure I understand this argument. Are you saying that d1+d2 is
> > bad code because it will copy the dictionary, and therefore it
> > shouldn't be done? Because the exact same considerations apply to the
> > addition of two lists, which already exists in the language. Is it bad
> > to add lists together instead of using extend()?
>
> Yes, that exactly.
>

Okay, fair. Though that doesn't necessarily push people towards
operators. Your example from below:

> blocks += [block]   # Normally done with append()
> blocks = blocks + [block]  # Not good for you.

contrasts two different ways of using operators, not operators vs
methods (and as you say, the "good" example is more usually spelled
with a method anyway). So I'm not sure what this means in terms of
dictionary merging.

I'm in favour of having both "merge to new" and "merge into this"
operations (spelled as either + and +=, or | and |=, and I'm not
fussed which of those is picked). As with everything else, "x += y"
can be assumed to be the better option over "x = x + y", but the
difference between copy/update and in-place update is the job of
augmented assignment, not an operator/method distinction.

> Consider a table in a database. Usually what people want/need/ought-to-do is 
> an SQL UPDATE rather than copy and update which would double the memory 
> requirement and be potentially many times slower.
>

(Heh. Funny you mention that example, because PostgreSQL actually
implements updates by copying a row and then marking the old one as
"will be deleted by transaction X". But that's unrelated to this, as
it's a design decision for concurrency.)

So in terms of "design pushing people to the performant option", the
main takeaway is that, if dict addition is implemented, augmented
addition should also be implemented. I don't think that's really under
dispute. The question is, should addition (or bitwise-or, same diff)
be implemented at all? Performance shouldn't kill that.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Raymond Hettinger]

> On Mar 15, 2019, at 6:49 PM, Chris Angelico  wrote:
> 
> On Sat, Mar 16, 2019 at 12:40 PM Raymond Hettinger
>  wrote:
>> Also, it seems like the efficiency concerns were dismissed with hand-waving. 
>> But usually, coping and updating aren't the desired behavior. When teaching 
>> Python, I like to talk about how the design of the language nudges you 
>> towards fast, clear, correct code.  The principle is that things that are 
>> good for you are put within easy reach. Things that require more thought are 
>> placed a little further away.  That is the usual justification for copy() 
>> and deepcopy() having to be imported rather than being builtins.  Copying is 
>> an obvious thing to do; it is also not usually good for you; so, we have you 
>> do one extra step to get to it.
>> 
> 
> I'm not sure I understand this argument. Are you saying that d1+d2 is
> bad code because it will copy the dictionary, and therefore it
> shouldn't be done? Because the exact same considerations apply to the
> addition of two lists, which already exists in the language. Is it bad
> to add lists together instead of using extend()?

Yes, that exactly.

Consider a table in a database. Usually what people want/need/ought-to-do is an 
SQL UPDATE rather than copy and update which would double the memory 
requirement and be potentially many times slower.  The same applies to Python 
lists. Unless you actually have a requirement for three distinct lists (c = a + 
b), it is almost always better to extend in place.  Adding lists rather than 
extending them is a recipe for poor performance (especially if it occurs in a 
loop):


Raymond



 Performant version 

s = socket.socket()
try:
s.connect((host, port))
s.send(request)
blocks = []
while True:
block = s.recv(4096)
if not block:
break
blocks += [block]   # Normally done with append()
page = b''.join(blocks)  
print(page.replace(b'\r\n', b'\n').decode())
finally:
s.close()

 Catastrophic version 

s = socket.socket()
try:
s.connect((host, port))
s.send(request)
blocks = []
while True:
block = s.recv(4096)
if not block:
break
blocks = blocks + [block]  # Not good for you.
page = b''.join(blocks)  
print(page.replace(b'\r\n', b'\n').decode())
finally:
s.close()

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Chris Angelico]

On Sat, Mar 16, 2019 at 12:40 PM Raymond Hettinger
 wrote:
> Also, it seems like the efficiency concerns were dismissed with hand-waving. 
> But usually, coping and updating aren't the desired behavior. When teaching 
> Python, I like to talk about how the design of the language nudges you 
> towards fast, clear, correct code.  The principle is that things that are 
> good for you are put within easy reach. Things that require more thought are 
> placed a little further away.  That is the usual justification for copy() and 
> deepcopy() having to be imported rather than being builtins.  Copying is an 
> obvious thing to do; it is also not usually good for you; so, we have you do 
> one extra step to get to it.
>

I'm not sure I understand this argument. Are you saying that d1+d2 is
bad code because it will copy the dictionary, and therefore it
shouldn't be done? Because the exact same considerations apply to the
addition of two lists, which already exists in the language. Is it bad
to add lists together instead of using extend()?

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Raymond Hettinger]

> On Mar 15, 2019, at 12:28 PM, Rhodri James  wrote:
> 
> I suspect this is a bit personal; I had sufficiently evil lecturers in my 
> university Algebra course that I still don't automatically take the 
> commutativity of "+" over a particular group as a given :-)  Nothing is 
> obvious unless you already know it.

We don't design Python for ourselves. We design it for everyday users. Telling 
them that they can assume nothing is an anti-pattern. People do rely quite a 
bit on their intuitions. They also rely on implicit patterns already present in 
the language (i.e. in no other place is + idempotent, in no other place is + a 
destructive rather than concatenative or accumulative operator).  As for 
commutativity, + would be obviously commutative for numeric types and obviously 
noncommutative for sequence concatenation, but for dicts the non-commutativity 
isn't obvious at all. And since the "|" operator is already used for mapping 
views, the + operator for merging would be unexpected.

What is missing from the discussion is that we flat out don't need an operator 
for this.  Use of explicit method names, update() or merge(), is already clear 
and already brief.  Also, if we're honest with ourselves, most of us would use 
this less than once a year. So why make a pervasive change for this?

Today, at least one PEP was rejected that had a stronger case than this 
proposal.  We should consider asking why other major languages haven't gone 
down this path. The most likely reasons are 1) insufficient need, 2) the "+" 
operator doesn't make sense, and 3) there are already clean ways to do it.

Also, it seems like the efficiency concerns were dismissed with hand-waving. 
But usually, coping and updating aren't the desired behavior. When teaching 
Python, I like to talk about how the design of the language nudges you towards 
fast, clear, correct code.  The principle is that things that are good for you 
are put within easy reach. Things that require more thought are placed a little 
further away.  That is the usual justification for copy() and deepcopy() having 
to be imported rather than being builtins.  Copying is an obvious thing to do; 
it is also not usually good for you; so, we have you do one extra step to get 
to it.


Raymond


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[MRAB]

On 2019-03-15 19:05, Jonathan Fine wrote:> Guido wrote:
>
>> There's been a lot of discussion about an operator to merge two 
dicts. I participated in the beginning but quickly felt overwhelmed by 
the endless repetition, so I muted most of the threads.

>
>> But I have been thinking about the reason (some) people like 
operators, and a discussion I had with my mentor Lambert Meertens over 
30 years ago came to mind.

>
>> For mathematicians, operators are essential to how they think.
>
> I agree about the endless repetition. I hope Steven D'A is making good
> progress with the revised PEP. I think that could help us focus
> discussion.
>
> A few days ago, I drafted but did not send a post on binary operators.
> Prompted by Guido's helpful post, I'm appending it below. My approach
> and opinions are not the same as Guido's, but have much in common.
> Perhaps later, I'll clarify where I agree with Guido, and where my
> opinions differ.
>
> Certainly, I think we have in common an emphasis on usability and in
> particular readability of code.
>
> 
> SUBJECT: Naming things: would having more binary operators help?
>
> SUMMARY
> I'm refocusing our earlier discussion on binary operators. I suggest
> we discuss the question:
> Providing more binary operators. When would this make naming things
> this easier? And when harder?
>
> THE PROBLEM
> Naming things is hard.
>
> For example https://hilton.org.uk/blog/why-naming-things-is-hard
> "Naming is communication. Bad names prevent code from clearly
> communicating its intent, which is why code with obfuscated names is
> spectacularly hard to understand. The compiler might not care, but the
> humans benefit from naming that communicates effectively."
>
> AN EXAMPLE
> One person wrote:
> using + to merge dicts is simple, non-disruptive, and unlikely to
> really confuse anyone - so why not?
>
> Another person proposed:
> d1 << d2 merges d2 into a copy of d1 and returns it, with keys from d2
> overriding keys from d2.
>
> A third person wrote:
> "|" (especially "|=") *is* suitable for "update"
> [So] reserve "+" for some alternative future commutative extension
>
[snip]
There was also the suggestion of having both << and >>.

Actually, now that dicts are ordered, that would provide a use-case, 
because you would then be able to choose which values were overwritten 
whilst maintaining the order of the dict on the LHS.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rhodri James]

On 15/03/2019 18:54, Raymond Hettinger wrote:

So, we have to ask whether we're stretching too far from "operators are good" to "we 
need this operator".  Here are some considerations:

Frequency of usage:   Math provides ∑ and ∏ because they are common. It doesn't 
provide a special operator for sqrt(c**2 - b**2) because the latter is less 
fundamental and less common.  To me, f=d.copy() followed by f.update(e) arises 
so rarely that an operator isn't warranted.  The existing code is already 
concise, clear, and rare.


I think the "less fundamental" in your argument is more relevant than 
the "less common".  Mathematicians will cheerfully invent operators for 
whatever is fundamental to their field and then use them twice in a 
paper, but still write out fully common combinations.  I would suggest 
that merging is merging is a fairly fundamental operation for 
dictionaries, so is a good candidate for an operator.


The combination "f=d.copy(); f.update(e)" is rare in my code.  I suspect 
that's partly because it doesn't occur to me that I can do it.  Guido's 
argument about recognisability is strong here.  I know that 
dict.update() exists, and that I can destructively merge dictionaries. 
The extra step of doing the copy first for a non-destructive merge makes 
for a much less memorable pattern, to the point where I just don't think 
of it unless it would be more than usually useful.  "f = d | e" (however 
it gets spelled) is much easier to remember the existence of.



Familiarity:  We know about + because we use it a lot in addition and concatenation contexts. However, a symbol like ⊗ is more opaque unless we're 
using it every day for a particular purpose.  To me, the "+" operator implies "add/extend" semantics rather than 
"replace" semantics.  Successive applications of "+" are never idempotent unless one operand is an identity element.  So for me, 
"+" isn't familiar for dict merges.  Loosely put, it isn't "plus-like".  I think this is why so many other languages decided not 
use "+" for dict merges even when that would have been a trivially easy implementation choice.


I'm beginning to be swayed by the arguments that merging is more 
"or-like" and the right analogy is with set union.  Personally I don't 
find "|" for set union at all obvious, but that argument was lost long 
ago, and like I said it's just personal.  I don't have the same problem 
you have with the semantics of "+", but when I was a maths student I was 
used to using "+" as an entirely generic operator not necessarily 
meaning addition, so it's probably just me.



Obviousness: When working with "+" on numeric types, it is obvious it should be commutative. When using 
"+" when sequence types, it is obvious that concatenation is non-commutative. When using "+" for 
mapping types, it is not obvious that it isn't commutative. Likewise, it isn't obvious that "+" is a 
destructive operation for mappings (consider that adding to a log file never destroys existing log entries, while 
updating a dict will overwrite existing values).


I suspect this is a bit personal; I had sufficiently evil lecturers in 
my university Algebra course that I still don't automatically take the 
commutativity of "+" over a particular group as a given :-)  Nothing is 
obvious unless you already know it.


(There is a probably apocryphal tale of a lecturer in full flow saying 
"It is obvious that..." and pausing.  He then turned to the blackboard 
and scribbled furiously in one corner for five minutes.  "I was right," 
he said triumphantly, "it is obvious!")



Harmony: The operators on dict views use "|" but regular dicts would use "+". 
That doesn't seem harmonious.


Yes, that's probably the killer argument against "+", damn it.


Impact: When a class in the standard library adds a method or operator, the 
reverberations are felt only locally.  In contrast, the dict API is 
fundamental.  Changing it will reverberate for years. It will be felt in the 
ABCs, typeshed, and every mapping-like object.  IMO such an impactful change 
should only be made if it adds significant new functionality rather than 
providing a slightly shorter spelling of something we already have.


I am inclined that adding significant new utility (which this does) is 
also a good enough reason to make such an impactful change.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Jonathan Fine]

Raymond Hettinger wrote:

> Frequency of usage:   Math provides ∑ and ∏ because they are common. It 
> doesn't provide a special operator for sqrt(c**2 - b**2) because the latter 
> is less fundamental and less common.

Here's some more information. Below is an example of an area, where
sqrt(c**2 - b**2) is both fundamental and common. And that it might be
helpful for Python to provide a (named) function for this operation.
Whether or not, or how, a symbolic expression should be provided is
another question.

This one example by itself does not refute Raymond's argument. I
certainly think caution is required, in promoting the needs of one
group of users at the expense of another. Best avoided, if possible.

GORY DETAILS
Don Knuth, in METAFONT, implemented special '++' and '+-+' operators,
that he called Pythagorean addition and subtraction. The latter is
precisely Raymond's sqrt(c**2 - b**2), but calculated more efficiently
and accurately.

This is described on page 66 of Don Knuth's METAFONT Book.
https://ctan.org/tex-archive/systems/knuth/dist/mf/mfbook.tex

The `^|++|' operation is called {\sl^{Pythagorean addition}\/}; $a\pyth+b$
is the same thing as $\sqrt{\stt a^2+b^2}$. Most of the ^{square root}
operations in computer programs could probably be avoided if $++$ were
more widely available, because people seem to want square roots primarily
when they are computing distances. Notice that $a\pyth+b\pyth+c=
\sqrt{\stt a^2+b^2+c^2}$; we have the identity $(a\pyth+b)\pyth+c=a\pyth+(
b\pyth+c)$ as well as $a\pyth+b=b\pyth+a$. It is better to use Pythagorean
addition than to calculate $\sqrt{\stt a^2+b^2}$, because the computation
of $a^2$ and $b^2$ might produce numbers that are too large even when
$a\pyth+b$ is rather small. There's also an inverse operation,
^{Pythagorean subtraction}, which is denoted by `^|+-+|'; the quantity
$a\mathbin{+{-}+}b$ is equal to $\sqrt{\stt a^2-b^2}$.

ASIDE - wikipedia
In https://en.wikipedia.org/wiki/Pythagorean_addition, wikipedia using
the symbol \oplus for Pythagorean addition, and does not mention
Pythagorean subtraction.

ASIDE- \pyth and Python
Don Knuth uses \pyth as a macro (shorthand) for Pythagorean. It's got
nothing to do with Python. The METAFONT book goes back to 1986, which
predates Pyth-on by about 5 years. That said, Pythagoras was the
founder of a new way of life, and Python is a new way of programming.

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Rémi Lapeyre]

 Le 15 mars 2019 à 18:52:51, Guido van Rossum
(gu...@python.org(mailto:gu...@python.org)) a écrit:

…

> The power of visual processing really becomes apparent when you combine 
> multiple operators. For example, consider the distributive law:
>
> mul(n, add(x, y)) == add(mul(n, x), mul(n, y)) (5)
>
> That was painful to write, and I believe that at first you won't see the 
> pattern (or at least you wouldn't have immediately seen it if I hadn't 
> mentioned this was the distributive law).
>
> Compare to:
>
> n * (x + y) == n * x + n * y (5a)

Thanks for the insight. I think this omit a very important property of
mathematic equations thought, maths is a very strongly typed language
which can be a significant improvement for readability. For example, a
mathematician working within the space of linear maps over a vector
space will easily recognize the meaning of every symbol in:

f(a * x + y) = a * f(x) + f(y)

And know that the + in the above expression is very different from the
meaning of + in:

x = a * y + z

when he is working over the C complex field.

For example, he instinctively knows what 1 / z means in the second
case but that 1 / f in the first is completely bogus.

In Python there is not that much contextual information, but we can
use explicit names to overcome this, for example if I wrote:

o = d1 + d2 + d3

you would have no idea what this is but:

options = defaults + environment_variables + command_line_arguments

is meaningful.

...
> Of course, it's definitely possible to overdo this -- then you get Perl. But 
> I think that the folks who point out "there is already a way to do this" are 
> missing the point that it really is easier to grasp the meaning of this:
>
> d = d1 + d2
>
> compared to this:
>
> d = d1.copy()
> d = d1.update(d2)

Of course. I may have missed something but I don’t understand why
INADA Naoki proposal does not get more attention.

It is not binary, and we could use names to convey the meaning of the operation:

options = dict.merge(defaults, environment_variables, command_line_arguments)

His alternative options = defaults.merge(environment_variables,
command_line_arguments) could also be used if preferred.

Is there really something wrong with this? It would do exactly what
most proponent of + want but could be more readable.

I agree that the argument of performance may not be very strong as
most of the time, the updated dict might be smalls, but it would also
solved this elegantly.

I’m sorry if what I’m saying is not clear or I’m not able to convey my
thoughts clearly as English is not my mother tongue, many others are
better suited then me to discuss this proposal on this list but I
don’t understand why this possibility is not more discussed.

Rémi
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why operators are useful

[Jonathan Fine]

Guido wrote:

> There's been a lot of discussion about an operator to merge two dicts. I 
> participated in the beginning but quickly felt overwhelmed by the endless 
> repetition, so I muted most of the threads.

> But I have been thinking about the reason (some) people like operators, and a 
> discussion I had with my mentor Lambert Meertens over 30 years ago came to 
> mind.

> For mathematicians, operators are essential to how they think.

I agree about the endless repetition. I hope Steven D'A is making good
progress with the revised PEP. I think that could help us focus
discussion.

A few days ago, I drafted but did not send a post on binary operators.
Prompted by Guido's helpful post, I'm appending it below. My approach
and opinions are not the same as Guido's, but have much in common.
Perhaps later, I'll clarify where I agree with Guido, and where my
opinions differ.

Certainly, I think we have in common an emphasis on usability and in
particular readability of code.


SUBJECT: Naming things: would having more binary operators help?

SUMMARY
I'm refocusing our earlier discussion on binary operators. I suggest
we discuss the question:
Providing more binary operators. When would this make naming things
this easier? And when harder?

THE PROBLEM
Naming things is hard.

For example https://hilton.org.uk/blog/why-naming-things-is-hard
"Naming is communication. Bad names prevent code from clearly
communicating its intent, which is why code with obfuscated names is
spectacularly hard to understand. The compiler might not care, but the
humans benefit from naming that communicates effectively."

AN EXAMPLE
One person wrote:
using + to merge dicts is simple, non-disruptive, and unlikely to
really confuse anyone - so why not?

Another person proposed:
d1 << d2 merges d2 into a copy of d1 and returns it, with keys from d2
overriding keys from d2.

A third person wrote:
"|" (especially "|=") *is* suitable for "update"
[So] reserve "+" for some alternative future commutative extension

A fourth person provided a '+' operator on a subclass of dict, that
merged items using a special addition on numeric values.

A fifth person suggested adding arrow operators, with symbols '->' and '<-'.

A six person said that '<-' would break code valid such as '-2<-1'.

A seventh person noted that annotations already use '->' as a symbol.

An eighth person said the infix module allows you to write a @cup@ b

An nineth person (me) will soon suggest that we add dict.gapfill
current.update(changes) # If conflict, prefer value in changes.
options.gapfill(defaults) # If conflict, prefer value in options.
(and so '+' or '|' for update not so clear).

BENEFITS OF BINARY OPERATORS
Binary operators, such as '+' allow us to write:

c = a + b # Infix notation
a += x # mutation or augmented assignment
a[key] += x # as above, but 'in place'

At school, when we learn arithmetic, we learn it using infix notation.
   Two plus two is four.
   Seven times eight is fifty-six.

I think the above indicates the benefits of binary operators.
Particular when binary operation does not mutate the operands.

DIFFICULTIES
Sometimes, having few names to choose makes naming things easier.
That's obvious. Sometimes, having a wider choose makes naming things
easier. Think Unicode's visually similar characters.

At present, Python has 20 operators, the majority being binary
evaluation operators.
https://docs.python.org/3/reference/lexical_analysis.html#operators
+   -   *   **  /   //  %  @
<<  >>  &   |   ^   ~
<   >   <=  >=  ==  !=

The last row gives the (binary) comparison operators. The symbols '^'
and '~' are unary operators. For clarity, I'm thinking of the binary
evaluation operators, or in other words '+' through to '|'.

Aside: '+' and '-' can be used as binary and unary operators.
>>> 5 + -- +++  + -- - 4
1

ONE SUGGESTION
The twelve binary evaluation operators sounds a lot, but perhaps some
users will need more.  Even it might be nice if the same symbol didn't
have too many different meanings. Python2 used '/' for both float and
integer division. To reduce cognitive overload, Python3 introduced
'//' for integer division.

>>> 4.3 // 2.1
2.0

For example https://oeis.org/wiki/List_of_LaTeX_mathematical_symbols#Arrows
lists 10 types of horizontal arrow, and 6 types of vertical arrow.

Providing more binary operators is the motivation for my proposal

 # https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle
 len( A @cup B) == len( A ) + len( B ) - len( A @cap B )

(By the way, we might prefer 'union' and 'intersection' to 'cup' and
'cap'. Also there are alternatives, such as using $ instead of @, or
using Unicode Math Characters.)

If there is a shared wish to have more binary operators, it might then
be useful to discuss how.

DISCUSSION QUESTION
Please 

Re: [Python-ideas] Why operators are useful

[Raymond Hettinger]

> On Mar 15, 2019, at 10:51 AM, Guido van Rossum  wrote:
> 
> The general idea here is that once you've learned this simple notation, 
> equations written using them are easier to *manipulate* than equations 
> written using functional notation -- it is as if our brains grasp the 
> operators using different brain machinery, and this is more efficient.

There is no question that sometimes operators can be easier to manipulate and 
reason about than equivalent methods.  The use of "+" and "*" are a major win 
for numeric and sequence types.

There is also no question that sometimes method names are better than operators 
(otherwise, we wouldn't use method names at all).  APL is an extreme example of 
a rich set of operators being both powerful and opaque.

So, we have to ask whether we're stretching too far from "operators are good" 
to "we need this operator".  Here are some considerations:

Frequency of usage:   Math provides ∑ and ∏ because they are common. It doesn't 
provide a special operator for sqrt(c**2 - b**2) because the latter is less 
fundamental and less common.  To me, f=d.copy() followed by f.update(e) arises 
so rarely that an operator isn't warranted.  The existing code is already 
concise, clear, and rare.

Familiarity:  We know about + because we use it a lot in addition and 
concatenation contexts. However, a symbol like ⊗ is more opaque unless we're 
using it every day for a particular purpose.  To me, the "+" operator implies 
"add/extend" semantics rather than "replace" semantics.  Successive 
applications of "+" are never idempotent unless one operand is an identity 
element.  So for me, "+" isn't familiar for dict merges.  Loosely put, it isn't 
"plus-like".  I think this is why so many other languages decided not use "+" 
for dict merges even when that would have been a trivially easy implementation 
choice.

Obviousness: When working with "+" on numeric types, it is obvious it should be 
commutative. When using "+" when sequence types, it is obvious that 
concatenation is non-commutative. When using "+" for mapping types, it is not 
obvious that it isn't commutative. Likewise, it isn't obvious that "+" is a 
destructive operation for mappings (consider that adding to a log file never 
destroys existing log entries, while updating a dict will overwrite existing 
values).

Harmony: The operators on dict views use "|" but regular dicts would use "+". 
That doesn't seem harmonious.

Impact: When a class in the standard library adds a method or operator, the 
reverberations are felt only locally.  In contrast, the dict API is 
fundamental.  Changing it will reverberate for years. It will be felt in the 
ABCs, typeshed, and every mapping-like object.  IMO such an impactful change 
should only be made if it adds significant new functionality rather than 
providing a slightly shorter spelling of something we already have.



Raymond

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/