[Python-ideas] dict.merge(d1, d2, ...) (Counter proposal for PEP 584)

[INADA Naoki]

I think some people in favor of PEP 584 just want
single expression for merging dicts without in-place update.

But I feel it's abuse of operator overload.  I think functions
and methods are better than operator unless the operator
has good math metaphor, or very frequently used as concatenate
strings.

This is why function and methods are better:

* Easy to search.
* Name can describe it's behavior better than abused operator.
* Simpler lookup behavior. (e.g. subclass and __iadd__)

Then, I propose `dict.merge` method.  It is outer-place version
of `dict.update`, but accepts multiple dicts.  (dict.update()
can be updated to accept multiple dicts, but it's not out of scope).

* d = d1.merge(d2)  # d = d1.copy(); d.update(d2)
* d = d1.merge(d2, d3)  # d = d1.copy(); d.update(d2); d2.update(d3)
* d = d1.merge(iter_of_pairs)
* d = d1.merge(key=value)


## Merits of dict.merge() over operator +

* Easy to Google (e.g. "python dict merge").
* Easy to help(dict.merge). (or dict.merge? in IPython)
* No inefficiency of d1+d2+d3+...+dN, or sum(list_of_many_dicts)
* Type of returned value is always same to d1.copy().  No issubclass,
no __iadd__.

## Why not dict.updated()?

sorted() is a function so it looks different from L.sort()
But d.updated() is very similar to d.update() for human eyes.

## How about d1 - d2?

If it is really useful, it can be implemented as method too.

dict.discard(sequence_of_keys)

Regards,
-- 
INADA Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Serhiy Storchaka]

04.03.19 21:24, Guido van Rossum пише:
* Dicts are not like sets because the ordering operators (<, <=, >, >=) 
are not defined on dicts, but they implement subset comparisons for 
sets. I think this is another argument pleading against | as the 
operator to combine two dicts.


Well, I suppose that the next proposition will be to implement the 
ordering operators for dicts. Because why not? Lists and numbers support 
them. /sarcasm/


Jokes aside, dicts have more common with sets than with sequences. Both 
can not contain duplicated keys/elements. Both have the constant 
computational complexity of the containment test. For both the size of 
the merging/unioning can be less than the sum of sizes of original 
containers. Both have the same restrictions for keys/elements (hashability).


* Regarding how to construct the new set in __add__, I now think this 
should be done like this:


class dict:
     
     def __add__(self, other):
     
     new = self.copy()  # A subclass may or may not choose to override
     new.update(other)
     return new

AFAICT this will give the expected result for defaultdict -- it keeps 
the default factory from the left operand (i.e., self).


No one builtin type that implements __add__ uses the copy() method. Dict 
would be the only exception from the general rule.


And it would be much less efficient than {**d1, **d2}.

* Regarding how often this is needed, we know that this is proposed and 
discussed at length every few years, so I think this will fill a real need.


And every time this proposition was rejected. What has been changed 
since it was rejected the last time? We now have the expression form of 
dict merging ({**d1, **d2}), this should be decrease the need of the 
plus operator for dicts.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Anders Hovmöller]

> Adding the + operator for dictionaries feels like it would be a mistake in 
> that it offers at most sugar-y benefits, but introduces the significant 
> drawback of making it easier to introduced unintended errors. 

I disagree. This argument only really applies to the case "a = a + b", not "a = 
b + c". Making it easier and more natural to produce code that doesn't mutate 
in place is something that should reduce errors, not make them more common.

The big mistake here was * for strings which is unusual, would be just as well 
served by a method, and will ensure that type errors blow up much later than it 
could have been. This type of mistake for dicts when you expected numbers is a 
much stronger argument against this proposal in my opinion. Let's not create 
another pitfall! The current syntax is a bit unwieldy but is really fine. 

/ Anders 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[INADA Naoki]

On Tue, Mar 5, 2019 at 12:02 AM Stefan Behnel  wrote:
>
> INADA Naoki schrieb am 04.03.19 um 11:15:
> > Why statement is not enough?
>
> I'm not sure I understand why you're asking this, but a statement is "not
> enough" because it's a statement and not an expression. It does not replace
> the convenience of an expression.
>
> Stefan
>

It seems tautology and say nothing.
What is "convenience of an expression"?
Is it needed to make Python more readable language?

Anyway, If "there is expression" is the main reason for this proposal, symbolic
operator is not necessary.
`new = d1.updated(d2)` or `new = dict.merge(d1, d2)` are enough.

Python preferred name over symbol in general.
Symbols are readable and understandable only when it has good
math metaphor.

Sets has symbol operator because it is well known in set in math,
not because set is frequently used.

In case of dict, there is no simple metaphor in math.
It just cryptic and hard to Google.

-- 
INADA Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Brandt Bucher]

I agree with David here. Subtraction wasn’t even part of the original 
discussion — it seems that it was only added as an afterthought because Guido 
felt they were natural to propose together and formed a nice symmetry.

It’s odd that RHS values are not used at all, period. Further, there’s no 
precedent for bulk sequence/mapping removals like this... except for sets, for 
which it is certainly justified.

I’ve had the opportunity to play around with my reference implementation over 
the last few days, and despite my initial doubts, I have *absolutely* fallen in 
love with dictionary addition — I even accidentally tried to += two 
dictionaries at work on Friday (a good, but frustrating, sign). For context, I 
was updating a module-level mapping with an imported one, a use case I hadn’t 
even previously considered.

I have tried to fall in love with dict subtraction the same way, but every code 
sketch/test I come up with feels contrived and hack-y. I’m indifferent towards 
it, at best.

TL;DR: I’ve lived with both for a week. Addition is now habit, subtraction is 
still weird.

> Nice branch name! :)

I couldn’t help myself.

Brandt
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Amber Yust]

Adding the + operator for dictionaries feels like it would be a mistake in
that it offers at most sugar-y benefits, but introduces the significant
drawback of making it easier to introduced unintended errors. This would be
the first instance of "addition" where the result can potentially
lose/overwrite data (lists and strings both preserve the full extent of
each operand; Counters include the full value from each operand, etc).

Combining dictionaries is fundamentally an operation that requires more
than one piece of information, because there's no single well-defined way
to combine a pair of them. Off the top of my head, I can think of at least
2 different common options (replacement aka .update(), combination of
values a la Counter). Neither of these is really a more valid "addition" of
dictionaries.

For specific dict-like subclasses, addition may make sense - Counter is a
great example of this, because the additional context adds definition to
the most logical method via which two instances would be combined. If
anything, this seems like an argument to avoid implementing __ladd__ on
dict itself, to leave the possibility space open for interpretation by more
specific classes.

*From: *Raymond Hettinger 
*Date: *Mon, Mar 4, 2019 at 9:53 PM
*To: *Guido van Rossum
*Cc: *python-ideas


>
> > On Mar 4, 2019, at 11:24 AM, Guido van Rossum  wrote:
> >
> > * Regarding how often this is needed, we know that this is proposed and
> discussed at length every few years, so I think this will fill a real need.
>
> I'm not sure that conclusion follows from the premise :-)  Some ideas get
> proposed routinely because they are obvious things to propose, not because
> people actually need them.  One hint is that the proposals always have
> generic variable names, "d = d1 + d2", and another is that they are almost
> never accompanied by actual use cases or real code that would be made
> better. I haven't seen anyone in this thread say they would use this more
> than once a year or that their existing code was unclear or inefficient in
> any way.  The lack of dict addition support in other languages (like Java
> example) is another indicator that there isn't a real need -- afaict there
> is nothing about Python that would cause us to have a unique requirement
> that other languages don't have.
>
> FWIW, there are some downsides to the proposal -- it diminishes some of
> the unifying ideas about Python that I typically present on the first day
> of class:
>
> * One notion is that the APIs nudge users toward good code.  The
> "copy.copy()" function has to be imported -- that minor nuisance is a
> subtle hint that copying isn't good for you.  Likewise for dicts, writing
> "e=d.copy(); e.update(f)" is a minor nuisance that either serves to
> dissuade people from unnecessary copying or at least will make very clear
> what is happening.  The original motivating use case for ChainMap() was to
> make a copy free replacement for excessively slow dict additions in
> ConfigParser.  Giving a plus-operator to mappings is an invitation to
> writing code that doesn't scale well.
>
> * Another unifying notion is that the star-operator represents repeat
> addition across multiple data types.  It is a nice demo to show that "a * 5
> == a + a + a + a + a" where "a" is an int, float, complex, str, bytes,
> tuple, or list.  Giving __add__() to dicts breaks this pattern.
>
> * When teaching dunder methods, the usual advice regarding operators is to
> use them only when their meaning is unequivocal; otherwise, have a
> preference for named methods where the method name clarifies what is being
> done -- don't use train+car to mean train.shunt_to_middle(car). For dicts
> that would mean not having the plus-operator implement something that isn't
> inherently additive (it applies replace/overwrite logic instead), that
> isn't commutative, and that isn't linear when applied in succession
> (d1+d2+d3).
>
> * In the advanced class where C extensions are covered, the organization
> of the slots is shown as a guide to which methods make sense together:
> tp_as_number, tp_as_sequence, and tp_as_mapping.  For dicts to gain the
> requisite methods, they will have to become numbers (in the sense of
> filling out the tp_as_number slots).  That will slow down the abstract
> methods that search the slot groups, skipping over groups marked as NULL.
> It also exposes method groups that don't typically appear together,
> blurring their distinction.
>
> * Lastly, there is a vague piece of zen-style advice, "if many things in
> the language have to change to implement idea X, it stops being worth it".
>  In this case, it means that every dict-like API and the related abstract
> methods and typing equivalents would need to grow support for addition in
> mappings (would it even make sense to add to shelve objects or os.environ
> objects together?)
>
> That's my two cents worth.  I'm ducking out now (nothing more to offer on
> the subject). Guido's participation in 

Re: [Python-ideas] Add a "week" function or attribute to datetime.date

[Steve Barnes]

If anybody is looking for such components then wx.DateTime 
(https://wxpython.org/Phoenix/docs/html/datetime_overview.html) it is derived 
from wxDateTime (https://docs.wxwidgets.org/3.1/classwx_date_time.html) and 
should support all of its methods including things like DST changes, etc., 
supported dates from about 4714 B.C. to some 480 million years in the future.

From: Python-ideas  On 
Behalf Of Christopher Barker
Sent: 04 March 2019 21:09
To: python-ideas 
Subject: Re: [Python-ideas] Add a "week" function or attribute to datetime.date

There are all sorts of "Calendar" operations one might want -- I think those 
belong in a separate library, rather than a few tacked on to datetime.

-CHB


On Fri, Mar 1, 2019 at 2:48 AM Robert Vanden Eynde 
mailto:robertv...@gmail.com>> wrote:
Currently one can do week = d.isocalendar()[1]

The iso definition of a week number has some nice properties.

robertvandeneynde.be

On Fri, 1 Mar 2019, 11:44 Antonio Galán, 
mailto:angala@gmail.com>> wrote:
The week number is usually refered to the week of the year, but the week of the 
month is also interesting, for example for some holiday which depend on the 
week number of the month, so in analogy with "weekday" we can use "yearweek" 
and "monthweek"
El vie., 1 de marzo de 2019 9:33, Adrien Ricocotam 
mailto:ricoco...@gmail.com>> escribió:
I like the idea. But how to distinguish it from the number of week past since 
the beginning of the month ?

But that’s great.

On Fri 1 Mar 2019 at 09:31, Antonio Galán 
mailto:angala@gmail.com>> wrote:
Hi, datetime.date.today() (or other day)  has attributes .year and .month wich 
return the year and the month of that date, also it has a function weekday() 
wich return the number of the day in the week.

I think it is a good idea add a function or attribute "week" wich return the 
number of the week on the year. It is useful to execute scripts once a week for 
example.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Raymond Hettinger]

> On Mar 4, 2019, at 11:24 AM, Guido van Rossum  wrote:
> 
> * Regarding how often this is needed, we know that this is proposed and 
> discussed at length every few years, so I think this will fill a real need.

I'm not sure that conclusion follows from the premise :-)  Some ideas get 
proposed routinely because they are obvious things to propose, not because 
people actually need them.  One hint is that the proposals always have generic 
variable names, "d = d1 + d2", and another is that they are almost never 
accompanied by actual use cases or real code that would be made better. I 
haven't seen anyone in this thread say they would use this more than once a 
year or that their existing code was unclear or inefficient in any way.  The 
lack of dict addition support in other languages (like Java example) is another 
indicator that there isn't a real need -- afaict there is nothing about Python 
that would cause us to have a unique requirement that other languages don't 
have.

FWIW, there are some downsides to the proposal -- it diminishes some of the 
unifying ideas about Python that I typically present on the first day of class:

* One notion is that the APIs nudge users toward good code.  The "copy.copy()" 
function has to be imported -- that minor nuisance is a subtle hint that 
copying isn't good for you.  Likewise for dicts, writing "e=d.copy(); 
e.update(f)" is a minor nuisance that either serves to dissuade people from 
unnecessary copying or at least will make very clear what is happening.  The 
original motivating use case for ChainMap() was to make a copy free replacement 
for excessively slow dict additions in ConfigParser.  Giving a plus-operator to 
mappings is an invitation to writing code that doesn't scale well.

* Another unifying notion is that the star-operator represents repeat addition 
across multiple data types.  It is a nice demo to show that "a * 5 == a + a + a 
+ a + a" where "a" is an int, float, complex, str, bytes, tuple, or list.  
Giving __add__() to dicts breaks this pattern.

* When teaching dunder methods, the usual advice regarding operators is to use 
them only when their meaning is unequivocal; otherwise, have a preference for 
named methods where the method name clarifies what is being done -- don't use 
train+car to mean train.shunt_to_middle(car). For dicts that would mean not 
having the plus-operator implement something that isn't inherently additive (it 
applies replace/overwrite logic instead), that isn't commutative, and that 
isn't linear when applied in succession (d1+d2+d3).

* In the advanced class where C extensions are covered, the organization of the 
slots is shown as a guide to which methods make sense together: tp_as_number, 
tp_as_sequence, and tp_as_mapping.  For dicts to gain the requisite methods, 
they will have to become numbers (in the sense of filling out the tp_as_number 
slots).  That will slow down the abstract methods that search the slot groups, 
skipping over groups marked as NULL.  It also exposes method groups that don't 
typically appear together, blurring their distinction.

* Lastly, there is a vague piece of zen-style advice, "if many things in the 
language have to change to implement idea X, it stops being worth it".   In 
this case, it means that every dict-like API and the related abstract methods 
and typing equivalents would need to grow support for addition in mappings 
(would it even make sense to add to shelve objects or os.environ objects 
together?)

That's my two cents worth.  I'm ducking out now (nothing more to offer on the 
subject). Guido's participation in the thread has given it an air of 
inevitability so this post will likely not make a difference.


Raymond


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[David Foster]

I have seen a ton of discussion about what dict addition should do, but 
have seen almost no mention of dict difference.


This lack of discussion interest combined with me not recalling having 
needed the proposed subtraction semantics personally makes me wonder if 
we should hold off on locking in subtraction semantics just yet. Perhaps 
we could just scope the proposal to dictionary addition only for now?


If I *were* to define dict difference, my intuition suggests supporting 
a second operand that is any iterable of keys and not just dicts. 
(Augmented dict subtraction is already proposed to accept such a broader 
second argument.)


David Foster | Seattle, WA, USA

On 3/1/19 8:26 AM, Steven D'Aprano wrote:

Attached is a draft PEP on adding + and - operators to dict for
discussion.

This should probably go here:

https://github.com/python/peps

but due to technical difficulties at my end, I'm very limited in what I
can do on Github (at least for now). If there's anyone who would like to
co-author and/or help with the process, that will be appreciated.




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Brett Cannon]

On Mon, Mar 4, 2019 at 1:29 PM Neil Girdhar  wrote:

> On Mon, Mar 4, 2019 at 3:58 PM Christopher Barker 
> wrote:
> >
> >
> >
> > On Mon, Mar 4, 2019 at 12:41 PM Guido van Rossum 
> wrote:
> >>
> >> Honestly I would rather withdraw the subtraction operators than reopen
> the discussion about making dict more like set.
>
> I think that's unfortunate.
> >
> >
> > +1
> >
> > I think the "dicts are like more-featured" sets is a math-geek
> perspective, and unlikely to make things more clear for the bulk of users.
> And may make it less clear.
>
> I'd say reddit has some pretty "common users", and they're having a
> discussion of this right now
> (
> https://www.reddit.com/r/Python/comments/ax4zzb/pep_584_add_and_operators_to_the_builtin_dict/
> ).
> The most popular comment is how it should be |.
>
> Anyway, I think that following the mathematical metaphors tends to
> make things more intuitive in the long run.


Only if you know the mathematical metaphors. ;)


>   Python is an adventure.
> You learn it for years and then it all makes sense.  If dict uses +,
> yes, new users might find that sooner than |.  However, when they
> learn set union, I think they will wonder why it's not consistent with
> dict union.
>

Not to me. I barely remember that | is supported for sets, but I sure know
about + and lists (and strings, etc.) and I'm willing to bet the vast
majority of folks are the some; addition is much more widely known than set
theory.


>
> The PEP's main justification for + is that it matches Counter, but
> counter is adding the values whereas | doesn't touch the values.   I
> think it would be good to at least make a list of pros and cons of
> each proposed syntax.
>

I suspect Steven will add more details to a Rejected Ideas section.


>
> > We need to be careful -- there are a lot more math geeks on this list
> than in the general Python coding population.
> >
> > Simply adding "+" is a non-critical nice to have, but it seems unlikely
> to really confuse anyone.
>

I agree with Chris.

-Brett


> >
> > -CHB
> >
> >
> > --
> > Christopher Barker, PhD
> >
> > Python Language Consulting
> >   - Teaching
> >   - Scientific Software Development
> >   - Desktop GUI and Web Development
> >   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[James Lu]

By the way, my “no same keys with different values” proposal would not apply to 
+=.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[James Lu]

> On Mar 4, 2019, at 11:25 AM, Steven D'Aprano  wrote:
> 
> The PEP gives a good example of when this "invariant" would be 
> unnecessarily restrictive:
> 
>For example, updating default configuration values with 
>user-supplied values would most often fail under the 
>requirement that keys are unique::
> 
>prefs = site_defaults + user_defaults + document_prefs
> 
> 
> Another example would be when reading command line options, where the 
> most common convention is for "last option seen" to win:
> 
> [steve@ando Lib]$ grep --color=always --color=never "zero" f*.py
> fileinput.py: numbers are zero; nextfile() has no effect.
> fractions.py: # the same way for any finite a, so treat a as zero.
> functools.py: # prevent their ref counts from going to zero during
> 
Indeed, in this case you would want to use {**, **} syntax. 


> and the output is printed without colour.
> 
> (I've slightly edited the above output so it will fit in the email 
> without wrapping.)
> 
> The very name "update" should tell us that the most useful behaviour is 
> the one the devs decided on back in 1.5: have the last seen value win. 
> How can you update values if the operation raises an error if the key 
> already exists? If this behaviour is ever useful, I would expect that it 
> will be very rare.

> An update or merge is effectively just running through a loop setting 
> the value of a key. See the pre-Python 1.5 function above. Having update 
> raise an exception if the key already exists would be about as useful as 
> having ``d[key] = value`` raise an exception if the key already exists.
> 
> Unless someone can demonstrate that the design of dict.update() was a 
> mistake
You’re making a logical mistake here. + isn’t supposed to have .update’s 
behavior and it never was supposed to.

> , and the "require unique keys" behaviour is more common,
I just have. 99% of the time you want to have keys from one dict override 
another, you’d be better off doing it in-place and so would be using .update() 
anyways.

> then 
> I maintain that for the very rare cases you want an exception, you can 
> subclass dict and overload the __add__ method:
Well, yes, the whole point is to define the best default behavior.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[James Lu]

> On Mon, Mar 04, 2019 at 10:01:23AM -0500, James Lu wrote:
> 
> If you want to merge it without a KeyError, learn and use the more explicit 
> {**d1, **d2} syntax.

On Mar 4, 2019, at 10:25 AM, Steven D'Aprano  wrote:

> In your previous email, you said the {**d ...} syntax was implicit:
> 
>In other words, explicit + is better than implicit {**, **#, unless 
>explicitly suppressed.  Here + is explicit whereas {**, **} is 
>implicitly allowing inclusive keys, and the KeyError is expressed 
>suppressed by virtue of not using the {**, **} syntax.
> 
> It is difficult to take your "explicit/implicit" argument seriously when 
> you cannot even decided which is which.
I misspoke. 
> In your previous email, you said the {**d ...} syntax was implicit:
> 
>In other words, explicit + is better than implicit {**, **#, unless 
>explicitly suppressed.  Here + is explicit whereas {**, **} is 
>implicitly allowing inclusive keys, and the KeyError is expressed 
>suppressed by virtue of not using the {**, **} syntax.
> 
> It is difficult to take your "explicit/implicit" argument seriously when 
> you cannot even decided which is which.

Yes, + is explicit. {**, **} is implicit. 

My argument:

We should set the standard that + is for non-conflicting merge and {**, **} is 
for overriding merge. That standard should be so that + explicitly asserts that 
the keys will not conflict whereas {**d1, **d2} is ambiguous on why d2 is 
overriding d1.^

^Presumably you’re making a copy of d1 so why should d3 have d2 take priority? 
The syntax deserves a comment, perhaps explaining that items from d2 are newer 
in time or that the items in d1 are always nonces. 

The + acts as an implicit assertion and an opportunity to catch an invariant 
violation or data input error.

Give me an example of a situation where you need a third dictionary from two 
existing dictionaries and having conflict where a key has a different value in 
both is desirable behavior. 

The situation where non-conflicting merge is what’s desired is more common and 
in that case throwing an exception in the case of a conflicting value is a good 
thing, a way to catch code smell.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Guido van Rossum]

On Mon, Mar 4, 2019 at 3:31 PM Del Gan  wrote:

> > the augmented assignment version allows anything the ``update`` method
> allows, such as iterables of key/value pairs
>
> I am a little surprised by this choice.
>
> First, this means that "a += b" would not be equivalent to "a = a +
> b". Is there other built-in types which act differently if called with
> the operator or augmented assignment version?
>

Yes. The same happens for lists. [1] + 'a' is a TypeError, but a += 'a'
works:

>>> a = [1]
>>> a + 'a'
Traceback (most recent call last):
  File "", line 1, in 
TypeError: can only concatenate list (not "str") to list
>>> a += 'a'
>>> a
[1, 'a']
>>>


> Secondly, that would imply I would no longer be able to infer the type
> of "a" while reading "a += [('foo', 'bar')]". Is it a list? A dict?
>

Real code more likely looks like "a += b" and there you already don't have
much of a clue -- the author of the code should probably communicate this
using naming conventions or type annotations.


> Those two points make me uncomfortable with "+=" strictly behaving
> like ".update()".
>

And yet that's how it works for lists. (Note that dict.update() still has
capabilities beyond +=, since you can also invoke it with keyword args.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Dan Sommers]

On 3/4/19 5:11 PM, Steven D'Aprano wrote:

On Mon, Mar 04, 2019 at 11:56:54AM -0600, Dan Sommers wrote:

On 3/4/19 10:44 AM, Steven D'Aprano wrote:

> If you know ahead of time which order you want, you can simply reverse
> it:
>
>  # prefs = site_defaults + user_defaults + document_prefs
>  prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))
>
> but that seems a little awkward to me, and reads backwards. I'm used to
> thinking reading left-to-right, not right-to-left.

I read that as use document preferences first, then user
defaults, then site defautls, exactly as I'd explain the
functionality to someone else.


If you explained it to me like that, with the term "use", I'd think that
the same feature would be done three times: once with document prefs,
then with user defaults, then site defaults.

Clearly that's not what you mean, so I'd then have to guess what you
meant by "use", since you don't actually mean use. That would leave me
trying to guess whether you meant that *site defaults* overrode document
prefs or the other way.

I don't like guessing, so I'd probably explicitly ask: "Wait, I'm
confused, which wins? It sounds like site defaults wins, surely that's
not what you meant."


You're right:  "use" is the wrong word.  Perhaps "prefer"
is more appropriate.  To answer the question of which wins:
the first one in the list [document, user, site] that
contains a given preference in question.  Users don't see
dictionary updates; they see collections of preferences in
order of priority.

Documentation is hard.  :-)

Sorry.


So maybe we're agreeing:  if you think in terms of updating
a dictionary of preferences, then maybe it reads backwards,
but if you think of implementing features, then adding
dictionaries of preferences reads backwards.


Do you think "last seen wins" is backwards for dict.update() or for
command line options?


As a user, "last seen wins" is clearly superior for command
line options.  As a programmer, because object methods
operate on their underlying object, it's pretty obvious that
d1.update(d2) starts with d1 and applies the changes
expressed in d2, which is effectively "last seen wins."

If I resist the temptation to guess in the face of
ambiguity, though, I don't think that d1 + d2 is any less
ambiguous than a hypothetical dict_update(d1, d2) function.
When I see a + operator, I certainly don't think of one
operand or the other winning.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Del Gan]

Hi.

> the augmented assignment version allows anything the ``update`` method 
> allows, such as iterables of key/value pairs

I am a little surprised by this choice.

First, this means that "a += b" would not be equivalent to "a = a +
b". Is there other built-in types which act differently if called with
the operator or augmented assignment version?

Secondly, that would imply I would no longer be able to infer the type
of "a" while reading "a += [('foo', 'bar')]". Is it a list? A dict?

Those two points make me uncomfortable with "+=" strictly behaving
like ".update()".

2019-03-04 17:44 UTC+01:00, Rhodri James :
> On 04/03/2019 15:12, James Lu wrote:
>>
>>> On Mar 4, 2019, at 10:02 AM, Stefan Behnel  wrote:
>>>
>>> INADA Naoki schrieb am 04.03.19 um 11:15:
 Why statement is not enough?
>>>
>>> I'm not sure I understand why you're asking this, but a statement is
>>> "not
>>> enough" because it's a statement and not an expression. It does not
>>> replace
>>> the convenience of an expression.
>>>
>>> Stefan
>> There is already an expression for key-overriding merge. Why do we need a
>> new one?
>
> Because the existing one is inobvious, hard to discover and ugly.
>
> --
> Rhodri James *-* Kynesim Ltd
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Steven D'Aprano]

On Mon, Mar 04, 2019 at 11:56:54AM -0600, Dan Sommers wrote:
> On 3/4/19 10:44 AM, Steven D'Aprano wrote:
> 
> > If you know ahead of time which order you want, you can simply reverse
> > it:
> >
> >  # prefs = site_defaults + user_defaults + document_prefs
> >  prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))
> >
> > but that seems a little awkward to me, and reads backwards. I'm used to
> > thinking reading left-to-right, not right-to-left.
> 
> I read that as use document preferences first, then user
> defaults, then site defautls, exactly as I'd explain the
> functionality to someone else.

If you explained it to me like that, with the term "use", I'd think that 
the same feature would be done three times: once with document prefs, 
then with user defaults, then site defaults.

Clearly that's not what you mean, so I'd then have to guess what you 
meant by "use", since you don't actually mean use. That would leave me 
trying to guess whether you meant that *site defaults* overrode document 
prefs or the other way.

I don't like guessing, so I'd probably explicitly ask: "Wait, I'm 
confused, which wins? It sounds like site defaults wins, surely that's 
not what you meant."


> So maybe we're agreeing:  if you think in terms of updating
> a dictionary of preferences, then maybe it reads backwards,
> but if you think of implementing features, then adding
> dictionaries of preferences reads backwards.

Do you think "last seen wins" is backwards for dict.update() or for 
command line options?


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Steven D'Aprano]

On Sat, Mar 02, 2019 at 11:14:18AM -0800, Raymond Hettinger wrote:

> If the existing code were in the form of "d=e.copy(); d.update(f); 
> d.update(g); d.update(h)", converting it to "d = e + f + g + h" would 
> be a tempting but algorithmically poor thing to do (because the 
> behavior is quadratic).

I mention this in the PEP. Unlike strings, but like lists and tuples, I 
don't expect that this will be a problem in practice:

- it's easy to put repeated string concatenation in a tight loop;
  it is harder to think of circumstances where one needs to
  concatenate lists or tuples, or merge dicts, in a tight loop;

- it's easy to have situations where one is concatenating thousands
  of strings; its harder to imagine circumstances where one would be
  merging more than three or four dicts;

- concatentation s1 + s2 + ... for strings, lists or tuples results
  in a new object of length equal to the sum of the lengths of each
  of the inputs, so the output is constantly growing; but merging 
  dicts d1 + d2 + ... typically results in a smaller object of 
  length equal to the number of unique keys.


> Most likely, the right thing to do would be 
> "d = ChainMap(e, f, g, h)" for a zero-copy solution or "d = 
> dict(ChainMap(e, f, g, h))" to flatten the result without incurring 
> quadratic costs.  Both of those are short and clear.

And both result in the opposite behaviour of what you probably intended 
if you were trying to match e + f + g + h. Dict merging/updating 
operates on "last seen wins", but ChainMap is "first seen wins". To get 
the same behaviour, we have to write the dicts in opposite order 
compared to update, from most to least specific:

# least specific to most specific
prefs = site_defaults + user_defaults + document_prefs

# most specific to least
prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))

To me, the later feels backwards: I'm applying document prefs first, and 
then trusting that the ChainMap doesn't overwrite them with the 
defaults. I know that's guaranteed behaviour, but every time I read it 
I'll feel the need to check :-)


> Lastly, I'm still bugged by use of the + operator for replace-logic 
> instead of additive-logic.  With numbers and lists and Counters, the 
> plus operator creates a new object where all the contents of each 
> operand contribute to the result.  With dicts, some of the contents 
> for the left operand get thrown-away.  This doesn't seem like addition 
> to me (IIRC that is also why sets have "|" instead of "+").

I'm on the fence here. Addition seems to be the most popular operator 
(it often gets requested) but you might be right that this is more like 
a union operation than concatenation or addition operation. MRAB also 
suggested this earlier.

One point in its favour is that + goes nicely with - but on the other 
hand, sets have | and - with no + and that isn't a problem.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Paul Moore]

On Mon, 4 Mar 2019 at 20:42, Guido van Rossum  wrote:
>
> Honestly I would rather withdraw the subtraction operators than reopen the 
> discussion about making dict more like set.

I'm neutral on dict addition, but dict subtraction seemed an odd
extension to the proposal. Using b in a - b solely for its keys, and
ignoring its values, seems weird to me. Even if dict1 - dict2 were
added to the language, I think I'd steer clear of it as being too
obscure.

I'm not going to get sucked into this debate, but I'd be happy to see
the subtraction operator part of the proposal withdrawn.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Neil Girdhar]

On Mon, Mar 4, 2019 at 3:58 PM Christopher Barker  wrote:
>
>
>
> On Mon, Mar 4, 2019 at 12:41 PM Guido van Rossum  wrote:
>>
>> Honestly I would rather withdraw the subtraction operators than reopen the 
>> discussion about making dict more like set.

I think that's unfortunate.
>
>
> +1
>
> I think the "dicts are like more-featured" sets is a math-geek perspective, 
> and unlikely to make things more clear for the bulk of users. And may make it 
> less clear.

I'd say reddit has some pretty "common users", and they're having a
discussion of this right now
(https://www.reddit.com/r/Python/comments/ax4zzb/pep_584_add_and_operators_to_the_builtin_dict/).
The most popular comment is how it should be |.

Anyway, I think that following the mathematical metaphors tends to
make things more intuitive in the long run.  Python is an adventure.
You learn it for years and then it all makes sense.  If dict uses +,
yes, new users might find that sooner than |.  However, when they
learn set union, I think they will wonder why it's not consistent with
dict union.

The PEP's main justification for + is that it matches Counter, but
counter is adding the values whereas | doesn't touch the values.   I
think it would be good to at least make a list of pros and cons of
each proposed syntax.

> We need to be careful -- there are a lot more math geeks on this list than in 
> the general Python coding population.
>
> Simply adding "+" is a non-critical nice to have, but it seems unlikely to 
> really confuse anyone.
>
> -CHB
>
>
> --
> Christopher Barker, PhD
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a "week" function or attribute to datetime.date

[Christopher Barker]

There are all sorts of "Calendar" operations one might want -- I think
those belong in a separate library, rather than a few tacked on to datetime.

-CHB


On Fri, Mar 1, 2019 at 2:48 AM Robert Vanden Eynde 
wrote:

> Currently one can do week = d.isocalendar()[1]
>
> The iso definition of a week number has some nice properties.
>
> robertvandeneynde.be
>
> On Fri, 1 Mar 2019, 11:44 Antonio Galán,  wrote:
>
>> The week number is usually refered to the week of the year, but the week
>> of the month is also interesting, for example for some holiday which depend
>> on the week number of the month, so in analogy with "weekday" we can use
>> "yearweek" and "monthweek"
>> El vie., 1 de marzo de 2019 9:33, Adrien Ricocotam 
>> escribió:
>>
>>> I like the idea. But how to distinguish it from the number of week past
>>> since the beginning of the month ?
>>>
>>> But that’s great.
>>>
>>> On Fri 1 Mar 2019 at 09:31, Antonio Galán  wrote:
>>>
 Hi, datetime.date.today() (or other day)  has attributes .year and
 .month wich return the year and the month of that date, also it has a
 function weekday() wich return the number of the day in the week.

 I think it is a good idea add a function or attribute "week" wich
 return the number of the week on the year. It is useful to execute scripts
 once a week for example.
 ___
 Python-ideas mailing list
 Python-ideas@python.org
 https://mail.python.org/mailman/listinfo/python-ideas
 Code of Conduct: http://python.org/psf/codeofconduct/

>>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Christopher Barker]

On Mon, Mar 4, 2019 at 12:41 PM Guido van Rossum  wrote:

> Honestly I would rather withdraw the subtraction operators than reopen the
> discussion about making dict more like set.
>

+1

I think the "dicts are like more-featured" sets is a math-geek perspective,
and unlikely to make things more clear for the bulk of users. And may make
it less clear.

We need to be careful -- there are a lot more math geeks on this list than
in the general Python coding population.

Simply adding "+" is a non-critical nice to have, but it seems unlikely to
really confuse anyone.

-CHB


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Guido van Rossum]

Honestly I would rather withdraw the subtraction operators than reopen the
discussion about making dict more like set.

On Mon, Mar 4, 2019 at 12:33 PM Neil Girdhar  wrote:

> On Mon, Mar 4, 2019 at 3:22 PM Guido van Rossum  wrote:
> >
> > On Mon, Mar 4, 2019 at 12:12 PM Neil Girdhar 
> wrote:
> >>
> >> On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum 
> wrote:
> >> >
> >> > * Dicts are not like sets because the ordering operators (<, <=, >,
> >=) are not defined on dicts, but they implement subset comparisons for
> sets. I think this is another argument pleading against | as the operator
> to combine two dicts.
> >> >
> >>
> >> I feel like dict should be treated like sets with the |, &, and -
> >> operators since in mathematics a mapping is sometimes represented as a
> >> set of pairs with unique first elements.  Therefore, I think the set
> >> metaphor is stronger.
> >
> >
> > That ship has long sailed.
>
> Maybe, but reading through the various replies, it seems that if you
> are adding "-" to be analogous to set difference, then the combination
> operator should be analogous to set union "|".  And it also opens an
> opportunity to add set intersection "&".  After all, how do you filter
> a dictionary to a set of keys?
>
> >> d = {'some': 5, 'extra': 10, 'things': 55}
> >> d &= {'some', 'allowed', 'options'}
> >> d
> {'some': 5}
>
> >>
> >> > * Regarding how to construct the new set in __add__, I now think this
> should be done like this:
> >> >
> >> > class dict:
> >> > 
> >> > def __add__(self, other):
> >> > 
> >> > new = self.copy()  # A subclass may or may not choose to
> override
> >> > new.update(other)
> >> > return new
> >>
> >> I like that, but it would be inefficient to do that for __sub__ since
> >> it would create elements that it might later delete.
> >>
> >> def __sub__(self, other):
> >>  new = self.copy()
> >>  for k in other:
> >>   del new[k]
> >> return new
> >>
> >> is less efficient than
> >>
> >> def __sub__(self, other):
> >>  return type(self)({k: v for k, v in self.items() if k not in other})
> >>
> >> when copying v is expensive.  Also, users would probably not expect
> >> values that don't end up being returned to be copied.
> >
> >
> > No, the values won't be copied -- it is a shallow copy that only increfs
> the keys and values.
>
> Oh right, good point.  Then your way is better since it would preserve
> any other data stored by the dict subclass.
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Neil Girdhar]

On Mon, Mar 4, 2019 at 3:22 PM Guido van Rossum  wrote:
>
> On Mon, Mar 4, 2019 at 12:12 PM Neil Girdhar  wrote:
>>
>> On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum  wrote:
>> >
>> > * Dicts are not like sets because the ordering operators (<, <=, >, >=) 
>> > are not defined on dicts, but they implement subset comparisons for sets. 
>> > I think this is another argument pleading against | as the operator to 
>> > combine two dicts.
>> >
>>
>> I feel like dict should be treated like sets with the |, &, and -
>> operators since in mathematics a mapping is sometimes represented as a
>> set of pairs with unique first elements.  Therefore, I think the set
>> metaphor is stronger.
>
>
> That ship has long sailed.

Maybe, but reading through the various replies, it seems that if you
are adding "-" to be analogous to set difference, then the combination
operator should be analogous to set union "|".  And it also opens an
opportunity to add set intersection "&".  After all, how do you filter
a dictionary to a set of keys?

>> d = {'some': 5, 'extra': 10, 'things': 55}
>> d &= {'some', 'allowed', 'options'}
>> d
{'some': 5}

>>
>> > * Regarding how to construct the new set in __add__, I now think this 
>> > should be done like this:
>> >
>> > class dict:
>> > 
>> > def __add__(self, other):
>> > 
>> > new = self.copy()  # A subclass may or may not choose to override
>> > new.update(other)
>> > return new
>>
>> I like that, but it would be inefficient to do that for __sub__ since
>> it would create elements that it might later delete.
>>
>> def __sub__(self, other):
>>  new = self.copy()
>>  for k in other:
>>   del new[k]
>> return new
>>
>> is less efficient than
>>
>> def __sub__(self, other):
>>  return type(self)({k: v for k, v in self.items() if k not in other})
>>
>> when copying v is expensive.  Also, users would probably not expect
>> values that don't end up being returned to be copied.
>
>
> No, the values won't be copied -- it is a shallow copy that only increfs the 
> keys and values.

Oh right, good point.  Then your way is better since it would preserve
any other data stored by the dict subclass.
>
> --
> --Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Guido van Rossum]

On Mon, Mar 4, 2019 at 12:12 PM Neil Girdhar  wrote:

> On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum  wrote:
> >
> > * Dicts are not like sets because the ordering operators (<, <=, >, >=)
> are not defined on dicts, but they implement subset comparisons for sets. I
> think this is another argument pleading against | as the operator to
> combine two dicts.
> >
>
> I feel like dict should be treated like sets with the |, &, and -
> operators since in mathematics a mapping is sometimes represented as a
> set of pairs with unique first elements.  Therefore, I think the set
> metaphor is stronger.
>

That ship has long sailed.


> > * Regarding how to construct the new set in __add__, I now think this
> should be done like this:
> >
> > class dict:
> > 
> > def __add__(self, other):
> > 
> > new = self.copy()  # A subclass may or may not choose to override
> > new.update(other)
> > return new
>
> I like that, but it would be inefficient to do that for __sub__ since
> it would create elements that it might later delete.
>
> def __sub__(self, other):
>  new = self.copy()
>  for k in other:
>   del new[k]
> return new
>
> is less efficient than
>
> def __sub__(self, other):
>  return type(self)({k: v for k, v in self.items() if k not in other})
>
> when copying v is expensive.  Also, users would probably not expect
> values that don't end up being returned to be copied.
>

No, the values won't be copied -- it is a shallow copy that only increfs
the keys and values.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Neil Girdhar]

On Mon, Mar 4, 2019 at 2:26 PM Guido van Rossum  wrote:
>
> * Dicts are not like sets because the ordering operators (<, <=, >, >=) are 
> not defined on dicts, but they implement subset comparisons for sets. I think 
> this is another argument pleading against | as the operator to combine two 
> dicts.
>

I feel like dict should be treated like sets with the |, &, and -
operators since in mathematics a mapping is sometimes represented as a
set of pairs with unique first elements.  Therefore, I think the set
metaphor is stronger.

> * Regarding how to construct the new set in __add__, I now think this should 
> be done like this:
>
> class dict:
> 
> def __add__(self, other):
> 
> new = self.copy()  # A subclass may or may not choose to override
> new.update(other)
> return new

I like that, but it would be inefficient to do that for __sub__ since
it would create elements that it might later delete.

def __sub__(self, other):
 new = self.copy()
 for k in other:
  del new[k]
return new

is less efficient than

def __sub__(self, other):
 return type(self)({k: v for k, v in self.items() if k not in other})

when copying v is expensive.  Also, users would probably not expect
values that don't end up being returned to be copied.

>
> AFAICT this will give the expected result for defaultdict -- it keeps the 
> default factory from the left operand (i.e., self).
>
> * Regarding how often this is needed, we know that this is proposed and 
> discussed at length every few years, so I think this will fill a real need.
>
> * Regarding possible anti-patterns that this might encourage, I'm not aware 
> of problems around list + list, so this seems an unwarranted worry to me.
>

I agree with these points.

Best,

Neil
> --
> --Guido van Rossum (python.org/~guido)
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the Google 
> Groups "python-ideas" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/python-ideas/zfHYRHMIAdM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the Google 
> Groups "python-ideas" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/python-ideas/zfHYRHMIAdM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Guido van Rossum]

* Dicts are not like sets because the ordering operators (<, <=, >, >=) are
not defined on dicts, but they implement subset comparisons for sets. I
think this is another argument pleading against | as the operator to
combine two dicts.

* Regarding how to construct the new set in __add__, I now think this
should be done like this:

class dict:

def __add__(self, other):

new = self.copy()  # A subclass may or may not choose to override
new.update(other)
return new

AFAICT this will give the expected result for defaultdict -- it keeps the
default factory from the left operand (i.e., self).

* Regarding how often this is needed, we know that this is proposed and
discussed at length every few years, so I think this will fill a real need.

* Regarding possible anti-patterns that this might encourage, I'm not aware
of problems around list + list, so this seems an unwarranted worry to me.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Current use of addition in Python

[Jonathan Fine]

First, I thank Rhodri for his question, and Eric for his reply (see
earlier messages in this thread).

SUMMARY
I record the addition properties of collections.Counter and
numpy.array. Finally, some comments about len(), and a promise of more
tomorrow.

COUNTER
Now for collections.Counter -- this is "provided to support convenient
and rapid tallies."
https://docs.python.org/3/library/collections.html#collections.Counter

In my previous post, I noted that the built-in numeric types have the
properties:

Commutative
Associative
Left and right cancellation
Existence of zero
Multiplication by a non-negative integer

Instances of Counter 'usually' have all of these properties, except
for multiplication by a non-negative integer. Here's an example where
cancellation fails
>>> Counter(a=-1) + Counter()
Counter()

Here's two more examples:
   >>> Counter(a=+1, b=-1) + Counter(a=-1, b=+1)
   Counter()
   >>> Counter(a=+1, b=-2) + Counter(a=-2, b=+1)
   Counter()
In the first example, it seems that the counters cancel. But the
second example shows that something else is going on.

Here's an example of associativity failing:
>>> (Counter(a=+1) + Counter(a=-2)) + Counter(a=2)
Counter({'a': 2})
>>> Counter(a=+1) + (Counter(a=-2) + Counter(a=2))
Counter({'a': 1})

The Python docs (URL above) notes that the Counter "methods are
designed only for use cases with positive values."

NUMPY.ARRAY

These arrays have all the properties listed above (commutative,
associative, left and right cancellation, multiplication by
non-negative integer), provided all the arrays have the same shape.
(The shape of an array is a tuple of non-negative integers.) And for
numpy.array, the zero must also have the same shape.

Briefly, a numpy array acts like a multi-dimensional vector. Here's an example:
>>> array(range(3)), array(range(3, 6))
(array([0, 1, 2]), array([3, 4, 5]))
>>> array(range(3)) + array(range(3, 6))
array([3, 5, 7])

Here's another example:
>>> array(range(3, 6)) * 4
array([12, 16, 20])
>>> 4 * array(range(3, 6))
array([12, 16, 20])

LENGTH -- len()

The numeric types don't have a length:
>>> len(0)
TypeError: object of type 'int' has no len()

The sequence and mapping types (such as list, tuple, str, bytes, set,
dict) are iterable, and have a length. Also, numpy.array and
collections.Counter have a length.

More on length tomorrow.

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[David Mertz]

On Mon, Mar 4, 2019, 11:45 AM Steven D'Aprano  wrote:

> > Like other folks in the thread, I also want to merge dicts three times
> per
> > year.
>
> I'm impressed that you have counted it with that level of accuracy. Is it
> on the same three days each year, or do they move about? *wink*
>

To be respectful, I always merge dicts on Eid al-Fitr, Diwali, and Lent. I
was speaking approximate since those do not appears line up with the same
Gregorian year.

> And every one of those times, itertools.ChainMap is the right way to do
> that non-destructively, and without copying.
>
> Can you elaborate on why ChainMap is the right way to merge multiple dicts
> into a single, new dict?
>

Zero-copy.


> ChainMap also seems to implement the opposite behaviour to that usually
> desired: first value seen wins, instead of last:
>

True, the semantics are different, but equivalent, to the proposed dict
addition. I put the key I want to "win" first rather than last.

If you know ahead of time which order you want, you can simply reverse it:
>

This seems nonsensical. If I write, at some future time,
'dict1+dict2+dict3' I need exactly as much to know "ahead of time" which
keys I intend to win.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Dan Sommers]

On 3/4/19 10:44 AM, Steven D'Aprano wrote:

> If you know ahead of time which order you want, you can simply reverse
> it:
>
>  # prefs = site_defaults + user_defaults + document_prefs
>  prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))
>
> but that seems a little awkward to me, and reads backwards. I'm used to
> thinking reading left-to-right, not right-to-left.

I read that as use document preferences first, then user
defaults, then site defautls, exactly as I'd explain the
functionality to someone else.

So maybe we're agreeing:  if you think in terms of updating
a dictionary of preferences, then maybe it reads backwards,
but if you think of implementing features, then adding
dictionaries of preferences reads backwards.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Current use of addition in Python

[Eric V. Smith]

> On Mar 4, 2019, at 2:18 PM, Rhodri James  wrote:
> 
>> On 04/03/2019 14:03, Jonathan Fine wrote:
>> Summary: This thread is for recording current use of addition in
>> Python.
> 
> TL;DR.  Why is this is Python Ideas?

Because of the current discussion of dict + dict. I think this is helping 
answer the question: is there anything currently in python that’s a similar 
usage of “+”?

Personally, I don’t think it matters much, but it’s interesting. If the 
requirement were for new usages of “+” to be like old ones, would we have added 
str + str?

Eric


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Steven D'Aprano]

On Mon, Mar 04, 2019 at 03:43:48PM +0200, Serhiy Storchaka wrote:
> 01.03.19 12:44, Steven D'Aprano пише:
> >On Fri, Mar 01, 2019 at 08:47:36AM +0200, Serhiy Storchaka wrote:
> >
> >>Currently Counter += dict works and Counter + dict is an error. With
> >>this change Counter + dict will return a value, but it will be different
> >>from the result of the += operator.
> >
> >That's how list.__iadd__ works too: ListSubclass + list will return a
> >value, but it might not be the same as += since that operates in place
> >and uses a different dunder method.
> >
> >Why is it a problem for dicts but not a problem for lists?
> 
> Because the plus operator for lists predated any list subclasses.

That doesn't answer my question. Just because it is older is no 
explaination for why this behaviour is not a problem for lists, or a 
problem for dicts.

[...]
> >What's wrong with doing this?
> >
> > new = type(self)()
> >
> >Or the equivalent from C code. If that doesn't work, surely that's the
> >fault of the subclass, the subclass is broken, and it will raise an
> >exception.
> 
> Try to do this with defaultdict.

I did. It seems to work fine with my testing:

py> defaultdict()
defaultdict(None, {})

is precisely the behaviour I would expect.

If it isn't the right thing to do, then defaultdict can override __add__ 
and __radd__.


> Note that none of builtin sequences or sets do this. For good reasons 
> they always return an instance of the base type.

What are those good reasons?


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Rhodri James]

On 04/03/2019 15:12, James Lu wrote:



On Mar 4, 2019, at 10:02 AM, Stefan Behnel  wrote:

INADA Naoki schrieb am 04.03.19 um 11:15:

Why statement is not enough?


I'm not sure I understand why you're asking this, but a statement is "not
enough" because it's a statement and not an expression. It does not replace
the convenience of an expression.

Stefan

There is already an expression for key-overriding merge. Why do we need a new 
one?


Because the existing one is inobvious, hard to discover and ugly.

--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Steven D'Aprano]

On Mon, Mar 04, 2019 at 09:42:53AM -0500, David Mertz wrote:
> On Mon, Mar 4, 2019, 8:30 AM Serhiy Storchaka  wrote:
> 
> > But is merging two dicts a common enough problem that needs introducing
> > an operator to solve it? I need to merge dicts maybe not more than one
> > or two times by year, and I am fine with using the update() method.
> > Perhaps {**d1, **d2} can be more appropriate in some cases, but I did not
> > encounter such cases yet.
> >
> 
> Like other folks in the thread, I also want to merge dicts three times per
> year. 

I'm impressed that you have counted it with that level of accuracy. Is 
it on the same three days each year, or do they move about? *wink*


> And every one of those times, itertools.ChainMap is the right way to
> do that non-destructively, and without copying.

Can you elaborate on why ChainMap is the right way to merge multiple 
dicts into a single, new dict?

ChainMap also seems to implement the opposite behaviour to that usually 
desired: first value seen wins, instead of last:

py> from collections import ChainMap
py> cm = ChainMap({'a': 1}, {'b': 2}, {'a': 999})
py> cm
ChainMap({'a': 1}, {'b': 2}, {'a': 999})
py> dict(cm)
{'a': 1, 'b': 2}


If you know ahead of time which order you want, you can simply reverse 
it:

# prefs = site_defaults + user_defaults + document_prefs
prefs = dict(ChainMap(document_prefs, user_defaults, site_defaults))

but that seems a little awkward to me, and reads backwards. I'm used to 
thinking reading left-to-right, not right-to-left.

ChainMap seems, to me, to be ideal for implementing "first wins" 
mappings, such as emulating nested scopes, but not so ideal for 
update/merge operations.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Steven D'Aprano]

On Mon, Mar 04, 2019 at 10:09:32AM -0500, James Lu wrote:

> How many situations would you need to make a copy of a dictionary and 
> then update that copy and override old keys from a new dictionary?

Very frequently.

That's why we have a dict.update method, which if I remember correctly, 
was introduced in Python 1.5 because people were frequently re-inventing 
the same wheel:

def update(d1, d2):
for key in d2.keys():
d1[key] in d2[key]


You should have a look at how many times it is used in the standard 
library:

[steve@ando cpython]$ cd Lib/
[steve@ando Lib]$ grep -U "\.update[(]" *.py */*.py | wc -l
373

Now some of those are false positives (docstrings, comments, non-dicts, 
etc) but that still leaves a lot of examples of wanting to override old 
keys. This is a very common need. Wanting an exception if the key 
already exists is, as far as I can tell, very rare.

It is true that many of the examples in the std lib involve updating an 
existing dict, not creating a new one. But that's only to be expected: 
since Python didn't provide an obvious functional version of update, 
only an in-place version, naturally people get used to writing 
in-place code.

(Think about how long we made do without sorted(). I don't know about 
other people, but I now find sorted indispensible, and probably use it 
ten or twenty times more often than the in-place version.)


[...]
> The KeyError of my proposal is a feature, a sign that something is 
> wrong, a sign an invariant is being violated.

Why is "keys are unique" an invariant?

The PEP gives a good example of when this "invariant" would be 
unnecessarily restrictive:

For example, updating default configuration values with 
user-supplied values would most often fail under the 
requirement that keys are unique::

prefs = site_defaults + user_defaults + document_prefs


Another example would be when reading command line options, where the 
most common convention is for "last option seen" to win:

[steve@ando Lib]$ grep --color=always --color=never "zero" f*.py
fileinput.py: numbers are zero; nextfile() has no effect.
fractions.py: # the same way for any finite a, so treat a as zero.
functools.py: # prevent their ref counts from going to zero during

and the output is printed without colour.

(I've slightly edited the above output so it will fit in the email 
without wrapping.)

The very name "update" should tell us that the most useful behaviour is 
the one the devs decided on back in 1.5: have the last seen value win. 
How can you update values if the operation raises an error if the key 
already exists? If this behaviour is ever useful, I would expect that it 
will be very rare.

An update or merge is effectively just running through a loop setting 
the value of a key. See the pre-Python 1.5 function above. Having update 
raise an exception if the key already exists would be about as useful as 
having ``d[key] = value`` raise an exception if the key already exists.

Unless someone can demonstrate that the design of dict.update() was a 
mistake, and the "require unique keys" behaviour is more common, then 
I maintain that for the very rare cases you want an exception, you can 
subclass dict and overload the __add__ method:

# Intentionally simplified version.
def __add__(self, other):
if self.keys() & other.keys():
raise KeyError
return super().__add__(self, other)


> The ugliness of the syntax makes one pause 
> and think and ask: “Why is it important that the keys from this 
> dictionary override the ones from another dictionary?”

Because that is the most common and useful behaviour. That's what it 
means to *update* a dict or database, and this proposal is for an update 
operator.

The ugliness of the existing syntax is not a feature, it is a barrier.


-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Current use of addition in Python

[Rhodri James]

On 04/03/2019 14:03, Jonathan Fine wrote:

Summary: This thread is for recording current use of addition in
Python.


TL;DR.  Why is this is Python Ideas?

--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Steven D'Aprano]

On Mon, Mar 04, 2019 at 10:01:23AM -0500, James Lu wrote:

> If you want to merge it without a KeyError, learn and use the more explicit 
> {**d1, **d2} syntax.

In your previous email, you said the {**d ...} syntax was implicit:

In other words, explicit + is better than implicit {**, **#, unless 
explicitly suppressed.  Here + is explicit whereas {**, **} is 
implicitly allowing inclusive keys, and the KeyError is expressed 
suppressed by virtue of not using the {**, **} syntax.

It is difficult to take your "explicit/implicit" argument seriously when 
you cannot even decided which is which.



-- 
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[James Lu]

> On Mar 4, 2019, at 10:02 AM, Stefan Behnel  wrote:
> 
> INADA Naoki schrieb am 04.03.19 um 11:15:
>> Why statement is not enough?
> 
> I'm not sure I understand why you're asking this, but a statement is "not
> enough" because it's a statement and not an expression. It does not replace
> the convenience of an expression.
> 
> Stefan
There is already an expression for key-overriding merge. Why do we need a new 
one? 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Stefan Behnel]

INADA Naoki schrieb am 04.03.19 um 11:15:
> Why statement is not enough?

I'm not sure I understand why you're asking this, but a statement is "not
enough" because it's a statement and not an expression. It does not replace
the convenience of an expression.

Stefan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[James Lu]

> On Mar 4, 2019, at 3:41 AM, Stefan Behnel  wrote:
> 
> James Lu schrieb am 04.03.19 um 03:28:
>> I propose that the + sign merge two python dictionaries such that if there 
>> are conflicting keys, a KeyError is thrown.
> 
> Please, no. That would be really annoying.
> 
> If you need that feature, it can become a new method on dicts.
> 
> Stefan
If you want to merge it without a KeyError, learn and use the more explicit 
{**d1, **d2} syntax.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[David Mertz]

On Mon, Mar 4, 2019, 8:30 AM Serhiy Storchaka  wrote:

> But is merging two dicts a common enough problem that needs introducing
> an operator to solve it? I need to merge dicts maybe not more than one
> or two times by year, and I am fine with using the update() method.
> Perhaps {**d1, **d2} can be more appropriate in some cases, but I did not
> encounter such cases yet.
>

Like other folks in the thread, I also want to merge dicts three times per
year. And every one of those times, itertools.ChainMap is the right way to
do that non-destructively, and without copying.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Current use of addition in Python

[Jonathan Fine]

Summary: This thread is for recording current use of addition in
Python. This post covers the built-in types. I'll do Counter and
numpy.array in another post. Please use another thread to discuss
possible possible future use of addition.

BACKGROUND
At present constructions such as
{'a': 1} + {'b': 2}
produce
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

Elsewhere on this list, we're discussing whether to extend dict so
that it supports such constructions, and if so what semantics to give
to addition of dictionaries.

In this thread I intend to record the CURRENT use of addition in
Python, in a neutral manner. (However, I will focus on what might be
called the mathematical properties of addition in Python.)

BUILT-IN TYPES
In this post I talk about the exiting built-in types: see
https://docs.python.org/3/library/stdtypes.html

The NUMERIC types support addition, which has the following properties

Commutative: a + b == b + a
Associative: (a + b) + c == a + (b + c)
Left-cancellation: if (a + b) == (a + c): assert b == c
Right-cancellation: if (a + b) == (c + b): assert a == c
Existence of zero (depending on type): zero + a == a + zero == a
Multiplication by non-negative integer: a + a +  + a == a * n == n * a

Note: For floats, the equalities are ideal. For example, sometimes
   huge + tiny == huge + zero
and so we don't exactly have cancellation.

The SEQUENCE types (except range) have the following properties

Associative
Left-cancellation
Right-cancellation
Existence of zero
Multiplication by non-negative integer.

Note: Even though range is a sequence type, range(3) + range(3) produces
TypeError: unsupported operand type(s) for +: 'range' and 'range'

By the way, although documented, I find this a bit surprising:
>>> (0, 1, 2) * (-1) == ()
True
I'd have expected a ValueError. As things stand
seq * (-1) * (-1)
is not associative. And
  (-1) * seq == -seq
is not true, although the left hand side is defined.

CONCLUSION
I've recorded the behaviour of addition for the built-in types. I'll
do Counter and numpy.array in another post, later today. Please use
another thread to discuss possible future use of addition. My aim in
this thread is to establish and record the present use, to better
support discussion of future use.

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Serhiy Storchaka]

01.03.19 12:44, Steven D'Aprano пише:

On Fri, Mar 01, 2019 at 08:47:36AM +0200, Serhiy Storchaka wrote:


Currently Counter += dict works and Counter + dict is an error. With
this change Counter + dict will return a value, but it will be different
from the result of the += operator.


That's how list.__iadd__ works too: ListSubclass + list will return a
value, but it might not be the same as += since that operates in place
and uses a different dunder method.

Why is it a problem for dicts but not a problem for lists?


Because the plus operator for lists predated any list subclasses.


Also, if the custom dict subclass implemented the plus operator with
different semantic which supports the addition with a dict, this change
will break it, because dict + CustomDict will call dict.__add__ instead
of CustomDict.__radd__.


That's not how operators work in Python or at least that's not how they
worked the last time I looked: if the behaviour has changed without
discussion, that's a breaking change that should be reverted.


You are right.


What's wrong with doing this?

 new = type(self)()

Or the equivalent from C code. If that doesn't work, surely that's the
fault of the subclass, the subclass is broken, and it will raise an
exception.


Try to do this with defaultdict.

Note that none of builtin sequences or sets do this. For good reasons 
they always return an instance of the base type.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[INADA Naoki]

On Mon, Mar 4, 2019 at 10:29 PM Serhiy Storchaka  wrote:
>
> It is not that I like to add an operator for dict merging, but dicts are
> more like sets than sequences: they can not contain duplicated keys and
> the size of the result of merging two dicts can be less than the sum of
> their sizes. Using "|" looks more natural to me than using "+". We
> should look at discussions for using the "|" operator for sets, if the
> alternative of using "+" was considered, I think the same arguments for
> preferring "|" for sets are applicable now for dicts.
>

I concur with Serhiy.  While I don't like adding operator to dict, proposed +/-
looks similar to set |/- than seq +/-.
If we're going to add such set-like operations, operators can be:

* dict & dict_or_set
* dict - dict_or_set
* dict | dict

Especially, dict - set can be more useful than proposed dict - dict.


> But is merging two dicts a common enough problem that needs introducing
> an operator to solve it? I need to merge dicts maybe not more than one
> or two times by year, and I am fine with using the update() method.

+1.

Adding new method to builtin should have a high bar.
Adding new operator to builtin should have a higher bar.
Adding new syntax should have a highest bar.

-- 
INADA Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Serhiy Storchaka]

01.03.19 21:31, Guido van Rossum пише:
On Thu, Feb 28, 2019 at 10:30 PM Serhiy Storchaka 
> wrote:

And this opens a non-easy problem: how to create a mapping of the same
type? Not all mappings, and even not all dict subclasses have a copying
constructor.


There's a compromise solution for this possible. We already do this for 
Sequence and MutableSequence: Sequence does *not* define __add__, but 
MutableSequence *does* define __iadd__, and the default implementation 
just calls self.update(other). I propose the same for Mapping (do 
nothing) and MutableMapping: make the default __iadd__ implementation 
call self.update(other).


This LGTM for mappings. But the problem with dict subclasses still 
exists. If use the copy() method for creating a copy, d1 + d2 will 
always return a dict (unless the plus operator or copy() are redefined 
in a subclass). If use the constructor of the left argument type, there 
will be problems with subclasses with non-compatible constructors (e.g. 
defaultdict).


Anyways, the main reason to prefer d1+d2 over {**d1, **d2} is that the 
latter is highly non-obvious except if you've already encountered that 
pattern before, while d1+d2 is what anybody familiar with other Python 
collection types would guess or propose. And the default semantics for 
subclasses of dict that don't override these are settled with the "d = 
d1.copy(); d.update(d2)" equivalence.


Dicts are not like lists or deques, or even sets. Iterating dicts 
produces keys, but not values. The "in" operator tests a key, but not a 
value.


It is not that I like to add an operator for dict merging, but dicts are 
more like sets than sequences: they can not contain duplicated keys and 
the size of the result of merging two dicts can be less than the sum of 
their sizes. Using "|" looks more natural to me than using "+". We 
should look at discussions for using the "|" operator for sets, if the 
alternative of using "+" was considered, I think the same arguments for 
preferring "|" for sets are applicable now for dicts.


But is merging two dicts a common enough problem that needs introducing 
an operator to solve it? I need to merge dicts maybe not more than one 
or two times by year, and I am fine with using the update() method. 
Perhaps {**d1, **d2} can be more appropriate in some cases, but I did 
not encounter such cases yet.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Left arrow and right arrow operators

[Todd]

What is the operator supposed to do?

On Sun, Mar 3, 2019, 09:52 francismb  wrote:

> Hi,
> the idea here is just to add the __larrow__ and __rarrow__ operators for
> <- and ->.
>
>
> E.g. of use on dicts :
> >>> d1 = {'a':1, 'b':1 }
> >>> d2 = {'a':2 }
> >>> d3 = d1 -> d2
> >>> d3
> {'a':1, 'b':1 }
>
> >>> d1 = {'a':1, 'b':1 }
> >>> d2 = {'a':2 }
> >>> d3 = d1 <- d2
> >>> d3
> {'a':2, 'b':1 }
>
> Or on bools as Modus Ponens [1]
>
> Or your idea/imagination here :-)
>
>
>
> Regards,
> --francis
>
> [1] https://en.wikipedia.org/wiki/Modus_ponens
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Left arrow and right arrow operators

[Calvin Spealman]

I don't like the idea of arrows in both directions when you can just swap
the operands instead

On Sun, Mar 3, 2019 at 9:52 AM francismb  wrote:

> Hi,
> the idea here is just to add the __larrow__ and __rarrow__ operators for
> <- and ->.
>
>
> E.g. of use on dicts :
> >>> d1 = {'a':1, 'b':1 }
> >>> d2 = {'a':2 }
> >>> d3 = d1 -> d2
> >>> d3
> {'a':1, 'b':1 }
>
> >>> d1 = {'a':1, 'b':1 }
> >>> d2 = {'a':2 }
> >>> d3 = d1 <- d2
> >>> d3
> {'a':2, 'b':1 }
>
> Or on bools as Modus Ponens [1]
>
> Or your idea/imagination here :-)
>
>
>
> Regards,
> --francis
>
> [1] https://en.wikipedia.org/wiki/Modus_ponens
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

cspea...@redhat.com  M: +1.336.210.5107

TRIED. TESTED. TRUSTED. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

[Ivan Levkivskyi]

On Sat, 2 Mar 2019 at 19:15, Raymond Hettinger 
wrote:

>
> > On Mar 1, 2019, at 11:31 AM, Guido van Rossum  wrote:
> >
> > There's a compromise solution for this possible. We already do this for
> Sequence and MutableSequence: Sequence does *not* define __add__, but
> MutableSequence *does* define __iadd__, and the default implementation just
> calls self.update(other). I propose the same for Mapping (do nothing) and
> MutableMapping: make the default __iadd__ implementation call
> self.update(other).
>
> Usually, it's easy to add methods to classes without creating disruption,
> but ABCs are more problematic.  If MutableMapping grows an __iadd__()
> method, what would that mean for existing classes that register as
> MutableMapping but don't already implement __iadd__?  When "isinstance(m,
> MutableMapping)" returns True, is it a promise that the API is fully
> implemented? Is this something that mypy could would or should complain
> about?
>

Just to clarify the situation, currently Mapping and MutableMapping are not
protocols from both runtime and mypy points of view. I.e. they don't have
the structural __subclasshook__() (as e.g. Iterable), and are not declared
as Protocol in typeshed. So to implement these (and be considered a subtype
by mypy) one needs to explicitly subclass them (register() isn't supported
by mypy). This means that adding a new method will not cause any problems
here, since the new method will be non-abstract with a default
implementation that calls update() (the same way as for MutableSequence).

The only potential for confusion I see is if there is a class that de-facto
implements current MutableMapping API and made a subclass (at runtime) of
MutableMapping using register(). Then after we add __iadd__, users of that
class might expect that __iadd__ is implemented, while it might be not.
This is however OK I think, since register() is already non type safe. Also
there is a simple way to find if there are any subclassses of
MutableMapping in typeshed that don't have __iadd__: one can *try*
declaring MutableMapping.__iadd__ as abstract, and mypy will error on all
such subclasses.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jimmy Girardet]

> but requires either some obscure syntax or a statement instead of a simple
> expression.
>
> The proposal is to enable the obvious syntax for something that should be
> obvious.
>
> Stefan

The discussions on this list show that the behavior of `+` operator with
dict will never be obvious (first wins or second wins or add results or
raise Exception). So the user will always have to look at the doc or
test it to know the intended behavior.

That said, [1,2] + [3] equals  [1,2,3]  but not[1,2, [3]] and that was
not obvious to me, and I survived.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[INADA Naoki]

On Mon, Mar 4, 2019 at 6:52 PM Stefan Behnel  wrote:
>
> I think the main intentions is to close a gap in the language.
>
> [1,2,3] + [4,5,6]
>
> works for lists and tuples,
>
> {1,2,3} | {4,5,6}
>
> works for sets, but joining two dicts isn't simply
>
> {1:2, 3:4} + {5:6}
>

Operators are syntax borrowed from math.

* Operators are used for concatenate and repeat (Kleene star) in
regular language.
  https://en.wikipedia.org/wiki/Regular_language
  seq + seq and seq * N are very similar to it, although Python used +
instead of
  middle dot (not in ASCII) for concatenate.

* set is directly relating to set in math.  | is well known operator for union.

* In case of merging dict, I don't know obvious background in math or
computer science.

So I feel it's very natural that dict don't have operator for merging.
Isn't "for consistency with other types" a wrong consistency?

> but requires either some obscure syntax or a statement instead of a simple
> expression.
>
> The proposal is to enable the obvious syntax for something that should be
> obvious.

dict.update is obvious already.  Why statement is not enough?

Regards,
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Stefan Behnel]

Jimmy Girardet schrieb am 04.03.19 um 10:12:
> I'm not old on this list but every time there is a proposal, the answer
> is "what are you trying to solve ?".
> 
> Since
> 
> |z ={**x,**y} and z.update(y) Exists, I can"t find the answer.

I think the main intentions is to close a gap in the language.

[1,2,3] + [4,5,6]

works for lists and tuples,

{1,2,3} | {4,5,6}

works for sets, but joining two dicts isn't simply

{1:2, 3:4} + {5:6}

but requires either some obscure syntax or a statement instead of a simple
expression.

The proposal is to enable the obvious syntax for something that should be
obvious.

Stefan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jimmy Girardet]

Hi,

I'm not old on this list but every time there is a proposal, the answer
is "what are you trying to solve ?".

Since

|z ={**x,**y} and z.update(y) Exists, I can"t find the answer.
|

|
|


Le 02/03/2019 à 04:52, Steven D'Aprano a écrit :
> Executive summary:
>
> - I'm going to argue for subclass-preserving behaviour;
>
> - I'm not wedded to the idea that dict += should actually call the 
> update method, so long as it has the same behaviour;
>
> - __iadd__ has no need to return NotImplemented or type-check its 
> argument.
>
> Details below.
>
>
> On Fri, Mar 01, 2019 at 04:10:44PM -0800, Brandt Bucher wrote:
>
> [...]
>> In your Python implementation samples from the PEP, dict subclasses will
>> behave differently from how list subclasses do. List subclasses, without
>> overrides, return *list* objects for bare "+" operations 
> Right -- and I think they are wrong to do so, for reasons I explained 
> here:
>
> https://mail.python.org/pipermail/python-ideas/2019-March/055547.html
>
> I think the standard handling of subclasses in Python builtins is wrong, 
> and I don't wish to emulate that wrong behaviour without a really good 
> reason. Or at least a better reason than "other methods break 
> subclassing unless explicitly overloaded, so this should do so too".
>
> Or at least not without a fight :-)
>
>
>
>> (and "+=" won't call an overridden "extend" method).
> I'm slightly less opinionated about that. Looking more closely into the 
> docs, I see that they don't actually say that += calls list.extend:
>
> s.extend(t)   extends s with the contents of t (for
> or s += t the most part the same as s[len(s):len(s)] = t)
>
> https://docs.python.org/3/library/stdtypes.html#mutable-sequence-types
>
> only that they have the same effect. So the wording re lists calling 
> extend certainly needs to be changed. But that doesn't mean that we must 
> change the implementation. We have a choice:
>
> - regardless of what lists do, we define += for dicts as literally 
>   calling dict.update; the more I think about it, the less I like this.
>
> - Or we say that += behaves similarly to update, without actually 
>   calling the method. I think I prefer this.
>
> (The second implies either that += either contains a duplicate of the 
> update logic, or that += and update both delegate to a private, C-level 
> function that does most of the work.)
>
> I think that the second approach (define += as having the equivalent 
> semantics of update but without actually calling the update method) is 
> probably better. That decouples the two methods, allows subclasses to 
> change one without necessarily changing the other.
>
>
>> So a more analogous
>> pseudo-implementation (if that's what we seek) would look like:
>>
>> def __add__(self, other):
>> if isinstance(other, dict):
>> new = dict.copy(self)
>> dict.update(new, other)
>> return new
>> return NotImplemented
> We should not require the copy method.
>
> The PEP should be more explicit that the approximate implementation does 
> not imply the copy() and update() methods are actually called.
>
>
>> def __iadd__(self, other):
>> if isinstance(other, dict):
>> dict.update(self, other)
>> return self
>> return NotImplemented
> I don't agree with that implementation.
>
> According to PEP 203, which introduced augmented assignment, the 
> sequence of calls in ``d += e`` is:
>
> 1. Try to call ``d.__iadd__(e)``.
>
> 2. If __iadd__ is not present, try ``d.__add__(e)``.
>
> 3. If __add__ is missing too, try ``e.__radd__(d)``.
>
> but my tests suggest this is inaccurate. I think the correct behaviour 
> is this:
>
> 1. Try to call ``d.__iadd__(e)``.
>
> 2. If __iadd__ is not present, or if it returns NotImplemented, 
>try ``d.__add__(e)``.
>
> 3. If __add__ is missing too, or if it returns NotImplemented,
>fail with TypeError.
>
> In other words, e.__radd__ is not used.
>
> We don't want dict.__iadd__ to try calling __add__, since the later is 
> more restrictive and less efficient than the in-place merge. So there is 
> no need for __iadd__ to return NotImplemented. It should either succeed 
> on its own, or fail hard:
>
> def __iadd__(self, other):
> self.update(other)
> return self
>
> Except that the actual C implementation won't call the update method 
> itself, but will follow the same semantics.
>
> See the docstring for dict.update for details of what is accepted by 
> update.
>
>

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Stefan Behnel]

James Lu schrieb am 04.03.19 um 03:28:
> I propose that the + sign merge two python dictionaries such that if there 
> are conflicting keys, a KeyError is thrown. 

Please, no. That would be really annoying.

If you need that feature, it can become a new method on dicts.

Stefan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/