[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Changes by Martin Panter :


--
assignee: docs@python -> martin.panter
nosy: +berker.peksag
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Roundup Robot]

Roundup Robot added the comment:

New changeset ec373d762213 by Martin Panter in branch '2.7':
Issue #16701: Document += and *= for mutable sequences
https://hg.python.org/cpython/rev/ec373d762213

New changeset f83db23bec7f by Martin Panter in branch '3.4':
Issue #16701: Document += and *= for mutable sequences
https://hg.python.org/cpython/rev/f83db23bec7f

New changeset 6e43a3833293 by Martin Panter in branch '3.5':
Issue #16701: Merge sequence docs from 3.4 into 3.5
https://hg.python.org/cpython/rev/6e43a3833293

New changeset a92466bf16cc by Martin Panter in branch 'default':
Issue #16701: Merge sequence docs from 3.5
https://hg.python.org/cpython/rev/a92466bf16cc

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

Looks good to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Martin Panter added the comment:

“For the most part” works for me. Here is the patch.

--
Added file: http://bugs.python.org/file40637/seq-inplace.v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

Something I missed on the first review: why did you change "the same as" to 
"usually the same as"?  When is it different?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Martin Panter added the comment:

The “s += t” operation assigns the result back to s. So it could involve an 
extra __setattr__()/__setitem__() call, or an exception trying to modify a 
tuple item, etc. The extend() and slice assignment versions don’t have this 
extra stage.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

Ah, good point.  I'd say something like "for the most part" then instead of 
"usually", since what you describe always happens.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

Wording looks ok...except that technically it is not that 'n' is an integer, 
it's that 'n' can play the role of an integer (ie: it has an __index__ method).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Martin Panter added the comment:

New patch mentioning __index__()

--
Added file: http://bugs.python.org/file40560/seq-inplace.v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Martin Panter]

Martin Panter added the comment:

Here is a patch documenting the += and *= mutable sequence operations. Please 
review my wording.

These operations already seem to be tested, at least on the basic mutable 
sequences: see /Lib/test/list_tests.py, test_array, test_collections, 
test_bytes (tests bytearray).

The only other places that I thought might be missing augmented assignment were 
for sets, but there is no problem there: 
.

However, there are other operations that I think may be missing from this page 
of the documentation. But it might be better to handle those in a separate bug 
report. Some of this could build off the work in Issue 12067.

* Equality comparisons (mentioned for range and dict, but apparently not tuple, 
set, strings, etc)
* Ordering comparisons (not supported for range)
* min() and max() don’t really belong; maybe substitute with iter()

--
keywords: +patch
nosy: +martin.panter
stage: needs patch -> patch review
versions: +Python 3.6
Added file: http://bugs.python.org/file40552/seq-inplace.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

I suggested updating the library reference in my first reply on this issue.  No 
one has proposed a patch yet, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Brendan Barnwell]

Brendan Barnwell added the comment:

This needs to be fixed.  The documentation for the behavior of +=  on lists 
needs to be with the documentation on lists.  The existing, vague documentation 
that += works in-place "when possible" is insufficient.

A central feature of Python is that the behavior of operators like + and += is 
overridable on a per-type basis.  Hence, the Language Reference is not the 
appropriate place for describing the behavior of += on a particular type.  The 
behavior of += on lists should be documented where the behavior of lists is 
documented (as, for instance, the behavior of + on lists already is), not where 
the syntax of += is documented.

Someone just asked a question on StackOverflow about this 
(http://stackoverflow.com/questions/32657637/python-changing-variables-vs-arrays-in-functions/32657770#32657770).
  It is embarrassing to have to tell people, "To know what += does on a type, 
you need to look at the documentation for that type. . . except that the 
documentation for the builtin types doesn't document what some operators do."

--
nosy: +BrenBarn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
nosy: +rhettinger

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Terry J. Reedy]

Terry J. Reedy added the comment:

Augmented assignment confuses enough people that I think we can improve the 
doc. In #21358 I suggest an augmented version of the previous claim, about 
evaluation just once. I think something here is needed perhaps even more. I 
have not decided what just yet.

--
nosy: +terry.reedy
versions: +Python 3.5 -Python 2.6, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[benrg]

benrg added the comment:

 AFAIK in C x += 1 is equivalent to x++, and both are semantically
 more about incrementing (mutating) the value of x than about creating a
 new value that gets assigned to x. Likewise it seems to me more natural
 to interpret x += y as add the value of y to the object x than add
 x and y together and save the result in x.

Look, it's very simple: in C, ++x and x += 1 and x = x + 1 all mean the same 
thing. You can argue about how to describe the thing that they do, but there's 
only one thing to describe. Likewise, in every other language that borrows the 
op= syntax from C, it is a shorthand for the expanded version with the bare 
operator. As far as I know, Python is the only exception. If you know of 
another exception please say so.


  x = ([],)
  x[0] += [1]
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: 'tuple' object does not support item assignment
  x
 ([1],)

I actually knew about this. It's an understandably difficult corner case, since 
the exception is raised after __iadd__ returns, so there's no chance for it to 
roll back its changes.

At least, I thought it was a difficult corner case back when I thought the 
in-place update was a mere optimization. But if += really means .extend() on 
lists, this should not raise an exception at all. In fact there's no sense in 
having __iadd__ return a value that gets assigned anywhere, since mutable 
objects always mutate and return themselves and immutable objects don't define 
__iadd__. It looks like the interface was designed with the standard semantics 
in mind but the implementation did something different, leaving a vestigial 
assignment that's always a no-op. What a disaster.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[benrg]

benrg added the comment:

This is bizarre:

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit 
(Intel)] on win32
Type help, copyright, credits or license for more information.
 x = y = [1, 2]
 x += [3]
 y
[1, 2, 3]
 x = y = {1, 2}
 x -= {2}
 y
{1}


Since when has this been standard behavior? The documentation says:

An augmented assignment expression like x += 1 can be rewritten as x = x + 1 
to achieve a similar, but not exactly equal effect. In the augmented version, x 
is only evaluated once. Also, when possible, the actual operation is performed 
in-place, meaning that rather than creating a new object and assigning that to 
the target, the old object is modified instead.

What is when possible supposed to mean here? I always thought it meant when 
there are known to be no other references to the object. If op= is always 
destructive on lists and sets, then where possible needs to be changed to 
always and a prominent warning added, like WARNING: X OP= EXPR DOES NOT 
BEHAVE EVEN REMOTELY LIKE X = X OP EXPR IN PYTHON WHEN X IS A MUTABLE OBJECT, 
IN STARK CONTRAST TO EVERY OTHER LANGUAGE WITH A SIMILAR SYNTAX.

--
nosy: +benrg

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ezio Melotti]

Ezio Melotti added the comment:

 What is when possible supposed to mean here?

Generally it means when the object is mutable:
 l = [1,2,3]
 id(l)
3074713484
 l += [4]
 id(l)
3074713484
 t = (1,2,3)
 id(t)
3074704004
 t += (4,)
 id(t)
3075304860

Tuples are not mutable, so it's not possible to modify them in place, and a new 
tuple needs to be created.
Note that while most mutable objects in the stdlib that support += do indeed 
modify the object rather than creating a new one, I don't think this is 
strictly required.
IOW that paragraph is already warning you that (with mutable objects) the 
object might be reused, depending on the implementation.  Maybe this should be 
clarified?
(IIRC in CPython it could be possible that in some situations an immutable 
object still has the same id after an augmented assignment, but, if it really 
happens, it is an implementation detail and shouldn't affect semantics.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

If you really want to freak out, try this:

 x = ([],)
 x[0] += [1]
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: 'tuple' object does not support item assignment
 x
([1],)

but to answer your question, it has *always* worked that way, from the time 
augmented assignment was introduced (see, eg, issue 1306777, which was reported 
against python 2.4).  

Remember, Python names refer to pointers to objects, they are not variables in 
the sense that other languages have variables.

Guido resisted augmented assignment for a long time.  These confusions speak to 
why.

As far as I know Ezio is correct, when possible means when the target is 
mutable.  The documentation should probably be clarified on that point.  I'm 
not sure it is practical to let whether or not the target is mutated be an 
implementation detail. IMO the behavior must be clearly defined for each type 
that is built in to Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ezio Melotti]

Ezio Melotti added the comment:

To clarify, with depends on the implementation I meant the way a particular 
class is implemented (i.e. a class might decide to return a new object even if 
it's mutable).
The behavior of built-in types is well defined and should be the same across 
all the Python implementations.
Regarding the comment about immutable types, it's something specific to CPython 
(I don't remember the specific details though, so I might be wrong), and 
somewhat similar to:
 'a'*20 is 'a'*20
True
 'a'*25 is 'a'*25
False
This shouldn't be a problem though, so if you e.g. do x = y = immutableobj;  y 
+= 1, 'x' should never be affected.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[benrg]

benrg added the comment:

 As far as I know Ezio is correct, when possible means when the target is 
 mutable.  The documentation should probably be clarified on that point.

Yes, it needs to be made very, very clear in the documentation. As I said, I'm 
not aware of any other language in which var op= expr does not mean the same 
thing as var = var op expr. I'm actually amazed that neither of you recognize 
the weirdness of this behavior (and even more amazed that GvR apparently 
didn't). I'm an experienced professional programmer, and I dutifully read the 
official documentation cover to cover when I started programming in Python, and 
I interpreted this paragraph wrongly, because I interpreted it in the only way 
that made sense given the meaning of these operators in every other language 
that has them. Python is designed to be unsurprising; constructs generally mean 
what it looks like they mean. You need to explain this unique feature of Python 
in terms so clear that it can't possibly be mistaken for the behavior of all of 
the other languages.

 Remember, Python names refer to pointers to objects, they are not variables 
 in the sense that other languages have variables.

That has nothing to do with this. Yes, in Python (and Java and Javascript and 
many other languages) all objects live on the heap, local variables are not 
first-class objects, and var = expr is a special form. That doesn't change the 
fact that in all of those other languages, var += expr means var = var + expr. 
In C++ local variables are first-class objects and var += expr means 
var.operator+=(expr) or operator+=(var, expr), and this normally modifies the 
thing on the left in a way that's visible through references. But in C++, var = 
var + expr also modifies the thing on the left, in the same way.

In Python and Java and Javascript and ..., var = value never visibly mutates 
any heap object, and neither does var = var + value (in any library that 
defines a sane + operator), and therefore neither should var += value (again, 
in any sanely designed library). And it doesn't. Except in Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ezio Melotti]

Ezio Melotti added the comment:

 Python is designed to be unsurprising; constructs generally mean
 what it looks like they mean.

AFAIK in C x += 1 is equivalent to x++, and both are semantically more 
about incrementing (mutating) the value of x than about creating a new value 
that gets assigned to x.
Likewise it seems to me more natural to interpret x += y as add the value of 
y to the object x than add x and y together and save the result in x.
Clearly if you are used to other languages with different semantics you might 
expect a different behavior, but you could say the same about the fact that 
int/int gives float on Python 3: it's surprising if you are used to other 
languages like C, but otherwise it's more natural.

 I interpreted this paragraph wrongly, because I interpreted it in the
 only way that made sense given the meaning of these operators in 
 every other language that has them.

It seems to me that the documentation doesn't leave much room for 
interpretation regarding the fact that the object is mutated in place; the only 
problem is that it doesn't specify clearly what are the objects that do this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
nosy: +ezio.melotti
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[Ashwini Chaudhary]

New submission from Ashwini Chaudhary:

I think the python docs are missing the behavior of += for lists. It actually 
calls list.extend() but can't find that anywhere in docs expect in source code, 
http://hg.python.org/cpython/file/2d2d4807a3ed/Objects/listobject.c#l892.

--
assignee: docs@python
components: Documentation
messages: 177627
nosy: docs@python, montysinngh
priority: normal
severity: normal
status: open
title: Docs missing the behavior of += (in-place add) for lists.
type: enhancement
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 
3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16701] Docs missing the behavior of += (in-place add) for lists.

[R. David Murray]

R. David Murray added the comment:

Well, it is effectively documented by the text here:

   
http://docs.python.org/3/reference/simple_stmts.html#augmented-assignment-statements

since a + b is logically equivalent to a.extend(b) when a is being updated 
in-place.  The fact that it is in fact implemented using extend is an 
implementation detail.

That said, it would be logical to add an entry for the augmented assignment to 
the table here:

   http://docs.python.org/3/library/stdtypes.html#mutable-sequence-types

There also may be other places in that chapter where augmented assignment 
deserves mention.

--
nosy: +r.david.murray
versions:  -Python 3.1, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16701
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com