[Numpy-discussion] Array addition inconsistency
Dear all, After some surprise, I noticed an inconsistency while adding array slices: a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7]) versus inplace a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10]) My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum(). This behaviour is certainly surprising, and leads to unexpected errors if used without testing. Best, Martin -- Dr. Martin Lüthi lue...@vaw.baug.ethz.ch VAW Glaciology http://www.vaw.ethz.ch/gz ETH Zürich http://people.ee.ethz.ch/~luethim 8093 ZürichTel: +41 44 632 40 93 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array addition inconsistency
On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi lue...@vaw.baug.ethz.ch wrote: Dear all, After some surprise, I noticed an inconsistency while adding array slices: a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7]) versus inplace a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10]) My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum(). Correct. Not creating intermediate storage is the point of using augmented assignment. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array addition inconsistency
On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern robert.k...@gmail.com wrote: On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi lue...@vaw.baug.ethz.ch wrote: Dear all, After some surprise, I noticed an inconsistency while adding array slices: a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7]) versus inplace a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10]) My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum(). Correct. Not creating intermediate storage is the point of using augmented assignment. This can be very sneaky. a = np.arange(5) a[:-1] = a[:-1] + a[1:] a array([1, 3, 5, 7, 4]) a = np.arange(5) a[:-1] += a[1:] a array([1, 3, 5, 7, 4]) So, if someone is semi-careful and tries out that example, they might (incorrectly) assume that such operations are safe without realizing that it was safe because the values of a[1:] were ahead of the values of a[:-1] in memory. I could easily imagine a situation where views of an array are passed around only to finally end up in an in-place operation like this and sometimes be right and sometimes be wrong. Maybe there can be some simple check that could be performed to detect this sort of situation? Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array addition inconsistency
On Thu, Aug 29, 2013 at 9:39 AM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern robert.k...@gmail.comwrote: On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi lue...@vaw.baug.ethz.ch wrote: Dear all, After some surprise, I noticed an inconsistency while adding array slices: a = np.arange(5) a[1:] = a[1:] + a[:-1] a array([0, 1, 3, 5, 7]) versus inplace a = np.arange(5) a[1:] += a[:-1] a array([ 0, 1, 3, 6, 10]) My suspicition is that the second variant does not create intermediate storage, and thus works on the intermediate result, effectively performing a.cumsum(). Correct. Not creating intermediate storage is the point of using augmented assignment. This can be very sneaky. a = np.arange(5) a[:-1] = a[:-1] + a[1:] a array([1, 3, 5, 7, 4]) a = np.arange(5) a[:-1] += a[1:] a array([1, 3, 5, 7, 4]) So, if someone is semi-careful and tries out that example, they might (incorrectly) assume that such operations are safe without realizing that it was safe because the values of a[1:] were ahead of the values of a[:-1] in memory. I could easily imagine a situation where views of an array are passed around only to finally end up in an in-place operation like this and sometimes be right and sometimes be wrong. Maybe there can be some simple check that could be performed to detect this sort of situation? I think the main message is that you don't use inplace operation with mutables unless you know what you are doing, and you really need them. inplace cumsum in python a = range(5) for i in xrange(1, 5): a[i] += a[i-1] ... a [0, 1, 3, 6, 10] Defensive programming. Josef Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
