Re: [Numpy-discussion] problem with assigning to recarrays
Many thanks for your willingness to help out with this. Not to
belabor the point, but I notice that the rules you lay out below don't
quite explain why the following syntax works as I originally expected:
r[0].field1 = 1
I'm guessing this is because r[0].field1 is already an existing scalar
object, with an address in memory, so it can be changed in place,
whereas this syntax would need to create an entirely new array object
(not even a copy, exactly):
r[where(r.field1 == 0)].field1
No need to respond if this understanding is correct. I just wanted to
write it down in case someone else is searching the archive with this
question in the future.
BFG
On Feb 27, 2009, at 10:58 PM, Robert Kern wrote:
On Fri, Feb 27, 2009 at 19:06, Brian Gerke
bge...@slac.stanford.edu wrote:
On Feb 27, 2009, at 4:30 PM, Robert Kern wrote:
r[where(r.field1 == 1.)] make a copy. There is no way for us to
construct a view onto the original memory for this circumstance
given
numpy's memory model.
Many thanks for the quick reply. I assume that this is true only for
record arrays, not for ordinary arrays? Certainly I can make an
assignment in this way with a normal array.
Well, you are doing two very different things. Let's back up a bit.
Python gives us two hooks to modify an object in-place with an
assignment: __setitem__ and __setattr__.
x[item] = y == x.__setitem__(item, y)
x.attr = y == x.__setattr__('attr', y)
Now, we don't need to restrict ourselves to just variables for 'x'; we
can have any expression that evaluates to an object.
(expr)[item] = y == (expr).__setitem__(item, y)
(expr).attr = y == (expr).__setattr__('attr', y)
The key here is that the (expr) on the LHS is evaluated just like
any expression appearing anywhere else in your code. The only special
in-place behavior is restricted to the *outermost* [item] or
.attr.
So when you do this:
r[where(r.field1 == 1.)].field2 = 1.0
it translates to something like this:
tmp = r.__getitem__(where(r.field1 == 1.0)) # Makes a copy!
tmp.__setattr__('field2', 1.0)
Note that the first line is a __getitem__, not a __setitem__ which can
modify r in-place.
Also, if it is truly impossible to change this behavior, or to have
it
raise an error--then are there any best-practice suggestions for how
to remember and avoid running into this non-obvious behavior? If one
thinks of record arrays as inheriting from numpy arrays, then this
problem is certainly unexpected.
It's a natural consequence of the preceding rules. This a Python
thing, not a difference between numpy arrays and record arrays. Just
keep those rules in mind.
Also, I've just found that the following syntax does do what is
expected:
(r.field2)[where(field1 == 1.)] = 1.
It is at least a little aesthetically displeasing that the syntax
works one way but not the other. Perhaps my best bet is to stick
with
this syntax and forget that the other exists? A less-than-satisfying
solution, but workable.
If you drop the extraneous bits, it becomes a fair bit more readable:
r.field2[r.field1 == 1] = 1
This is idiomatic; you'll see it all over the place where record
arrays are used. The reason that this form modifies r in-place is
because r.__getattr__('field2') is able to return a view rather than a
copy.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] problem with assigning to recarrays
On Mon, Mar 2, 2009 at 19:20, Brian Gerke bge...@slac.stanford.edu wrote: Many thanks for your willingness to help out with this. Not to belabor the point, but I notice that the rules you lay out below don't quite explain why the following syntax works as I originally expected: r[0].field1 = 1 I'm guessing this is because r[0].field1 is already an existing scalar object, with an address in memory, so it can be changed in place, whereas this syntax would need to create an entirely new array object (not even a copy, exactly): r[where(r.field1 == 0)].field1 No need to respond if this understanding is correct. I just wanted to write it down in case someone else is searching the archive with this question in the future. Close. It's not already existing, but a record scalar that gets created by a[0] is a view onto the corresponding element in the original array and is mutable. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] problem with assigning to recarrays
Hi- I'm quite new to numpy and to python in general, so I apologize if I'm missing something obvious, but I've come across some seemingly nasty behavior when trying to assign values to the fields of an indexed subarray of a numpy record array. Perhaps an example would explain it best. First, I make a boring record array: In [124]: r = rec.fromarrays([zeros(5), zeros(5)], names='field1,field2') This has five elements with two fields, all values are zero. Now I can change the values for field1 for a few of the array elements: In [125]: r[1].field1=1 In [126]: r[3].field1=1 Let's check and make sure that worked: In [127]: print r.field1 [ 0. 1. 0. 1. 0.] So far, so good. Now I want to change the value of field2 for those same elements: In [128]: r[where(r.field1 == 1.)].field2 = 1 Ok, so now the values of field 2 have been changed, for those elements right? In [129]: r.field2 Out[129]: array([ 0., 0., 0., 0., 0.]) Wait. What? That can't be right. Let's check again: In [130]: print r[where(r.field1 == 1.)].field2 [ 0. 0.] Ok, so it appears that I can *access* fields in this array with an array of indices, but I can't assign new values to fields so accessed. However, I *can* change the values if I use a scalar index. This is different from the behavior of ordinary arrays, for which I can reassign elements' values either way. Moreover, when I try to reassign record array fields by indexing with an array of indices, it would appear that nothing at all happens. This syntax is equivalent to the pass command. So, my question is this: is there some reason for this behavior in record arrays, which is unexpectedly different from the behavior of normal arrays, and rather confusing. If so, why does the attempt to assign values to fields of an indexed subarray not raise some kind of error, rather than doing nothing? I think it's unlikely that I've actually found a bug in numpy, but this behavior does not make sense to me. Thanks for any insights, Brian ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] problem with assigning to recarrays
On Fri, Feb 27, 2009 at 18:26, Brian Gerke bge...@slac.stanford.edu wrote: Hi- I'm quite new to numpy and to python in general, so I apologize if I'm missing something obvious, but I've come across some seemingly nasty behavior when trying to assign values to the fields of an indexed subarray of a numpy record array. Perhaps an example would explain it best. First, I make a boring record array: In [124]: r = rec.fromarrays([zeros(5), zeros(5)], names='field1,field2') This has five elements with two fields, all values are zero. Now I can change the values for field1 for a few of the array elements: In [125]: r[1].field1=1 In [126]: r[3].field1=1 Let's check and make sure that worked: In [127]: print r.field1 [ 0. 1. 0. 1. 0.] So far, so good. Now I want to change the value of field2 for those same elements: In [128]: r[where(r.field1 == 1.)].field2 = 1 Ok, so now the values of field 2 have been changed, for those elements right? In [129]: r.field2 Out[129]: array([ 0., 0., 0., 0., 0.]) Wait. What? That can't be right. Let's check again: In [130]: print r[where(r.field1 == 1.)].field2 [ 0. 0.] Ok, so it appears that I can *access* fields in this array with an array of indices, but I can't assign new values to fields so accessed. However, I *can* change the values if I use a scalar index. This is different from the behavior of ordinary arrays, for which I can reassign elements' values either way. Moreover, when I try to reassign record array fields by indexing with an array of indices, it would appear that nothing at all happens. This syntax is equivalent to the pass command. So, my question is this: is there some reason for this behavior in record arrays, which is unexpectedly different from the behavior of normal arrays, and rather confusing. If so, why does the attempt to assign values to fields of an indexed subarray not raise some kind of error, rather than doing nothing? I think it's unlikely that I've actually found a bug in numpy, but this behavior does not make sense to me. r[where(r.field1 == 1.)] make a copy. There is no way for us to construct a view onto the original memory for this circumstance given numpy's memory model. r[where(r.field1 == 1.)].field2 = 0.0 assigns to the copy. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] problem with assigning to recarrays
As a follow-up to Robert's answer: r[r.field1 == 1].field2 = 1 doesn't work, but r.field2[r.field1==1] = 1 does. So far, so good. Now I want to change the value of field2 for those same elements: In [128]: r[where(r.field1 == 1.)].field2 = 1 Ok, so now the values of field 2 have been changed, for those elements right? In [129]: r.field2 Out[129]: array([ 0., 0., 0., 0., 0.]) Wait. What? That can't be right. Let's check again: In [130]: print r[where(r.field1 == 1.)].field2 [ 0. 0.] Ok, so it appears that I can *access* fields in this array with an array of indices, but I can't assign new values to fields so accessed. However, I *can* change the values if I use a scalar index. This is different from the behavior of ordinary arrays, for which I can reassign elements' values either way. Moreover, when I try to reassign record array fields by indexing with an array of indices, it would appear that nothing at all happens. This syntax is equivalent to the pass command. So, my question is this: is there some reason for this behavior in record arrays, which is unexpectedly different from the behavior of normal arrays, and rather confusing. If so, why does the attempt to assign values to fields of an indexed subarray not raise some kind of error, rather than doing nothing? I think it's unlikely that I've actually found a bug in numpy, but this behavior does not make sense to me. Thanks for any insights, Brian ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] problem with assigning to recarrays
On Feb 27, 2009, at 4:30 PM, Robert Kern wrote: r[where(r.field1 == 1.)] make a copy. There is no way for us to construct a view onto the original memory for this circumstance given numpy's memory model. Many thanks for the quick reply. I assume that this is true only for record arrays, not for ordinary arrays? Certainly I can make an assignment in this way with a normal array. Also, if it is truly impossible to change this behavior, or to have it raise an error--then are there any best-practice suggestions for how to remember and avoid running into this non-obvious behavior? If one thinks of record arrays as inheriting from numpy arrays, then this problem is certainly unexpected. Also, I've just found that the following syntax does do what is expected: (r.field2)[where(field1 == 1.)] = 1. It is at least a little aesthetically displeasing that the syntax works one way but not the other. Perhaps my best bet is to stick with this syntax and forget that the other exists? A less-than-satisfying solution, but workable. Brian r[where(r.field1 == 1.)].field2 = 0.0 assigns to the copy. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] problem with assigning to recarrays
On Fri, Feb 27, 2009 at 19:06, Brian Gerke bge...@slac.stanford.edu wrote:
On Feb 27, 2009, at 4:30 PM, Robert Kern wrote:
r[where(r.field1 == 1.)] make a copy. There is no way for us to
construct a view onto the original memory for this circumstance given
numpy's memory model.
Many thanks for the quick reply. I assume that this is true only for
record arrays, not for ordinary arrays? Certainly I can make an
assignment in this way with a normal array.
Well, you are doing two very different things. Let's back up a bit.
Python gives us two hooks to modify an object in-place with an
assignment: __setitem__ and __setattr__.
x[item] = y == x.__setitem__(item, y)
x.attr = y == x.__setattr__('attr', y)
Now, we don't need to restrict ourselves to just variables for 'x'; we
can have any expression that evaluates to an object.
(expr)[item] = y == (expr).__setitem__(item, y)
(expr).attr = y == (expr).__setattr__('attr', y)
The key here is that the (expr) on the LHS is evaluated just like
any expression appearing anywhere else in your code. The only special
in-place behavior is restricted to the *outermost* [item] or
.attr.
So when you do this:
r[where(r.field1 == 1.)].field2 = 1.0
it translates to something like this:
tmp = r.__getitem__(where(r.field1 == 1.0)) # Makes a copy!
tmp.__setattr__('field2', 1.0)
Note that the first line is a __getitem__, not a __setitem__ which can
modify r in-place.
Also, if it is truly impossible to change this behavior, or to have it
raise an error--then are there any best-practice suggestions for how
to remember and avoid running into this non-obvious behavior? If one
thinks of record arrays as inheriting from numpy arrays, then this
problem is certainly unexpected.
It's a natural consequence of the preceding rules. This a Python
thing, not a difference between numpy arrays and record arrays. Just
keep those rules in mind.
Also, I've just found that the following syntax does do what is
expected:
(r.field2)[where(field1 == 1.)] = 1.
It is at least a little aesthetically displeasing that the syntax
works one way but not the other. Perhaps my best bet is to stick with
this syntax and forget that the other exists? A less-than-satisfying
solution, but workable.
If you drop the extraneous bits, it becomes a fair bit more readable:
r.field2[r.field1 == 1] = 1
This is idiomatic; you'll see it all over the place where record
arrays are used. The reason that this form modifies r in-place is
because r.__getattr__('field2') is able to return a view rather than a
copy.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
