Re: [Numpy-discussion] slicing / indexing question
Hi Brett, Josef, Gökhan, and Anne, Many thanks for the suggestions! This indexing problem was not as straight-forward as I had anticipated. I have it working now, thanks to your help. Gökhan, I'm driving an ecosystem--atmosphere carbon dioxide flux model with MODIS measurements, so this_par and pars are parameter values that I need to combine with the measurements. The arrays represent time series of model parameters and MODIS reflectances and land surface products (enhanced vegetation index, land cover class, vegetation dynamics) at 1 km resolution for 1200 km by 1200 km tiles. There are lots of IDL and Matlab folks in my department too. I've been using R and, more recently, Scipy... Being able to test out code offline on my own machine without worrying about licenses (not only cost but also talking to license servers, availability of site licenses, etc.) is a big big help! Along those lines, many thanks also to all who have put so much time and energy into developing python and Scipy. It's a great platform! -Tim On Wed, Sep 2010, 22 at 02:22:50PM -0400, Anne Archibald wrote: On 21 September 2010 19:20, Timothy W. Hilton hil...@meteo.psu.edu wrote: I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. How about this? In [1]: A = np.zeros((2,3,5)) In [2]: B = np.array([1,2]) In [3]: C = np.zeros((3,5), dtype=np.bool) In [4]: C[1,1] = True In [5]: C[2,3] = True In [6]: C Out[6]: array([[False, False, False, False, False], [False, True, False, False, False], [False, False, False, True, False]], dtype=bool) In [7]: A[:,C] = B[:,np.newaxis] In [8]: A Out[8]: array([[[ 0., 0., 0., 0., 0.], [ 0., 1., 0., 0., 0.], [ 0., 0., 0., 1., 0.]], [[ 0., 0., 0., 0., 0.], [ 0., 2., 0., 0., 0.], [ 0., 0., 0., 2., 0.]]]) The key is that indexing with C replaces the two axes C is indexing with only one; boolean indexing necessarily flattens the relevant axes. You can check this with (e.g.) A[:,C].shape. Be careful with these mixed indexing modes (partly fancy indexing, partly slicing) as they can sometimes seem to reorder your axes for you: In [16]: np.zeros((2,3,7))[:,np.ones(5,dtype=int),np.ones(5,dtype=int)].shape Out[16]: (2, 5) In [17]: np.zeros((2,3,7))[np.ones(5,dtype=int),:,np.ones(5,dtype=int)].shape Out[17]: (5, 3) In [18]: np.zeros((2,3,7))[np.ones(5,dtype=int),np.ones(5,dtype=int),:].shape Out[18]: (5, 7) Anne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] slicing / indexing question
On 21 September 2010 19:20, Timothy W. Hilton hil...@meteo.psu.edu wrote: I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. How about this? In [1]: A = np.zeros((2,3,5)) In [2]: B = np.array([1,2]) In [3]: C = np.zeros((3,5), dtype=np.bool) In [4]: C[1,1] = True In [5]: C[2,3] = True In [6]: C Out[6]: array([[False, False, False, False, False], [False, True, False, False, False], [False, False, False, True, False]], dtype=bool) In [7]: A[:,C] = B[:,np.newaxis] In [8]: A Out[8]: array([[[ 0., 0., 0., 0., 0.], [ 0., 1., 0., 0., 0.], [ 0., 0., 0., 1., 0.]], [[ 0., 0., 0., 0., 0.], [ 0., 2., 0., 0., 0.], [ 0., 0., 0., 2., 0.]]]) The key is that indexing with C replaces the two axes C is indexing with only one; boolean indexing necessarily flattens the relevant axes. You can check this with (e.g.) A[:,C].shape. Be careful with these mixed indexing modes (partly fancy indexing, partly slicing) as they can sometimes seem to reorder your axes for you: In [16]: np.zeros((2,3,7))[:,np.ones(5,dtype=int),np.ones(5,dtype=int)].shape Out[16]: (2, 5) In [17]: np.zeros((2,3,7))[np.ones(5,dtype=int),:,np.ones(5,dtype=int)].shape Out[17]: (5, 3) In [18]: np.zeros((2,3,7))[np.ones(5,dtype=int),np.ones(5,dtype=int),:].shape Out[18]: (5, 7) Anne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] slicing / indexing question
Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim -- Timothy W. Hilton PhD Candidate, Department of Meteorology The Pennsylvania State University 503 Walker Building, University Park, PA 16802 hil...@meteo.psu.edu ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] slicing / indexing question
On Tue, Sep 21, 2010 at 6:20 PM, Timothy W. Hilton hil...@meteo.psu.edu wrote: Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim This works, although I imagine it could be streamlined. In [1]: this_par = N.ones((2,4,4)) In [2]: idx = N.random.random((4,4)) 0.5 In [3]: pars = N.arange(2) - 10 In [4]: this_par[:,idx] = N.tile(pars, (idx.sum(), 1)).transpose() In [5]: idx Out[5] array([[ True, False, True, False], [False, False, True, True], [False, False, False, False], [False, False, False, False]], dtype=bool) In [6]: this_par Out[6]: array([[[-10., 1., -10., 1.], [ 1., 1., -10., -10.], [ 1., 1.,1., 1.], [ 1., 1.,1., 1.]], [[ -9., 1., -9., 1.], [ 1., 1., -9., -9.], [ 1., 1.,1., 1.], [ 1., 1.,1., 1.]]]) Brett ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] slicing / indexing question
On Tue, Sep 21, 2010 at 7:54 PM, Brett Olsen brett.ol...@gmail.com wrote: On Tue, Sep 21, 2010 at 6:20 PM, Timothy W. Hilton hil...@meteo.psu.edu wrote: Hello, I have an indexing problem which I suspect has a simple solution, but I've not been able to piece together various threads I've read on this list to solve. I have an 80x1200x1200 nd.array of floats this_par. I have a 1200x1200 boolean array idx, and an 80-element float array pars. For each element of idx that is True, I wish to replace the corresponding 80x1x1 slice of this_par with the elements of pars. I've tried lots of variations on the theme of this_par[idx[np.newaxis, ...]] = pars[:, np.newaxis, np.newaxis] but so far, no dice. Any help greatly appreciated! Thanks, Tim This works, although I imagine it could be streamlined. In [1]: this_par = N.ones((2,4,4)) In [2]: idx = N.random.random((4,4)) 0.5 In [3]: pars = N.arange(2) - 10 In [4]: this_par[:,idx] = N.tile(pars, (idx.sum(), 1)).transpose() In [5]: idx Out[5] array([[ True, False, True, False], [False, False, True, True], [False, False, False, False], [False, False, False, False]], dtype=bool) In [6]: this_par Out[6]: array([[[-10., 1., -10., 1.], [ 1., 1., -10., -10.], [ 1., 1., 1., 1.], [ 1., 1., 1., 1.]], [[ -9., 1., -9., 1.], [ 1., 1., -9., -9.], [ 1., 1., 1., 1.], [ 1., 1., 1., 1.]]]) introspection works easier with an example indexing with slice and boolean seems to flatten the boolean part, then we need only one newaxis ? seems to work this_par[:,idx].shape # = N.tile(pars, (idx.sum(), 1)).transpose() (2, 8) idx array([[ True, False, False, False], [ True, True, True, False], [ True, True, False, False], [ True, False, True, False]], dtype=bool) this_par[:,idx] = pars[:,N.newaxis] this_par array([[[-10., 1., 1., 1.], [-10., -10., -10., 1.], [-10., -10., 1., 1.], [-10., 1., -10., 1.]], [[ -9., 1., 1., 1.], [ -9., -9., -9., 1.], [ -9., -9., 1., 1.], [ -9., 1., -9., 1.]]]) Josef Brett ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
