Thank you Olsen,
My objective was to find out, how many values
are falling under different ranges. ie, find RMS < ,5 and then rms between .5
and .8 etc. If there is a speficic python way of handling mask and making
boolean operation with out any doubt, I was looking for that. The data I am
using has a mask and if I wanted to tell python do not consider the masked
values and masked grid points, ( a percentatge is calculated afterwards using
the number of grid points) while doing the calculation. I will try in detail
the example you send and see how python handles this.
with best regards,
Sudheer
***************************************************************
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:[email protected];[email protected]
Web- http://oppamthadathil.tripod.com
***************************************************************
--------------------------------------------
On Fri, 14/3/14, Brett Olsen <[email protected]> wrote:
Subject: Re: [Numpy-discussion] python array
To: "Discussion of Numerical Python" <[email protected]>
Date: Friday, 14 March, 2014, 2:07 AM
The difference appears
to be that the boolean selection pulls out all data values
<= 0.5 whether or not they are masked, and then carries
over the appropriate masks to the new array. So r2010 and
bt contain identical unmasked values but different numbers
of masked values. Because the initial fill value for your
masked values was a large negative number, in r2010 those
masked values are carried over. In bt, you've taken
the absolute value of the data array, so those fill values
are now positive and they are no longer carried over into
the indexed array.
Because the final arrays are still masked, you
are observing no difference in the statistical properties of
the arrays, only their sizes, because one contains many more
masked values than the other. I don't think this
should be a problem for your computations. If you're
concerned, you could always explicitly demask them before
your computations. See the example problem below.
~Brett
In [61]: import numpy as np
In [62]: import numpy.ma as ma
In [65]: a = np.arange(-8, 8).reshape((4,
4))
In [66]:
aOut[66]:array([[-8, -7, -6,
-5], [-4, -3, -2, -1],
[ 0, 1, 2, 3], [ 4, 5,
6, 7]])
In [68]: b = ma.masked_array(a, mask=a < 0)
In [69]: b
Out[69]:masked_array(data
= [[-- -- -- --] [-- -- --
--] [0 1 2 3] [4 5 6
7]], mask =
[[ True True True True] [ True True
True True] [False False False
False] [False False False False]],
fill_value = 999999)
In [70]: b.data
Out[70]:array([[-8, -7, -6,
-5], [-4, -3, -2, -1],
[ 0, 1, 2, 3], [ 4, 5,
6, 7]])
In [71]: c = abs(b)
In [72]: c[c <= 4].shapeOut[72]:
(9L,)
In [73]: b[b <= 4].shapeOut[73]:
(13L,)
In [74]: b[b <=
4]Out[74]:masked_array(data = [-- --
-- -- -- -- -- -- 0 1 2 3 4],
mask = [ True True True True
True True True True False False False
False False], fill_value =
999999)
In [75]: c[c <= 4]
Out[75]:masked_array(data = [-- -- -- -- 0 1
2 3 4], mask = [ True True
True True False False False False False],
fill_value = 999999)
On Thu, Mar 13, 2014
at 8:14 PM, Sudheer Joseph <[email protected]>
wrote:
Sorry,
The below solution I thoght working was not
working but was just giving array size.
--------------------------------------------
On Fri, 14/3/14, Sudheer Joseph
<[email protected]>
wrote:
Subject: Re: [Numpy-discussion] python array
To: "Discussion of Numerical Python" <[email protected]>
Date: Friday, 14 March, 2014, 1:09 AM
Thank you very much Nicolas and
Chris,
The
hint was helpful and from that I treid below steps ( a
crude
way I would say) and getting same result now
I have been using abs available by default and it is the
same with numpy.absolute( i checked).
nr= ((r2010>r2010.min()) & (r2010<r2010.max()))
nr[nr<.5].shape
Out[25]: (33868,)
anr=numpy.absolute(nr)
anr[anr<.5].shape
Out[27]: (33868,)
This way I used may have problem when mask used has
values
which can affect the min max operation.
So I would like to know if there is a standard formal (
python/numpy) way to handle masked array when they need
to
be subjected to boolean operations.
with best regards,
Sudheer
***************************************************************
Sudheer Joseph
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:[email protected];[email protected]
Web- http://oppamthadathil.tripod.com
***************************************************************
--------------------------------------------
On Thu, 13/3/14, Chris Barker - NOAA Federal <[email protected]>
wrote:
Subject: Re: [Numpy-discussion] python array
To: "Discussion of Numerical Python" <[email protected]>
Date: Thursday, 13 March, 2014, 11:53 PM
On Mar 13, 2014, at 9:39 AM, Nicolas
Rougier <[email protected]>
wrote:
>
> Seems to be related to the masked values:
Good hint -- a masked array keeps the "junk"
values in the
main array.
What "abs" are you using -- it may not be
mask-aware. (
you
want a
numpy abs anyway)
Also -- I'm not sure I know what happens with
Boolean
operators on
masked arrays when you use them to index. I'd
investigate
that.
(sorry, not at a machine I can play with now)
Chris
> print r2010[:3,:3]
> [[-- -- --]
> [-- -- --]
> [-- -- --]]
>
> print abs(r2010)[:3,:3]
> [[-- -- --]
> [-- -- --]
> [-- -- --]]
>
>
> print r2010[ r2010[:3,:3] <0 ]
> [-- -- -- -- -- -- -- -- --]
>
> print r2010[ abs(r2010)[:3,:3] < 0]
> []
>
> Nicolas
>
>
>
> On 13 Mar 2014, at 16:52, Sudheer Joseph <[email protected]>
wrote:
>
>> Dear experts,
>>
I am encountering a strange
behaviour of python data array as below. I have been
trying
to use the data from a netcdf file(attached herewith) to
do
certain calculation using below code. If I take absolute
value of the same array and look for values <.5 I
get a different value than the original array. But the
fact
is that this particular case do not have any negative
values
in the array( but there are other files where it can
have
negative values so the condition is put). I do not see
any
reason for getting different numbers for values <.5
in
case of bt and expected it to be same as that of r2010.
If
any one has a guess on what is behind this behaviour
please
help.
>>
>>
>> In [14]: from netCDF4 import Dataset as nc
>>
>> In [15]: nf=nc('r2010.nc')
>> In [16]: r2010=nf.variables['R2010'][:]
>> In [17]: bt=abs(r2010)
>> In [18]: bt[bt<=.5].shape
>> Out[18]: (2872,)
>> In [19]: r2010[r2010<.5].shape
>> Out[19]: (36738,)
>>
>>
>> bt.min()
>> Out[20]: 0.0027588337040836768
>>
>> In [21]: bt.max()
>> Out[21]: 3.5078965479057089
>> In [22]: r2010.max()
>> Out[22]: 3.5078965479057089
>> In [23]: r2010.min()
>> Out[23]: 0.0027588337040836768
>>
>>
>>
>>
***************************************************************
>> Sudheer Joseph
>> Indian National Centre for Ocean Information
Services
>> Ministry of Earth Sciences, Govt. of India
>> POST BOX NO: 21, IDA Jeedeemetla P.O.
>> Via Pragathi Nagar,Kukatpally, Hyderabad;
Pin:5000
55
>> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>> E-mail:[email protected];[email protected]
>> Web- http://oppamthadathil.tripod.com
>>
***************************************************************<r2010.nc>_______________________________________________
>> NumPy-Discussion mailing list
>> [email protected]
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> _______________________________________________
> NumPy-Discussion mailing list
> [email protected]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
-----Inline Attachment Follows-----
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
