[Numpy-discussion] Datetime overflow error, attn Stefan.
Hi Stefan, The windows buildbot throws the error = FAIL: test_creation_overflow (test_datetime.TestDateTime) -- Traceback (most recent call last): File C:\buildbot\numpy\b11\numpy-install25\Lib\site-packages\numpy\core\tests\test_datetime.py, line 68, in test_creation_overflow err_msg='Datetime conversion error for unit %s' % unit) File ..\numpy-install25\Lib\site-packages\numpy\testing\utils.py, line 313, in assert_equal AssertionError: Items are not equal: Datetime conversion error for unit ms ACTUAL: 567052800 DESIRED: 32268960 Because window's longs are always 32 bit, I think this is a good indication that somewhere a long is being used instead of an intp. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Another masked array question
Hello, Consider my masked arrays: I[28]: type basic.data['Air_Temp'] - type(basic.data['Air_Temp']) O[28]: numpy.ma.core.MaskedArray I[29]: basic.data['Air_Temp'] O[29]: masked_array(data = [-- -- -- ..., -- -- --], mask = [ True True True ..., True True True], fill_value = 99.) I[17]: basic.data['Air_Temp'].data = np.ones(len(basic.data['Air_Temp']))*30 --- AttributeErrorTraceback (most recent call last) 1 2 3 4 5 AttributeError: can't set attribute Why this assignment fails? I want to set each element in the original basic.data['Air_Temp'].data to another value. (Because the main instrument was forgotten to turn on for that day, and I am using a secondary measurement data for Air Temperature for my another calculation. However it fails. Although single assignment works: I[13]: basic.data['Air_Temp'].data[0] = 30 Shouldn't this be working like the regular NumPy arrays do? Thanks. -- Gökhan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Datetime overflow error, attn Stefan.
On Sat, May 8, 2010 at 6:44 PM, Charles R Harris charlesr.har...@gmail.comwrote: Hi Stefan, The windows buildbot throws the error = FAIL: test_creation_overflow (test_datetime.TestDateTime) -- Traceback (most recent call last): File C:\buildbot\numpy\b11\numpy-install25\Lib\site-packages\numpy\core\tests\test_datetime.py, line 68, in test_creation_overflow err_msg='Datetime conversion error for unit %s' % unit) File ..\numpy-install25\Lib\site-packages\numpy\testing\utils.py, line 313, in assert_equal AssertionError: Items are not equal: Datetime conversion error for unit ms ACTUAL: 567052800 DESIRED: 32268960 Because window's longs are always 32 bit, I think this is a good indication that somewhere a long is being used instead of an intp. Probably all references to long should be changed, it just isn't portable. And do you have any idea what is supposed to happen here: if (year = 0 || -1/4 == -1) I know this isn't your code, but you have been looking at it, so now you are responsible ;) Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Another masked array question
On Sat, May 8, 2010 at 7:52 PM, Gökhan Sever gokhanse...@gmail.com wrote: Hello, Consider my masked arrays: I[28]: type basic.data['Air_Temp'] - type(basic.data['Air_Temp']) O[28]: numpy.ma.core.MaskedArray I[29]: basic.data['Air_Temp'] O[29]: masked_array(data = [-- -- -- ..., -- -- --], mask = [ True True True ..., True True True], fill_value = 99.) I[17]: basic.data['Air_Temp'].data = np.ones(len(basic.data['Air_Temp']))*30 --- AttributeError Traceback (most recent call last) 1 2 3 4 5 AttributeError: can't set attribute Why this assignment fails? I want to set each element in the original basic.data['Air_Temp'].data to another value. (Because the main instrument was forgotten to turn on for that day, and I am using a secondary measurement data for Air Temperature for my another calculation. However it fails. Although single assignment works: I[13]: basic.data['Air_Temp'].data[0] = 30 Shouldn't this be working like the regular NumPy arrays do? Based on the traceback, I'd say it's because you're trying to replace the object pointed to by the .data attribute. Instead, try to just change the bits contained in .data: basic.data['Air_Temp'].data[:] = np.ones(len(basic.data['Air_Temp']))*30 Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Another masked array question
On Sat, May 8, 2010 at 9:16 PM, Ryan May rma...@gmail.com wrote: On Sat, May 8, 2010 at 7:52 PM, Gökhan Sever gokhanse...@gmail.com wrote: Hello, Consider my masked arrays: I[28]: type basic.data['Air_Temp'] - type(basic.data['Air_Temp']) O[28]: numpy.ma.core.MaskedArray I[29]: basic.data['Air_Temp'] O[29]: masked_array(data = [-- -- -- ..., -- -- --], mask = [ True True True ..., True True True], fill_value = 99.) I[17]: basic.data['Air_Temp'].data = np.ones(len(basic.data['Air_Temp']))*30 --- AttributeErrorTraceback (most recent call last) 1 2 3 4 5 AttributeError: can't set attribute Why this assignment fails? I want to set each element in the original basic.data['Air_Temp'].data to another value. (Because the main instrument was forgotten to turn on for that day, and I am using a secondary measurement data for Air Temperature for my another calculation. However it fails. Although single assignment works: I[13]: basic.data['Air_Temp'].data[0] = 30 Shouldn't this be working like the regular NumPy arrays do? Based on the traceback, I'd say it's because you're trying to replace the object pointed to by the .data attribute. Instead, try to just change the bits contained in .data: basic.data['Air_Temp'].data[:] = np.ones(len(basic.data['Air_Temp']))*30 Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Thanks for the pointer Ryan. Now it works as it is supposed to be. -- Gökhan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Another masked array question
On 05/08/2010 04:16 PM, Ryan May wrote: On Sat, May 8, 2010 at 7:52 PM, Gökhan Severgokhanse...@gmail.com wrote: Hello, Consider my masked arrays: I[28]: type basic.data['Air_Temp'] - type(basic.data['Air_Temp']) O[28]: numpy.ma.core.MaskedArray I[29]: basic.data['Air_Temp'] O[29]: masked_array(data = [-- -- -- ..., -- -- --], mask = [ True True True ..., True True True], fill_value = 99.) I[17]: basic.data['Air_Temp'].data = np.ones(len(basic.data['Air_Temp']))*30 --- AttributeErrorTraceback (most recent call last) 1 2 3 4 5 AttributeError: can't set attribute Why this assignment fails? I want to set each element in the original basic.data['Air_Temp'].data to another value. (Because the main instrument was forgotten to turn on for that day, and I am using a secondary measurement data for Air Temperature for my another calculation. However it fails. Although single assignment works: I[13]: basic.data['Air_Temp'].data[0] = 30 Shouldn't this be working like the regular NumPy arrays do? Based on the traceback, I'd say it's because you're trying to replace the object pointed to by the .data attribute. Instead, try to just change the bits contained in .data: basic.data['Air_Temp'].data[:] = np.ones(len(basic.data['Air_Temp']))*30 Also, you since you are setting all elements to a single value, you don't need to generate an array on the right-hand side. And, you don't need to manipulate .data directly--I think it is best to avoid doing so. Consider: In [1]:x = np.ma.array([1,2,3], mask=[True, True, True], dtype=float) In [2]:x Out[2]: masked_array(data = [-- -- --], mask = [ True True True], fill_value = 1e+20) In [3]:x[:] = 30 In [4]:x Out[4]: masked_array(data = [30.0 30.0 30.0], mask = [False False False], fill_value = 1e+20) In [5]:x[:] = np.ma.masked In [6]:x Out[6]: masked_array(data = [-- -- --], mask = [ True True True], fill_value = 1e+20) In [7]:x.data Out[7]:array([ 30., 30., 30.]) Eric Ryan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Another masked array question
On Sat, May 8, 2010 at 9:29 PM, Eric Firing efir...@hawaii.edu wrote:
On 05/08/2010 04:16 PM, Ryan May wrote:
On Sat, May 8, 2010 at 7:52 PM, Gökhan Severgokhanse...@gmail.com
wrote:
Hello,
Consider my masked arrays:
I[28]: type basic.data['Air_Temp']
- type(basic.data['Air_Temp'])
O[28]: numpy.ma.core.MaskedArray
I[29]: basic.data['Air_Temp']
O[29]:
masked_array(data = [-- -- -- ..., -- -- --],
mask = [ True True True ..., True True True],
fill_value = 99.)
I[17]: basic.data['Air_Temp'].data =
np.ones(len(basic.data['Air_Temp']))*30
---
AttributeErrorTraceback (most recent call
last)
1
2
3
4
5
AttributeError: can't set attribute
Why this assignment fails? I want to set each element in the original
basic.data['Air_Temp'].data to another value. (Because the main
instrument
was forgotten to turn on for that day, and I am using a secondary
measurement data for Air Temperature for my another calculation. However
it
fails. Although single assignment works:
I[13]: basic.data['Air_Temp'].data[0] = 30
Shouldn't this be working like the regular NumPy arrays do?
Based on the traceback, I'd say it's because you're trying to replace
the object pointed to by the .data attribute. Instead, try to just
change the bits contained in .data:
basic.data['Air_Temp'].data[:] = np.ones(len(basic.data['Air_Temp']))*30
Also, you since you are setting all elements to a single value, you
don't need to generate an array on the right-hand side. And, you don't
need to manipulate .data directly--I think it is best to avoid doing
so. Consider:
In [1]:x = np.ma.array([1,2,3], mask=[True, True, True], dtype=float)
In [2]:x
Out[2]:
masked_array(data = [-- -- --],
mask = [ True True True],
fill_value = 1e+20)
In [3]:x[:] = 30
In [4]:x
Out[4]:
masked_array(data = [30.0 30.0 30.0],
mask = [False False False],
fill_value = 1e+20)
In [5]:x[:] = np.ma.masked
In [6]:x
Out[6]:
masked_array(data = [-- -- --],
mask = [ True True True],
fill_value = 1e+20)
In [7]:x.data
Out[7]:array([ 30., 30., 30.])
Eric
Ryan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Good to see this :)
I[45]: x = np.ma.array([1,2,3], mask=[True, True, True], dtype=float)
I[46]: x
O[46]:
masked_array(data = [-- -- --],
mask = [ True True True],
fill_value = 1e+20)
I[47]: x.data[:] = 25
I[48]: x
O[48]:
masked_array(data = [-- -- --],
mask = [ True True True],
fill_value = 1e+20)
I[49]: x[:] = 25
I[50]: x
O[50]:
masked_array(data = [25.0 25.0 25.0],
mask = [False False False],
fill_value = 1e+20)
I was also updating mask values after updating data attribute. Now setting
the masked array itself to a number automatically flips the masks for me
which is very useful. I check if a valid temperature exists, otherwise
assign my calculation to another missing value.
--
Gökhan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] pareto docstring
The docstring for np.pareto says:
This is a simplified version of the Generalized Pareto distribution
(available in SciPy), with the scale set to one and the location set to
zero. Most authors default the location to one.
and also:
The probability density for the Pareto distribution is
.. math:: p(x) = \frac{am^a}{x^{a+1}}
where :math:`a` is the shape and :math:`m` the location
These two statements seem to be in contradiction. I think what was
meant is that m is the scale, rather than the location. For if m were
equal to zero, as the first portion of the docstring states, then the
entire pdf would be zero for all shapes a0.
Also, I'm not quite understanding how the stated pdf is actually the
same as the pdf for the generalized pareto with the scale=1 and
location=0. By the wikipedia definition of the generalized Pareto
distribution, if we take \sigma=1 (scale equal to one) and \mu=0
(location equal to zero), then we get:
(1 + a x)^(-1/a - 1)
which is normalized over $x \in (0, \infty)$. If we compare this to
the distribution stated in the docstring (with m=1)
a x^{-a-1}
we see that it is normalized over $x \in (1, \infty)$. And indeed,
the distribution requires x scale = 1.
If we integrate the generalized Pareto (with scale=1, location=0) over
$x \in (1, \infty)$ then we have to re-normalize. So should the
docstring say:
This is a simplified version of the Generalized Pareto distribution
(available in Scipy), with the scale set to one, the location set to zero,
and the distribution re-normalized over the range (1, \infty). Most
authors default the location to one.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
