2008/9/19 David Cournapeau [EMAIL PROTECTED]:
Anne Archibald wrote:
That was in amax/amin. Pretty much every other function that does
comparisons needs to be fixed to work with nans. In some cases it's
not even clear how: where should a sort put the nans in an array?
The problem is more on
Anne Archibald wrote:
Well, for example, you might ask that all the non-nan elements be in
order, even if you don't specify where the nan goes.
Ah, there are two problems, then:
- sort
- how median use sort.
For sort, I don't know how sort speed would be influenced by treating
nan.
On Friday 19 September 2008 03:11:05 David Cournapeau wrote:
Hm, I am always puzzled when I think about nan handling :) It always
seem there is not good answer.
Which is why we have masked arrays, of course ;)
___
Numpy-discussion mailing list
2008/9/19 Pierre GM [EMAIL PROTECTED]:
On Friday 19 September 2008 03:11:05 David Cournapeau wrote:
Hm, I am always puzzled when I think about nan handling :) It always
seem there is not good answer.
Which is why we have masked arrays, of course ;)
I think the numpy attitude to nans should
2008/9/19 Travis E. Oliphant [EMAIL PROTECTED]:
#---
def view(self, dtype=None, type=None):
if dtype is None:
return ndarray.view(self, type)
elif type is None:
try:
if issubclass(dtype, ndarray):
Robert Kern wrote:
On Thu, Sep 18, 2008 at 06:01, Robert Cimrman [EMAIL PROTECTED] wrote:
Hi Robert,
Robert Kern wrote:
On Mon, Sep 15, 2008 at 11:13, Arnar Flatberg [EMAIL PROTECTED] wrote:
That would make me an extremely happy user, I've been looking for this for
years!
I can't imagine
2008/9/19 Anne Archibald [EMAIL PROTECTED]:
I think the numpy attitude to nans should be that they are unexpected
bogus values that signify that something went wrong with the
calculation somewhere. They can be left in place for most operations,
but any operation that depends on the value
Ondrej Certik wrote:
On Thu, Sep 18, 2008 at 4:12 PM, Ryan May [EMAIL PROTECTED] wrote:
Ondrej Certik wrote:
On Thu, Sep 18, 2008 at 1:01 PM, Robert Cimrman [EMAIL PROTECTED] wrote:
It requires Cython and a C compiler to build. I'm still debating
myself about the desired workflow for using
On Friday 19 September 2008 04:13:39 Stéfan van der Walt wrote:
+1 and another +1 to your karma for requesting peer review. Let me
know if you need me to whip up a couple of tests for verifying the
different usage cases.
That'd be lovely. I'm a bit swamped with tricky issues in mrecords and
Stéfan van der Walt wrote:
I agree completely.
Me too, but I am extremely biased toward nan is always bogus by my own
usage of numpy/scipy (I never use NaN as missing value, and nan is
always caused by divide by 0 and co).
I like that sort raise an exception by default with NaN: it breaks the
On Friday 19 September 2008 04:31:38 David Cournapeau wrote:
Pierre GM wrote:
That said, numpy.nanmin, numpy.nansum... don't come with the heavy
machinery of numpy.ma, and are therefore faster.
I'm really going to have to learn C.
FWIW, nanmean/nanmean/etc... are written in python,
I
Robert Kern robert.kern at gmail.com writes:
On Thu, Sep 18, 2008 at 16:55, Paul Moore pf_moore at yahoo.co.uk wrote:
I want to generate a series of random samples, to do simulations based
on them. Essentially, I want to be able to produce a SAMPLESIZE * N
matrix, where each row of N values
2008/9/19 David Cournapeau [EMAIL PROTECTED]:
Stéfan van der Walt wrote:
I agree completely.
Me too, but I am extremely biased toward nan is always bogus by my own
usage of numpy/scipy (I never use NaN as missing value, and nan is
always caused by divide by 0 and co).
So am I. In all my
On Friday 19 September 2008 05:08:20 Paul Moore wrote:
Robert Kern robert.kern at gmail.com writes:
On Thu, Sep 18, 2008 at 16:55, Paul Moore pf_moore at yahoo.co.uk
wrote:
I want to generate a series of random samples, to do simulations based
on them. Essentially, I want to be able to
On Fri, Sep 19, 2008 at 03:33, Robert Cimrman [EMAIL PROTECTED] wrote:
I have Python 2.4.4
in pyconfig.h
#define HAVE_LONG_LONG 1
in pyport.h:
#ifdef HAVE_LONG_LONG
#ifndef PY_LONG_LONG
#define PY_LONG_LONG long long
#endif
#endif /* HAVE_LONG_LONG */
so it seems compatible with
On Fri, Sep 19, 2008 at 10:37 AM, Robert Cimrman [EMAIL PROTECTED] wrote:
Ondrej Certik wrote:
On Thu, Sep 18, 2008 at 4:12 PM, Ryan May [EMAIL PROTECTED] wrote:
Ondrej Certik wrote:
On Thu, Sep 18, 2008 at 1:01 PM, Robert Cimrman [EMAIL PROTECTED] wrote:
It requires Cython and a C compiler
On Wed, Sep 17, 2008 at 18:29, Robert Kern [EMAIL PROTECTED] wrote:
On Wed, Sep 17, 2008 at 18:09, Ondrej Certik [EMAIL PROTECTED] wrote:
This is what I am getting:
$ ./kernprof.py -l pystone.py
Wrote profile results to pystone.py.lprof
$ ./view_line_prof.py pystone.py.lprof
Timer unit:
Stéfan van der Walt wrote:
So am I. In all my use cases, NaNs indicate trouble.
Yes, so I would like to have the opinion of people with other usage
than ours.
Because we have x.max() silently ignoring NaNs, which causes a lot of
head-scratching, swearing and failed experiments.
But cannot
2008/9/19 Paul Moore [EMAIL PROTECTED]:
Robert Kern robert.kern at gmail.com writes:
On Thu, Sep 18, 2008 at 16:55, Paul Moore pf_moore at yahoo.co.uk wrote:
I want to generate a series of random samples, to do simulations based
on them. Essentially, I want to be able to produce a SAMPLESIZE
David Cournapeau david at ar.media.kyoto-u.ac.jp writes:
You can use nanmean (from scipy.stats):
I rejoiced when I saw this answer, because it looks like a function I can just
drop in and it works. Unfortunately, nanmedian seems to be quite a bit slower
than just using lists (ignoring nan
On Friday 19 September 2008 05:51:55 Peter Saffrey wrote:
I would like to try the masked array approach, but the Ubuntu packages for
scipy and matplotlib depend on numpy. Does anybody know whether I can
naively do sudo python setup.py install on a more modern numpy without
disturbing scipy
David Cournapeau david at ar.media.kyoto-u.ac.jp writes:
It may be that nanmedian is slow. But I would sincerly be surprised if
it were slower than python list, except for some pathological cases, or
maybe a bug in nanmedian. What do your data look like ? (size, number of
nan, etc...)
I've
Pierre GM pgmdevlist at gmail.com writes:
I think there were some changes on the C side of numpy between 1.0 and 1.1,
you may have to recompile scipy and matplotlib from sources. What versions
are you using for those 2 packages ?
$ dpkg -l | grep scipy
ii python-scipy
Peter Saffrey wrote:
Pierre GM pgmdevlist at gmail.com writes:
I think there were some changes on the C side of numpy between 1.0 and 1.1,
you may have to recompile scipy and matplotlib from sources. What versions
are you using for those 2 packages ?
$ dpkg -l | grep scipy
ii
Peter Saffrey wrote:
I've posted my test code below, which gives me the results:
$ ./arrayspeed3.py
list build time: 0.01
list median time: 0.01
array nanmedian time: 0.36
I must have done something wrong to hobble nanmedian in this way... I'm quite
new to numpy, so feel free to point
Robert Kern wrote:
Ah, found it. T_LONGLONG is a #define from structmember.h which is
used to describe the types of attributes. Apparently, this was not
added until Python 2.5. That particular member didn't actually need to
be long long, so I've fixed that.
Great, I will try it after it
2008/9/19 David Cournapeau [EMAIL PROTECTED]:
But cannot this be fixed at the python level of the max function ? I
Why shouldn't we have nanmin-like behaviour for the C min itself?
I'd rather have a specialised function to deal with the rare kinds of
datasets where NaNs are guaranteed never to
2008/9/19 mark [EMAIL PROTECTED]:
I need to multiply items in a list and need a list back. Which one of
the four options is best (I thought in Python there was only one way
to do something???)
With the emphasis on preferably and obvious :)
There should be one-- and preferably only one
On Fri, Sep 19, 2008 at 3:09 PM, Stéfan van der Walt [EMAIL PROTECTED]wrote:
2008/9/19 mark [EMAIL PROTECTED]:
I need to multiply items in a list and need a list back. Which one of
the four options is best (I thought in Python there was only one way
to do something???)
With the emphasis
On Fri, Sep 19, 2008 at 4:09 PM, lorenzo [EMAIL PROTECTED] wrote:
On Fri, Sep 19, 2008 at 2:50 PM, Arnar Flatberg [EMAIL PROTECTED]wrote:
I think
[x*y for x in a for y in b]
feels pythonic, however it has a surprisingly lousy performance.
This returns a len(x)*len(y) long list, which
Hi Arnar,
Your two commands below aren't doing the same thing - one is doing
a[i]*b[i] and the other is doing a[i]*b[j] for all i and j. As the
second is harder, it takes longer.
Cheers,
David
On Fri, 2008-09-19 at 09:08 -0500, [EMAIL PROTECTED]
wrote:
I think
[x*y for x in a for y in b]
Rick White rlw at stsci.edu writes:
It seems like numpy.random.permutation is pretty suboptimal in its
speed. Here's a Python 1-liner that does the same thing (I think)
but is a lot faster:
a = 1+numpy.random.rand(M).argsort()[0:N-1]
This still has the the problem that it generates
On 9/19/2008 11:09 AM Stefan Van der Walt apparently wrote:
Masked arrays. Using NaN's for missing values is dangerous. You may
do some operation, which generates invalid results, and then you have
a mixed bag of missing and invalid values.
That rather evades my full question, I think?
On Friday 19 September 2008 11:36:17 Alan G Isaac wrote:
On 9/19/2008 11:09 AM Stefan Van der Walt apparently wrote:
Masked arrays. Using NaN's for missing values is dangerous. You may
do some operation, which generates invalid results, and then you have
a mixed bag of missing and invalid
Hi all,
Attached is a newer version of my patch that adds new mgrid / ogrid
functionality for working with arrays in addition to slices. In fact, I
have attached two versions of the patch: index_tricks.patch, that is
just the last version of the patch I sent, and index_tricks.new.patch,
that has
Alan G Isaac aisaac at american.edu writes:
Recently I needed to fill a 2d array with values
from computations that could go wrong.
I created an array of NaN and then replaced
the elements where the computation produced
a useful value. I then applied ``nanmax``,
to get the maximum of the
On Friday 19 September 2008 12:02:08 Peter Saffrey wrote:
Alan G Isaac aisaac at american.edu writes:
Recently I needed to fill a 2d array with values
from computations that could go wrong.
Should I take the earlier advice and switch to masked arrays?
Peter
Yes. As you've noticed, you
On 9/19/2008 11:46 AM Pierre GM apparently wrote:
You can't compare NaNs to anything. How do you know this np.miss is a masked
value, when np.sqrt(-1.) is NaN ?
I thought you could use ``is``.
E.g.,
np.nan == np.nan
False
np.nan is np.nan
True
Alan
On 9/19/2008 11:46 AM Pierre GM apparently wrote:
No, but you may do the opposite: just start with an array completely masked,
and unmasked it as you need:
Very useful example.
I did not understand this possibility.
Alan
___
Numpy-discussion
On Fri, Sep 19, 2008 at 1:11 AM, David Cournapeau
[EMAIL PROTECTED] wrote:
Anne Archibald wrote:
Well, for example, you might ask that all the non-nan elements be in
order, even if you don't specify where the nan goes.
Ah, there are two problems, then:
- sort
- how median use
On 9/19/2008 12:02 PM Peter Saffrey apparently wrote:
a = array([1,2,nan])
nan in a
False
Huh. I'm inclined to call this a bug,
since normal Python behavior is that
``in`` should check for identity::
xl = [1.,np.nan]
np.nan in xl
True
Alan
Might someone explain this to me?
x = [1.,np.nan]
np.nan in x
True
np.nan in np.array(x)
False
np.nan in np.array(x).tolist()
False
np.nan is float(np.nan)
True
Thank you,
Alan Isaac
___
You, know, float are inmutable objects, and then 'float(f)' just
returns a new reference to 'f' is 'f' is (exactly) of type 'float'
In [1]: f = 1.234
In [2]: f is float(f)
Out[2]: True
I do not remember right now the implementations of comparisons in core
Python, but I believe the 'in' operator
On Fri, Sep 19, 2008 at 1:59 PM, Alan G Isaac [EMAIL PROTECTED] wrote:
Might someone explain this to me?
x = [1.,np.nan]
np.nan in x
True
np.nan in np.array(x)
False
np.nan in np.array(x).tolist()
False
np.nan is float(np.nan)
True
Alan G Isaac wrote:
Might someone explain this to me?
x = [1.,np.nan]
np.nan in x
True
np.nan in np.array(x)
False
np.nan in np.array(x).tolist()
False
np.nan is float(np.nan)
True
not quite -- but I do know that is is tricky -- it
On Fri, Sep 19, 2008 at 11:34, Alan G Isaac [EMAIL PROTECTED] wrote:
On 9/19/2008 12:02 PM Peter Saffrey apparently wrote:
a = array([1,2,nan])
nan in a
False
Huh. I'm inclined to call this a bug,
since normal Python behavior is that
``in`` should check for identity::
xl =
On Fri, Sep 19, 2008 at 12:59, Brad Malone [EMAIL PROTECTED] wrote:
Hi, I was wondering if someone could englighten me on what the geometrical
significance of numpy.mgrid is. I can play around with it and see trends in
the sizes and number of arrays, but why does it give the output that it
On Sep 19, 2008, at 7:52 PM, Christopher Barker wrote:
I don't know the interning rules, but I do know that you should never
count on them, then may not be consistent between implementations, or
even different runs.
There are a few things that Python-the-language guarantees are singleton
On Fri, Sep 19, 2008 at 07:00, Robert Cimrman [EMAIL PROTECTED] wrote:
Robert Kern wrote:
Ah, found it. T_LONGLONG is a #define from structmember.h which is
used to describe the types of attributes. Apparently, this was not
added until Python 2.5. That particular member didn't actually need to
Thanks for the response Robert.
So, at least in this case, the results of mgrid (or indices) only provides
information about the spacing of the grid and not on the absolute value of
the point coordinates?
In your example, is there a way to see within your x[ix], y[iy], and z[iz]
matrices the
On Fri, Sep 19, 2008 at 14:13, Brad Malone [EMAIL PROTECTED] wrote:
Thanks for the response Robert.
So, at least in this case, the results of mgrid (or indices) only provides
information about the spacing of the grid and not on the absolute value of
the point coordinates?
No, they give
Andrew Dalke wrote:
There are a few things that Python-the-language guarantees are singleton
objects which can be compared correctly with is. Those are:
True, False, None
The empty tuple () and all interned strings are also guaranteed to be
singletons. String interning is used to
On 9/19/2008 11:46 AM Pierre GM apparently wrote:
a.mask=True
This is great, but is apparently
new behavior as of NumPy 1.2?
Alan Isaac
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
On Friday 19 September 2008 16:28:34 Alan G Isaac wrote:
On 9/19/2008 11:46 AM Pierre GM apparently wrote:
a.mask=True
This is great, but is apparently
new behavior as of NumPy 1.2?
I'm not sure, sorry. Another way is
ma.array(np.empty(yourshape,yourdtype), mask=True)
which should work
On 9/19/2008 4:54 PM Pierre GM apparently wrote:
Another way is
ma.array(np.empty(yourshape,yourdtype), mask=True)
which should work with earlier versions.
Seems like ``mask`` would be a natural
keyword for ``ma.empty``?
Thanks,
Alan Isaac
___
On Friday 19 September 2008 17:25:53 Alan G Isaac wrote:
On 9/19/2008 4:54 PM Pierre GM apparently wrote:
Another way is
ma.array(np.empty(yourshape,yourdtype), mask=True)
which should work with earlier versions.
Seems like ``mask`` would be a natural
keyword for ``ma.empty``?
Not a bad
Hi,
What am I doing wrong here? The reshape doesnt take.
% cat test1.py
import numpy as np
a = np.uint8([39, 39, 231, 239, 39, 231, 39, 39, 231,
39, 39, 231, 239, 39, 231, 39, 39, 231,
39, 39, 231, 239, 39, 231, 39, 39, 231,
39, 39, 231, 239,
paul taney wrote:
Hi,
What am I doing wrong here? The reshape doesnt take.
Reshape does not act in place, it returns either a new view or a copy.
To reshape in place, you can assign to the shape attribute:
In [13]:a = np.arange(10)
In [14]:a.shape = (2,5)
In [15]:a
Out[15]:
array([[0, 1,
On Friday 19 September 2008 20:47:12 paul taney wrote:
Hi,
What am I doing wrong here? The reshape doesnt take.
help(reshape)
a.reshape(shape, order='C')
Returns an array containing the data of a, but with a new shape.
Refer to `numpy.reshape` for full documentation.
You see that
On Sep 19, 2008, at 10:04 PM, Christian Heimes wrote:
Andrew Dalke wrote:
There are a few things that Python-the-language guarantees are
singleton
objects which can be compared correctly with is.
The empty tuple () and all interned strings are also guaranteed to be
singletons.
Where's
Stéfan van der Walt wrote:
Why shouldn't we have nanmin-like behaviour for the C min itself?
Ah, I was not arguing we should not do it in C, but rather we did not
have to do in C. The current behavior for nan with functions relying on
ordering is broken; if someone prefer fixing it in C,
On Fri, Sep 19, 2008 at 22:25, David Cournapeau
[EMAIL PROTECTED] wrote:
Stéfan van der Walt wrote:
Why shouldn't we have nanmin-like behaviour for the C min itself?
Ah, I was not arguing we should not do it in C, but rather we did not
have to do in C. The current behavior for nan with
Robert Kern wrote:
On Fri, Sep 19, 2008 at 22:25, David Cournapeau
[EMAIL PROTECTED] wrote:
How, exactly? ndarray.min() is the where the implementation is.
Ah, I keep forgetting those are implemented in the array object, sorry
for that. Now I understand Stefan point. Do I understand
2008/9/19 David Cournapeau [EMAIL PROTECTED]:
I guess my formulation was poor: I never use NaN as missing values
because I never use missing values, which is why I wanted the opinion of
people who use NaN in a different manner (because I don't have a good
idea on how those people would like
To All,
I have now been able to generate a .pyd file from a FORTRAN file that I am
trying to interface with python. I was able to execute this with an additional
insight into how f2py operates. It seems as though the documentation requires
an upgrade, since there appears to be missing
Anne Archibald wrote:
I, on the other hand, was making specifically that suggestion: users
should not use nans to indicate missing values. Users should use
masked arrays to indicate missing values.
I agree it is the nicest solution in theory, but I think it is
impractical (as mentioned by
66 matches
Mail list logo
