Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 5:35 PM, Michael Gilbert wrote: > On Wed, 8 Sep 2010 15:44:02 -0400, Michael Gilbert wrote: >> On Wed, Sep 8, 2010 at 12:23 PM, Charles R Harris wrote: >> > >> > >> > On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert >> > wrote: >> >> >> >> On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: >> >> > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert >> >> > > >> > > wrote: >> >> > >> >> > > Hi, >> >> > > >> >> > > Are there any plans to add support for decimal floating point >> >> > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard >> >> > > [0], in numpy? >> >> > > >> >> > > >> >> > Not at the moment. There is currently no hardware or C support and >> >> > adding >> >> > new types to numpy isn't trivial. You can get some limited Decimal >> >> > functionality by using python classes and object arrays, for instance >> >> > the >> >> > Decimal class in the python decimal module, but the performance isn't >> >> > great. >> >> > >> >> > What is your particular interest in decimal support? >> >> >> >> Primarily avoiding catastrophic cancellation when subtracting >> >> large values. I was planning to use the decimal class, but was >> >> curious whether support for the IEEE standard was coming any time soon. >> >> >> > >> > If you just need more precision, mpmath has better performance than the >> > Decimal class. Also, it might be possible to avoid the loss of precision by >> > changing the computation, but I don't know the details of what you are >> > doing. >> >> Just wanted to say that numpy object arrays + decimal solved all of my >> problems, which were all caused by the disconnect between decimal and >> binary representation of floating point numbers. It would be really >> nice to have IEEE decimal floating point since python's decimal module >> is a bit slow, but I can see thats going to be a very long process. > > FYI, gcc 4.2 introduced initial support for decimal floats. However, > various macros like DEC32_MAX, and printf support for dd, df, and dl is > not yet implemented making it rather hard to work with. A reference would probably be useful: http://gcc.gnu.org/onlinedocs/gcc-4.2.4/gcc/Decimal-Float.html As with all gcc documentation, its rather terse :/ Best wishes, Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, 8 Sep 2010 15:44:02 -0400, Michael Gilbert wrote: > On Wed, Sep 8, 2010 at 12:23 PM, Charles R Harris wrote: > > > > > > On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert > > wrote: > >> > >> On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: > >> > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert > >> > >> > > wrote: > >> > > >> > > Hi, > >> > > > >> > > Are there any plans to add support for decimal floating point > >> > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard > >> > > [0], in numpy? > >> > > > >> > > > >> > Not at the moment. There is currently no hardware or C support and > >> > adding > >> > new types to numpy isn't trivial. You can get some limited Decimal > >> > functionality by using python classes and object arrays, for instance > >> > the > >> > Decimal class in the python decimal module, but the performance isn't > >> > great. > >> > > >> > What is your particular interest in decimal support? > >> > >> Primarily avoiding catastrophic cancellation when subtracting > >> large values. I was planning to use the decimal class, but was > >> curious whether support for the IEEE standard was coming any time soon. > >> > > > > If you just need more precision, mpmath has better performance than the > > Decimal class. Also, it might be possible to avoid the loss of precision by > > changing the computation, but I don't know the details of what you are > > doing. > > Just wanted to say that numpy object arrays + decimal solved all of my > problems, which were all caused by the disconnect between decimal and > binary representation of floating point numbers. It would be really > nice to have IEEE decimal floating point since python's decimal module > is a bit slow, but I can see thats going to be a very long process. FYI, gcc 4.2 introduced initial support for decimal floats. However, various macros like DEC32_MAX, and printf support for dd, df, and dl is not yet implemented making it rather hard to work with. Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On 8 September 2010 16:33, Robert Kern wrote:
> On Wed, Sep 8, 2010 at 15:10, Michael Gilbert
> wrote:
>> On Wed, 8 Sep 2010 15:04:17 -0500, Robert Kern wrote:
>>> On Wed, Sep 8, 2010 at 14:44, Michael Gilbert
>>> wrote:
>
>>> > Just wanted to say that numpy object arrays + decimal solved all of my
>>> > problems, which were all caused by the disconnect between decimal and
>>> > binary representation of floating point numbers.
>>>
>>> Are you sure? Unless if I'm failing to think through this properly,
>>> catastrophic cancellation for large numbers is an intrinsic property
>>> of fixed-precision floating point regardless of the base. decimal and
>>> mpmath both help with that problem because they have arbitrary
>>> precision.
>>
>> Here is an example:
>>
>> >>> 0.3/3.0 - 0.1
>> -1.3877787807814457e-17
>>
>> >>> mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
>> mpf('-1.3877787807814457e-17')
>>
>> >>> decimal.Decimal( '0.3' )/decimal.Decimal( '3.0' ) - decimal.Decimal (
>> '0.1' )
>> Decimal("0.0")
>>
>> Decimal solves the problem; whereas mpmath doesn't.
>
> Okay, that's not an example of catastrophic cancellation, just a
> representation issue.
Indeed - and as I understand it the motivation for decimal numbers is
not that they suffer less from roundoff than binary, but because the
round-off they suffer from is better suited to (for example) financial
applications, where representing exact decimal numbers can be
important.
If your problem is the fact of roundoff error, try using as your test
case, taking the square root of two and squaring it again. This will
suffer from the same sort of roundoff problems in decimal as binary.
Anne
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> -- Umberto Eco
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 15:10, Michael Gilbert
wrote:
> On Wed, 8 Sep 2010 15:04:17 -0500, Robert Kern wrote:
>> On Wed, Sep 8, 2010 at 14:44, Michael Gilbert
>> wrote:
>> > Just wanted to say that numpy object arrays + decimal solved all of my
>> > problems, which were all caused by the disconnect between decimal and
>> > binary representation of floating point numbers.
>>
>> Are you sure? Unless if I'm failing to think through this properly,
>> catastrophic cancellation for large numbers is an intrinsic property
>> of fixed-precision floating point regardless of the base. decimal and
>> mpmath both help with that problem because they have arbitrary
>> precision.
>
> Here is an example:
>
> >>> 0.3/3.0 - 0.1
> -1.3877787807814457e-17
>
> >>> mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
> mpf('-1.3877787807814457e-17')
>
> >>> decimal.Decimal( '0.3' )/decimal.Decimal( '3.0' ) - decimal.Decimal (
> '0.1' )
> Decimal("0.0")
>
> Decimal solves the problem; whereas mpmath doesn't.
Okay, that's not an example of catastrophic cancellation, just a
representation issue.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
-- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] inversion of large matrices
Wow, this is great! Thanks for all the great questions.
Sebastian Walter gmail.com> writes:
> is it really the covariance matrix you want to invert? Or do you want
> to compute something like
> x^T C^{-1} x,
> where x is an array of size N and C an array of size (N,N)?
Yes, this is what I am computing. I am computing the pdf of a very high-
dimensional multivariate normal. Is there a specialized method to compute this?
> It would also be interesting to know how the covariance matrix gets computed
> and what its condition number is, at least approximately.
Chuck and Sebastian are on to me. I should look into computing this using the
small covariance matrix. I suppose I could even postpone evaluation of this
covariance matrix by passing in the data matrix instead of the covariance
matrix. What do you guys think?
As for the condition number, the matrix will, initially, be ill-conditioned
because the dimension of the data will far outnumber the number of data points.
However, I am writing this code on the way to writing code that fits a mixture
of Gaussians but can use any number of covariance regularization techniques to
"fix" the covariance matrices.
Basically, I am writing code that does this:
http://www.cs.colostate.edu/~dane/papers/ANNIE2010.pdf. I am guessing you math
and stats guys might shake your head at my clumsy, computer science ways... :)
However, in addition to the two, simple regularization techniques described in
that paper, I also want to be able to do things like mixture of factor
analyzers
and mixture of principal component analyzers and so on by simply supplying
different regularization techniques.
FYI, I am currently adding this to PyMix. Does this seem like a good package
to
extend?
> On Wed, Sep 1, 2010 at 1:58 AM, Charles R Harris
> gmail.com> wrote:
> > On Tue, Aug 31, 2010 at 4:52 PM, Dan Elliott gmail.com>
> > wrote:
> >> David Warde-Farley cs.toronto.edu> writes:
> >> > On 2010-08-30, at 10:36 PM, Charles R Harris wrote:
> >> > I think he means that if he needs both the determinant and to solve the
> >> > system, it might be more efficient to do
> >> > the SVD, obtain the determinant from the diagonal values, and obtain the
> >> > solution by multiplying by U D^-1 V^T?
> >> Thank you, that is what I meant. Poorly worded on my part.
> >>
> >> In particular, I am writing code to invert a very large covariance matrix.
> >> I
> >> think David has some good information in another post in this thread.
> >>
> >
> > Where did the covariance array come from? It may be the case that you can
> > use a much smaller one, for instance in PCA of images.
> >
> > Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] clip() with None as argument
On Wed, Sep 8, 2010 at 14:42, Chris Ball wrote: > Robert Kern gmail.com> writes: > >> >> On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt >> gmail.com> wrote: >> > Ah, no need to answer, I do this myself: >> > >> > Friedrich, would you please use numpy.inf and -numpy.inf. >> >> But if you have an integer array, you will run into the same problem. >> The result will be upcast to float. I think we would accept a patch >> that interprets None to be the appropriate extreme bound given the >> input datatype. This will be tricky, though, since it must be done in >> C (PyArray_Clip in numpy/core/src/multiarray/calculation.c). > > I've been a bit confused about what numpy's clip is > supposed to support. The documentation > (e.g. http://docs.scipy.org/doc/numpy/reference/generated/numpy.clip.html) > doesn't make any mention of supporting only one of min or max, but I > remember a message some time back from Travis Oliphant that seemed to > suggest it does > (http://thread.gmane.org/gmane.comp.python.numeric.general/17844/focus=17877). > > Right now, I'm surprised by two aspect's of clip's behavior, both > demonstrated below... > > $ python > Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) > [GCC 4.4.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. import numpy numpy.__version__ > '2.0.0.dev8698' a = numpy.array([1,2,3,4,5]) a.clip(2,None) > array([2, 2, 2, 2, 2], dtype=object) > > I'm not sure why the returned array has a dtype of object The usual type-matching semantics. a.clip() tries to find a common type that fits `a`, 2 and None. Since None only fits in an object dtype, that's what you get. > (although that can be > avoided by using clip's "out" argument), and also > why the values are not like this: > numpy.maximum(a,2) > array([2, 2, 3, 4, 5]) clip(a, low, high) boils down to this in the C code itself (or at least the code path that it goes down when the input array needs to be upcast): maximum(minimum(a, high), low) Since all integer objects compare > None based on the default Python 2.x implementation of comparison across types, you would just get the low value out of that. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, 8 Sep 2010 22:20:30 +0200, Sandro Tosi wrote:
> On Wed, Sep 8, 2010 at 22:10, Michael Gilbert
> wrote:
> > Here is an example:
> >
> > >>> 0.3/3.0 - 0.1
> > -1.3877787807814457e-17
> >
> > >>> mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
> > mpf('-1.3877787807814457e-17')
> >
> > >>> decimal.Decimal( '0.3' )/decimal.Decimal( '3.0' ) - decimal.Decimal (
> > '0.1' )
> > Decimal("0.0")
> >
> > Decimal solves the problem; whereas mpmath doesn't.
>
> you can change mpmath precision up to an arbitrary high value:
>
> In [4]: mpmath.mp.prec = 100
>
> In [5]: mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
> Out[5]: mpf('0.0')
Thanks. I already knew that. I tried prec = 500, which did of course
get the error down quite a bit to begin with, but even so, it continued
to grow and grow. So decimal wins out.
Mike
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 22:10, Michael Gilbert
wrote:
> Here is an example:
>
> >>> 0.3/3.0 - 0.1
> -1.3877787807814457e-17
>
> >>> mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
> mpf('-1.3877787807814457e-17')
>
> >>> decimal.Decimal( '0.3' )/decimal.Decimal( '3.0' ) - decimal.Decimal (
> '0.1' )
> Decimal("0.0")
>
> Decimal solves the problem; whereas mpmath doesn't.
you can change mpmath precision up to an arbitrary high value:
In [4]: mpmath.mp.prec = 100
In [5]: mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
Out[5]: mpf('0.0')
Regards.
--
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, 8 Sep 2010 15:04:17 -0500, Robert Kern wrote:
> On Wed, Sep 8, 2010 at 14:44, Michael Gilbert
> wrote:
> > On Wed, Sep 8, 2010 at 12:23 PM, Charles R Harris wrote:
> >>
> >>
> >> On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert
> >> wrote:
> >>>
> >>> On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote:
> >>> > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert
> >>> > >>> > > wrote:
> >>> >
> >>> > > Hi,
> >>> > >
> >>> > > Are there any plans to add support for decimal floating point
> >>> > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard
> >>> > > [0], in numpy?
> >>> > >
> >>> > >
> >>> > Not at the moment. There is currently no hardware or C support and
> >>> > adding
> >>> > new types to numpy isn't trivial. You can get some limited Decimal
> >>> > functionality by using python classes and object arrays, for instance
> >>> > the
> >>> > Decimal class in the python decimal module, but the performance isn't
> >>> > great.
> >>> >
> >>> > What is your particular interest in decimal support?
> >>>
> >>> Primarily avoiding catastrophic cancellation when subtracting
> >>> large values. I was planning to use the decimal class, but was
> >>> curious whether support for the IEEE standard was coming any time soon.
> >>>
> >>
> >> If you just need more precision, mpmath has better performance than the
> >> Decimal class. Also, it might be possible to avoid the loss of precision by
> >> changing the computation, but I don't know the details of what you are
> >> doing.
> >
> > Just wanted to say that numpy object arrays + decimal solved all of my
> > problems, which were all caused by the disconnect between decimal and
> > binary representation of floating point numbers.
>
> Are you sure? Unless if I'm failing to think through this properly,
> catastrophic cancellation for large numbers is an intrinsic property
> of fixed-precision floating point regardless of the base. decimal and
> mpmath both help with that problem because they have arbitrary
> precision.
Here is an example:
>>> 0.3/3.0 - 0.1
-1.3877787807814457e-17
>>> mpmath.mpf( '0.3' )/mpmath.mpf( '3.0' ) - mpmath.mpf( '0.1' )
mpf('-1.3877787807814457e-17')
>>> decimal.Decimal( '0.3' )/decimal.Decimal( '3.0' ) - decimal.Decimal (
'0.1' )
Decimal("0.0")
Decimal solves the problem; whereas mpmath doesn't.
Mike
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 14:44, Michael Gilbert wrote: > On Wed, Sep 8, 2010 at 12:23 PM, Charles R Harris wrote: >> >> >> On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert >> wrote: >>> >>> On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: >>> > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert >>> > >> > > wrote: >>> > >>> > > Hi, >>> > > >>> > > Are there any plans to add support for decimal floating point >>> > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard >>> > > [0], in numpy? >>> > > >>> > > >>> > Not at the moment. There is currently no hardware or C support and >>> > adding >>> > new types to numpy isn't trivial. You can get some limited Decimal >>> > functionality by using python classes and object arrays, for instance >>> > the >>> > Decimal class in the python decimal module, but the performance isn't >>> > great. >>> > >>> > What is your particular interest in decimal support? >>> >>> Primarily avoiding catastrophic cancellation when subtracting >>> large values. I was planning to use the decimal class, but was >>> curious whether support for the IEEE standard was coming any time soon. >>> >> >> If you just need more precision, mpmath has better performance than the >> Decimal class. Also, it might be possible to avoid the loss of precision by >> changing the computation, but I don't know the details of what you are >> doing. > > Just wanted to say that numpy object arrays + decimal solved all of my > problems, which were all caused by the disconnect between decimal and > binary representation of floating point numbers. Are you sure? Unless if I'm failing to think through this properly, catastrophic cancellation for large numbers is an intrinsic property of fixed-precision floating point regardless of the base. decimal and mpmath both help with that problem because they have arbitrary precision. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] indexing with booleans without making a copy?
8/09/10 @ 15:35 (-0400), thus spake Anne Archibald: > 2010/9/8 Ernest Adrogué : > > I have a sorted, flat array: > > > > In [139]: a =np.array([0,1,2,2,2,3]) > > > > Basically, I want views of the areas where there > > are repeated numbers (since the array is sorted, they > > will be contiguous). > > > > But, of course, to find the numbers that are repeated > > I have to use comparison operations that return > > boolean arrays, so I suppose the problem is converting > > the boolean array into a slice. > > Well, you're going to have to do some allocation, but how's this? Use > unique1d to get an array of unique values. Then use searchsorted > twice, once to find the leftmost end of each hunk, and once to find > the rightmost end of each hunk. I like it. Thanks :) > > Anne > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 12:23 PM, Charles R Harris wrote: > > > On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert > wrote: >> >> On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: >> > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert >> > > > > wrote: >> > >> > > Hi, >> > > >> > > Are there any plans to add support for decimal floating point >> > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard >> > > [0], in numpy? >> > > >> > > >> > Not at the moment. There is currently no hardware or C support and >> > adding >> > new types to numpy isn't trivial. You can get some limited Decimal >> > functionality by using python classes and object arrays, for instance >> > the >> > Decimal class in the python decimal module, but the performance isn't >> > great. >> > >> > What is your particular interest in decimal support? >> >> Primarily avoiding catastrophic cancellation when subtracting >> large values. I was planning to use the decimal class, but was >> curious whether support for the IEEE standard was coming any time soon. >> > > If you just need more precision, mpmath has better performance than the > Decimal class. Also, it might be possible to avoid the loss of precision by > changing the computation, but I don't know the details of what you are > doing. Just wanted to say that numpy object arrays + decimal solved all of my problems, which were all caused by the disconnect between decimal and binary representation of floating point numbers. It would be really nice to have IEEE decimal floating point since python's decimal module is a bit slow, but I can see thats going to be a very long process. Thanks again for your help! Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] clip() with None as argument
Robert Kern gmail.com> writes: > > On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt > gmail.com> wrote: > > Ah, no need to answer, I do this myself: > > > > Friedrich, would you please use numpy.inf and -numpy.inf. > > But if you have an integer array, you will run into the same problem. > The result will be upcast to float. I think we would accept a patch > that interprets None to be the appropriate extreme bound given the > input datatype. This will be tricky, though, since it must be done in > C (PyArray_Clip in numpy/core/src/multiarray/calculation.c). I've been a bit confused about what numpy's clip is supposed to support. The documentation (e.g. http://docs.scipy.org/doc/numpy/reference/generated/numpy.clip.html) doesn't make any mention of supporting only one of min or max, but I remember a message some time back from Travis Oliphant that seemed to suggest it does (http://thread.gmane.org/gmane.comp.python.numeric.general/17844/focus=17877). Right now, I'm surprised by two aspect's of clip's behavior, both demonstrated below... $ python Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '2.0.0.dev8698' >>> a = numpy.array([1,2,3,4,5]) >>> a.clip(2,None) array([2, 2, 2, 2, 2], dtype=object) I'm not sure why the returned array has a dtype of object (although that can be avoided by using clip's "out" argument), and also why the values are not like this: >>> numpy.maximum(a,2) array([2, 2, 3, 4, 5]) I've been using minimum() and maximum() instead of clip() to have "clip upper" and "clip lower" functions, so it's not really a problem for me, I'm just curious (and thought maybe this could be useful for others searching in the future). Thanks, Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] indexing with booleans without making a copy?
2010/9/8 Ernest Adrogué : > Hello, > > I have a sorted, flat array: > > In [139]: a =np.array([0,1,2,2,2,3]) > > Basically, I want views of the areas where there > are repeated numbers (since the array is sorted, they > will be contiguous). > > But, of course, to find the numbers that are repeated > I have to use comparison operations that return > boolean arrays, so I suppose the problem is converting > the boolean array into a slice. > > This is what I have come up with: > > In [146]: np.flatnonzero(a==2) > Out[146]: array([2, 3, 4]) > > In [147]: b=np.flatnonzero(a==2) > > In [148]: b.min() > Out[148]: 2 > > In [149]: b.max() > Out[149]: 4 > > In [150]: a[b.min():b.max()+1] > Out[150]: array([2, 2, 2]) > > If you know a more straightforward way of doing this, > I'll be glad to know... np.nonzero(np.diff(a)) add boundaries index zero and end make two arrays one with starting points, one with endpoints. Josef > > Cheers :) > Ernest > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] indexing with booleans without making a copy?
2010/9/8 Ernest Adrogué : > I have a sorted, flat array: > > In [139]: a =np.array([0,1,2,2,2,3]) > > Basically, I want views of the areas where there > are repeated numbers (since the array is sorted, they > will be contiguous). > > But, of course, to find the numbers that are repeated > I have to use comparison operations that return > boolean arrays, so I suppose the problem is converting > the boolean array into a slice. Well, you're going to have to do some allocation, but how's this? Use unique1d to get an array of unique values. Then use searchsorted twice, once to find the leftmost end of each hunk, and once to find the rightmost end of each hunk. Anne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] indexing with booleans without making a copy?
Hello, I have a sorted, flat array: In [139]: a =np.array([0,1,2,2,2,3]) Basically, I want views of the areas where there are repeated numbers (since the array is sorted, they will be contiguous). But, of course, to find the numbers that are repeated I have to use comparison operations that return boolean arrays, so I suppose the problem is converting the boolean array into a slice. This is what I have come up with: In [146]: np.flatnonzero(a==2) Out[146]: array([2, 3, 4]) In [147]: b=np.flatnonzero(a==2) In [148]: b.min() Out[148]: 2 In [149]: b.max() Out[149]: 4 In [150]: a[b.min():b.max()+1] Out[150]: array([2, 2, 2]) If you know a more straightforward way of doing this, I'll be glad to know... Cheers :) Ernest ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 9:46 AM, Michael Gilbert wrote: > On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: > > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert < > michael.s.gilb...@gmail.com > > > wrote: > > > > > Hi, > > > > > > Are there any plans to add support for decimal floating point > > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard > > > [0], in numpy? > > > > > > > > Not at the moment. There is currently no hardware or C support and adding > > new types to numpy isn't trivial. You can get some limited Decimal > > functionality by using python classes and object arrays, for instance the > > Decimal class in the python decimal module, but the performance isn't > great. > > > > What is your particular interest in decimal support? > > Primarily avoiding catastrophic cancellation when subtracting > large values. I was planning to use the decimal class, but was > curious whether support for the IEEE standard was coming any time soon. > > If you just need more precision, mpmath has better performance than the Decimal class. Also, it might be possible to avoid the loss of precision by changing the computation, but I don't know the details of what you are doing. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Thu, Sep 9, 2010 at 12:43 AM, Charles R Harris wrote: > > > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert > wrote: >> >> Hi, >> >> Are there any plans to add support for decimal floating point >> arithmetic, as defined in the 2008 revision of the IEEE 754 standard >> [0], in numpy? >> > > Not at the moment. There is currently no hardware Strictly speaking, there is hardware support on some high end hardware (power6, but I have no idea how this is available through common languages, like C or Fortran. I guess this would be compiler specific. A software implementation would be much more interesting at this stage, as I guess it will takes years before support comes to commodity CPUs. cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, 8 Sep 2010 09:43:56 -0600, Charles R Harris wrote: > On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert > wrote: > > > Hi, > > > > Are there any plans to add support for decimal floating point > > arithmetic, as defined in the 2008 revision of the IEEE 754 standard > > [0], in numpy? > > > > > Not at the moment. There is currently no hardware or C support and adding > new types to numpy isn't trivial. You can get some limited Decimal > functionality by using python classes and object arrays, for instance the > Decimal class in the python decimal module, but the performance isn't great. > > What is your particular interest in decimal support? Primarily avoiding catastrophic cancellation when subtracting large values. I was planning to use the decimal class, but was curious whether support for the IEEE standard was coming any time soon. Thanks, Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 9:26 AM, Michael Gilbert wrote: > Hi, > > Are there any plans to add support for decimal floating point > arithmetic, as defined in the 2008 revision of the IEEE 754 standard > [0], in numpy? > > Not at the moment. There is currently no hardware or C support and adding new types to numpy isn't trivial. You can get some limited Decimal functionality by using python classes and object arrays, for instance the Decimal class in the python decimal module, but the performance isn't great. What is your particular interest in decimal support? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] IEEE 754-2008 decimal floating point support
On Wed, Sep 8, 2010 at 10:26, Michael Gilbert wrote: > Hi, > > Are there any plans to add support for decimal floating point > arithmetic, as defined in the 2008 revision of the IEEE 754 standard > [0], in numpy? No, there are no plans. Although IEEE 754-2008 defines the format and semantics of such numbers, they do not provide an implementation. If someone wishes to make a C implementation and integrate that into numpy as a new dtype, I think that would be welcome. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] IEEE 754-2008 decimal floating point support
Hi, Are there any plans to add support for decimal floating point arithmetic, as defined in the 2008 revision of the IEEE 754 standard [0], in numpy? Thanks for any info. Best wishes, Mike [0] http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4610935&tag=1 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] vectorize doesn't like functools.partial
josef.p...@gmail.com wrote: > On Wed, Sep 8, 2010 at 10:53 AM, Neal Becker wrote: >> josef.p...@gmail.com wrote: >> >>> On Wed, Sep 8, 2010 at 7:44 AM, Neal Becker wrote: If I try to use vectorize on the result of functools.partial, I seem to get: ValueError: failed to determine the number of arguments for Anything I can do about it? >>> >>> Set .nin (attribute of vectorized function, I think) directly with >>> number of arguments (int) >>> >>> Josef >>> ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion >> Not sure what you mean. Here is a demo of 2 problems of interop of numpy >> and functools.partial. Tried both vectorize and frompyfunc. vectorize >> gives: >> ValueError: failed to determine the number of arguments for >> >> >> frompyfunc gives: >> TypeError: H() got multiple values for keyword argument 'iir' >> >> Note that doc for frompyfunc says: >> "Takes an arbitrary Python function..." >> >> import numpy as np >> >> class iir (object): >> def H (z): >> return 1 >> >> def H (iir, f): >> z = complex (cos (2*pi*f/fs), sin (2*pi*f/fs)) >> return iir.H (z) >> >> f = np.linspace (0, 10e6, 1000) >> #bug1 >> from functools import partial >> the_iir = iir() >> p = partial (H, iir=the_iir) >> resp_pll = np.vectorize (p)(f) >> #bug2 >> resp_pll = np.frompyfunc (p, 1, 1)(f) > > looks like a limitation of vectorize (fails in constructor), and nin > cannot be included in the constructor > > this should be a bug report or enhancement ticket > > a work around, after fixing other problems in your code example: > > p = partial(H, iir=the_iir) > fun = np.vectorize(lambda x: p(x)) > fun.nin > 1 fun(5) > array(1) fun(f)[:10] > array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1]) > > Josef Yes, I found that using lamda there is no need for partial at all. fun = np.vectorize (lambda f: H(the_iir, f)) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] vectorize doesn't like functools.partial
On Wed, Sep 8, 2010 at 10:53 AM, Neal Becker wrote: > josef.p...@gmail.com wrote: > >> On Wed, Sep 8, 2010 at 7:44 AM, Neal Becker wrote: >>> If I try to use vectorize on the result of functools.partial, I seem to >>> get: >>> >>> ValueError: failed to determine the number of arguments for >>> >>> >>> Anything I can do about it? >> >> Set .nin (attribute of vectorized function, I think) directly with >> number of arguments (int) >> >> Josef >> >>> >>> ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> > Not sure what you mean. Here is a demo of 2 problems of interop of numpy > and functools.partial. Tried both vectorize and frompyfunc. vectorize > gives: > ValueError: failed to determine the number of arguments for > > > frompyfunc gives: > TypeError: H() got multiple values for keyword argument 'iir' > > Note that doc for frompyfunc says: > "Takes an arbitrary Python function..." > > import numpy as np > > class iir (object): > def H (z): > return 1 > > def H (iir, f): > z = complex (cos (2*pi*f/fs), sin (2*pi*f/fs)) > return iir.H (z) > > f = np.linspace (0, 10e6, 1000) > #bug1 > from functools import partial > the_iir = iir() > p = partial (H, iir=the_iir) > resp_pll = np.vectorize (p)(f) > #bug2 > resp_pll = np.frompyfunc (p, 1, 1)(f) looks like a limitation of vectorize (fails in constructor), and nin cannot be included in the constructor this should be a bug report or enhancement ticket a work around, after fixing other problems in your code example: p = partial(H, iir=the_iir) fun = np.vectorize(lambda x: p(x)) >>> fun.nin 1 >>> fun(5) array(1) >>> fun(f)[:10] array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1]) Josef > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] vectorize doesn't like functools.partial
josef.p...@gmail.com wrote: > On Wed, Sep 8, 2010 at 7:44 AM, Neal Becker wrote: >> If I try to use vectorize on the result of functools.partial, I seem to >> get: >> >> ValueError: failed to determine the number of arguments for >> >> >> Anything I can do about it? > > Set .nin (attribute of vectorized function, I think) directly with > number of arguments (int) > > Josef > >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> Not sure what you mean. Here is a demo of 2 problems of interop of numpy and functools.partial. Tried both vectorize and frompyfunc. vectorize gives: ValueError: failed to determine the number of arguments for frompyfunc gives: TypeError: H() got multiple values for keyword argument 'iir' Note that doc for frompyfunc says: "Takes an arbitrary Python function..." import numpy as np class iir (object): def H (z): return 1 def H (iir, f): z = complex (cos (2*pi*f/fs), sin (2*pi*f/fs)) return iir.H (z) f = np.linspace (0, 10e6, 1000) #bug1 from functools import partial the_iir = iir() p = partial (H, iir=the_iir) resp_pll = np.vectorize (p)(f) #bug2 resp_pll = np.frompyfunc (p, 1, 1)(f) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ValueError: objects are not aligned
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
did the trick, 1d is perfect
Thankyou very much!
Rui
From: "josef.p...@gmail.com"
To: Discussion of Numerical Python
Sent: Wed, September 8, 2010 3:30:54 PM
Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta wrote:
> Thanks again,
>
> I was trying to
>follow
>http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>3
> but with the key difference being, that instead of the data for the three
> arrays being constructed from literals, that i'd be sourcing from a
> relational database. The data is quite simple, in that I have 199 recorded
> days for 8 different variables, but am just trying to do a proof-of-concept
> first with three, when I had this problem.
> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
> =CORREL(rf1,rf1)
> =CORREL(rf1,rf2)
> =CORREL(rf1,rf3)
> =CORREL(rf2,rf1)
> =CORREL(rf2,rf2)
> =CORREL(rf2,rf3)
> =CORREL(rf3,rf1)
> =CORREL(rf3,rf2)
> =CORREL(rf3,rf3)
> I tried to simplify and isolate the problem, and hence the confusion.
> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
> of the 8), each 199 rows in length.
> From reading the lists and blogs, the way to put these into numpy seemed to
> be:
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
> Then to try and produce a correlation matrix:
> print (np.corrcoef([rf1, rf2, rf3]))
> But this is where I get the original error.
> From looking at column_stack, this doesn't seem like what I want to achieve,
> or am I looking at this the wrong way somehow?.
column_stack and rowvar=0 should work, but it might be easier to get
rid of the extra dimension that the cursor gives you
e.g.
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
depends on whether you want to work with 2d or 1d data arrays
Josef
> Thanks again,
> Rui
>
>
> From: "josef.p...@gmail.com"
> To: Discussion of Numerical Python
> Sent: Wed, September 8, 2010 2:34:42 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta wrote:
>> Thanks for your assistance,
>> I was following this example:
>>
>>http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>>3
>
> I didn't know,
> If arrays are 1d, then numpy does the stacking for you., but it
> doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
> array([[ 1., 1.],
> [ 1., 1.]])
np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
> array([[ NaN, NaN, NaN],
> [ NaN, NaN, NaN],
> [ NaN, NaN, NaN]])
>>
>> I intend to use different arrays once I have resolved this problem.
>> Trying your suggestion:
>> ...
>> print (rf1)
>> print (np.corrcoef(rf1, rf1, rowvar=0))
>> results in:
>> [[-0.00641625]
>> [-0.00498411]
>> [-0.0038015 ]]
>> [[ 1. 1.]
>> [ 1. 1.]]
>>
>> which should have been a 3x3 matrix
>
> I'm not sure what you want here, you only have three values
> variance would also be zero, and corrcoef wouldn't be defined if you
> only have 1 observation per variable
>
np.corrcoef(np.arange(3)[:,None])
> array([[ NaN, NaN, NaN],
> [ NaN, NaN, NaN],
> [ NaN, NaN, NaN]])
>
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
np.arange(5)**3]), rowvar=0)
> array([[ 1., 0.9589266 , 0.90588235],
> [ 0.9589266 , 1., 0.98713033],
> [ 0.90588235, 0.98713033, 1.]])
>
> examples are in docstring for np.cov
>
>
>
>
>> as for "stacking rf1 rf1 into a single array first" how would I do this?
>
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
> [1, 1],
> [2, 2]])
np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required
2d
> array([[0, 0],
> [1, 1],
> [2, 2]])
>
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
> array([[0, 0],
> [1, 1],
> [2, 2]])
np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
> [1, 1],
> [2, 2]])
>
>
> Josef
>
>> Regards,
>> Rui
>>
>> From: "josef.p...@gmail.com"
>> To: Discussion of Numerical Python
>> Sent: Wed, September 8, 2010 1:09:46 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>>>
>>> I'm getting this error, which must be a simple error in getting the
>>> result
>>> from the cursor in the right shape, how do I get the cursor into the
>>> array
>>> as a single dimension?:
>>>
>>>import numpy as np
>>>import sqlite3
>>>conn = sqlite3.connect("simpledb")
>>>cursor1 = conn.execute("select
Re: [Numpy-discussion] ValueError: objects are not aligned
On 09/08/2010 08:30 AM, josef.p...@gmail.com wrote:
> On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta wrote:
>> Thanks again,
>>
>> I was trying to
>> follow
>> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> but with the key difference being, that instead of the data for the three
>> arrays being constructed from literals, that i'd be sourcing from a
>> relational database. The data is quite simple, in that I have 199 recorded
>> days for 8 different variables, but am just trying to do a proof-of-concept
>> first with three, when I had this problem.
>> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
>> =CORREL(rf1,rf1)
>> =CORREL(rf1,rf2)
>> =CORREL(rf1,rf3)
>> =CORREL(rf2,rf1)
>> =CORREL(rf2,rf2)
>> =CORREL(rf2,rf3)
>> =CORREL(rf3,rf1)
>> =CORREL(rf3,rf2)
>> =CORREL(rf3,rf3)
>> I tried to simplify and isolate the problem, and hence the confusion.
>> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
>> of the 8), each 199 rows in length.
>> From reading the lists and blogs, the way to put these into numpy seemed to
>> be:
>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
>> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
>> Then to try and produce a correlation matrix:
>> print (np.corrcoef([rf1, rf2, rf3]))
>> But this is where I get the original error.
>> From looking at column_stack, this doesn't seem like what I want to achieve,
>> or am I looking at this the wrong way somehow?.
> column_stack and rowvar=0 should work, but it might be easier to get
> rid of the extra dimension that the cursor gives you
>
> e.g.
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
>
> depends on whether you want to work with 2d or 1d data arrays
>
> Josef
>
>> Thanks again,
>> Rui
>>
>>
>> From: "josef.p...@gmail.com"
>> To: Discussion of Numerical Python
>> Sent: Wed, September 8, 2010 2:34:42 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta wrote:
>>> Thanks for your assistance,
>>> I was following this example:
>>>
>>> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> I didn't know,
>> If arrays are 1d, then numpy does the stacking for you., but it
>> doesn't work with column-vectors or 2d arrays.
> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
>> array([[ 1., 1.],
>>[ 1., 1.]])
> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
>> array([[ NaN, NaN, NaN],
>>[ NaN, NaN, NaN],
>>[ NaN, NaN, NaN]])
>>> I intend to use different arrays once I have resolved this problem.
>>> Trying your suggestion:
>>> ...
>>> print (rf1)
>>> print (np.corrcoef(rf1, rf1, rowvar=0))
>>> results in:
>>> [[-0.00641625]
>>> [-0.00498411]
>>> [-0.0038015 ]]
>>> [[ 1. 1.]
>>> [ 1. 1.]]
>>>
>>> which should have been a 3x3 matrix
>> I'm not sure what you want here, you only have three values
>> variance would also be zero, and corrcoef wouldn't be defined if you
>> only have 1 observation per variable
>>
> np.corrcoef(np.arange(3)[:,None])
>> array([[ NaN, NaN, NaN],
>>[ NaN, NaN, NaN],
>>[ NaN, NaN, NaN]])
>>
> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
> np.arange(5)**3]), rowvar=0)
>> array([[ 1., 0.9589266 , 0.90588235],
>>[ 0.9589266 , 1., 0.98713033],
>>[ 0.90588235, 0.98713033, 1.]])
>>
>> examples are in docstring for np.cov
>>
>>
>>
>>
>>> as for "stacking rf1 rf1 into a single array first" how would I do this?
> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>>[1, 1],
>>[2, 2]])
> np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required
> 2d
>> array([[0, 0],
>>[1, 1],
>>[2, 2]])
>>
> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
>> array([[0, 0],
>>[1, 1],
>>[2, 2]])
> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>>[1, 1],
>>[2, 2]])
>>
>>
>> Josef
>>
>>> Regards,
>>> Rui
>>>
>>> From: "josef.p...@gmail.com"
>>> To: Discussion of Numerical Python
>>> Sent: Wed, September 8, 2010 1:09:46 PM
>>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>>
>>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
I'm getting this error, which must be a simple error in getting the
result
from the cursor in the right shape, how do I get the cursor into the
array
as a single dimension?:
import numpy as np
import sqlite3
conn = sqlite3.connect("simpledb")
cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
print (rf1)
Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta wrote:
> Thanks again,
>
> I was trying to
> follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
> but with the key difference being, that instead of the data for the three
> arrays being constructed from literals, that i'd be sourcing from a
> relational database. The data is quite simple, in that I have 199 recorded
> days for 8 different variables, but am just trying to do a proof-of-concept
> first with three, when I had this problem.
> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
> =CORREL(rf1,rf1)
> =CORREL(rf1,rf2)
> =CORREL(rf1,rf3)
> =CORREL(rf2,rf1)
> =CORREL(rf2,rf2)
> =CORREL(rf2,rf3)
> =CORREL(rf3,rf1)
> =CORREL(rf3,rf2)
> =CORREL(rf3,rf3)
> I tried to simplify and isolate the problem, and hence the confusion.
> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
> of the 8), each 199 rows in length.
> From reading the lists and blogs, the way to put these into numpy seemed to
> be:
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
> Then to try and produce a correlation matrix:
> print (np.corrcoef([rf1, rf2, rf3]))
> But this is where I get the original error.
> From looking at column_stack, this doesn't seem like what I want to achieve,
> or am I looking at this the wrong way somehow?.
column_stack and rowvar=0 should work, but it might be easier to get
rid of the extra dimension that the cursor gives you
e.g.
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
depends on whether you want to work with 2d or 1d data arrays
Josef
> Thanks again,
> Rui
>
>
> From: "josef.p...@gmail.com"
> To: Discussion of Numerical Python
> Sent: Wed, September 8, 2010 2:34:42 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta wrote:
>> Thanks for your assistance,
>> I was following this example:
>>
>> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>
> I didn't know,
> If arrays are 1d, then numpy does the stacking for you., but it
> doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
> array([[ 1., 1.],
> [ 1., 1.]])
np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
> array([[ NaN, NaN, NaN],
> [ NaN, NaN, NaN],
> [ NaN, NaN, NaN]])
>>
>> I intend to use different arrays once I have resolved this problem.
>> Trying your suggestion:
>> ...
>> print (rf1)
>> print (np.corrcoef(rf1, rf1, rowvar=0))
>> results in:
>> [[-0.00641625]
>> [-0.00498411]
>> [-0.0038015 ]]
>> [[ 1. 1.]
>> [ 1. 1.]]
>>
>> which should have been a 3x3 matrix
>
> I'm not sure what you want here, you only have three values
> variance would also be zero, and corrcoef wouldn't be defined if you
> only have 1 observation per variable
>
np.corrcoef(np.arange(3)[:,None])
> array([[ NaN, NaN, NaN],
> [ NaN, NaN, NaN],
> [ NaN, NaN, NaN]])
>
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
np.arange(5)**3]), rowvar=0)
> array([[ 1. , 0.9589266 , 0.90588235],
> [ 0.9589266 , 1. , 0.98713033],
> [ 0.90588235, 0.98713033, 1. ]])
>
> examples are in docstring for np.cov
>
>
>
>
>> as for "stacking rf1 rf1 into a single array first" how would I do this?
>
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
> [1, 1],
> [2, 2]])
np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required
2d
> array([[0, 0],
> [1, 1],
> [2, 2]])
>
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
> array([[0, 0],
> [1, 1],
> [2, 2]])
np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
> [1, 1],
> [2, 2]])
>
>
> Josef
>
>> Regards,
>> Rui
>>
>> From: "josef.p...@gmail.com"
>> To: Discussion of Numerical Python
>> Sent: Wed, September 8, 2010 1:09:46 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>>>
>>> I'm getting this error, which must be a simple error in getting the
>>> result
>>> from the cursor in the right shape, how do I get the cursor into the
>>> array
>>> as a single dimension?:
>>>
>>> import numpy as np
>>> import sqlite3
>>> conn = sqlite3.connect("simpledb")
>>> cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>> print (rf1)
>>> print (np.corrcoef([rf1, rf1]))
>>
>> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>>
>> try
>> np.corrcoef(rf1, rf1, rowvar=0) no [ ]
>> or
>> stacking rf1 rf1 into a single array first
>>
>> Josef
>>
>>>
>>> result
Re: [Numpy-discussion] ValueError: objects are not aligned
Thanks again,
I was trying to
follow
http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
but with the key difference being, that instead of the data for the three
arrays
being constructed from literals, that i'd be sourcing from a relational
database. The data is quite simple, in that I have 199 recorded days for 8
different variables, but am just trying to do a proof-of-concept first with
three, when I had this problem.
The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
=CORREL(rf1,rf1)
=CORREL(rf1,rf2)
=CORREL(rf1,rf3)
=CORREL(rf2,rf1)
=CORREL(rf2,rf2)
=CORREL(rf2,rf3)
=CORREL(rf3,rf1)
=CORREL(rf3,rf2)
=CORREL(rf3,rf3)
I tried to simplify and isolate the problem, and hence the confusion.
So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of
the 8), each 199 rows in length.
From reading the lists and blogs, the way to put these into numpy seemed to be:
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
Then to try and produce a correlation matrix:
print (np.corrcoef([rf1, rf2, rf3]))
But this is where I get the original error.
From looking at column_stack, this doesn't seem like what I want to achieve, or
am I looking at this the wrong way somehow?.
Thanks again,
Rui
From: "josef.p...@gmail.com"
To: Discussion of Numerical Python
Sent: Wed, September 8, 2010 2:34:42 PM
Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta wrote:
> Thanks for your assistance,
> I was following this example:
>http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>3
I didn't know,
If arrays are 1d, then numpy does the stacking for you., but it
doesn't work with column-vectors or 2d arrays.
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
array([[ 1., 1.],
[ 1., 1.]])
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
array([[ NaN, NaN, NaN],
[ NaN, NaN, NaN],
[ NaN, NaN, NaN]])
>
> I intend to use different arrays once I have resolved this problem.
> Trying your suggestion:
> ...
> print (rf1)
> print (np.corrcoef(rf1, rf1, rowvar=0))
> results in:
> [[-0.00641625]
> [-0.00498411]
> [-0.0038015 ]]
> [[ 1. 1.]
> [ 1. 1.]]
>
> which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values
variance would also be zero, and corrcoef wouldn't be defined if you
only have 1 observation per variable
>>> np.corrcoef(np.arange(3)[:,None])
array([[ NaN, NaN, NaN],
[ NaN, NaN, NaN],
[ NaN, NaN, NaN]])
>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
>>> np.arange(5)**3]),
>>>rowvar=0)
array([[ 1., 0.9589266 , 0.90588235],
[ 0.9589266 , 1., 0.98713033],
[ 0.90588235, 0.98713033, 1.]])
examples are in docstring for np.cov
> as for "stacking rf1 rf1 into a single array first" how would I do this?
>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
[1, 1],
[2, 2]])
Josef
> Regards,
> Rui
>
> From: "josef.p...@gmail.com"
> To: Discussion of Numerical Python
> Sent: Wed, September 8, 2010 1:09:46 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>>
>> I'm getting this error, which must be a simple error in getting the result
>> from the cursor in the right shape, how do I get the cursor into the array
>> as a single dimension?:
>>
>>import numpy as np
>>import sqlite3
>>conn = sqlite3.connect("simpledb")
>>cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>print (rf1)
>>print (np.corrcoef([rf1, rf1]))
>
> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>
> try
> np.corrcoef(rf1, rf1, rowvar=0) no [ ]
> or
> stacking rf1 rf1 into a single array first
>
> Josef
>
>>
>> result:
>> [[-0.00641625]
>> [-0.00498411]
>> [-0.0038015 ]]
>> Traceback (most recent call last):
>> File "C:\Documents and
>> Settings\rui\workspace\numpytest\src\numpytest.py",
>> line 38, in
>>print (np.corrcoef([rf1, rf1]))
>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 2003, in corrcoef
>>c = cov(x, y, rowvar, bias, ddof)
>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 1953, in cov
>>return (dot(X, X.T.conj()) / fact).squee
Re: [Numpy-discussion] new warning in 1.5.0: divide by zero encountered in log
On Wed, Sep 8, 2010 at 4:38 AM, John Reid wrote: > Hi, > > I recently upgraded to numpy 1.5.0 and now I get warnings when I take > the logarithm of 0. > > In [5]: np.log(0.) > Warning: divide by zero encountered in log > Out[5]: -inf > > > I want to evaluate x * log(x) where its value is defined as 0 when x=0 > so I have the following function: > > def safe_x_log_x(x): > "@return: x log(x) but replaces -inf with 0." > l = np.log(x) > result = x * l > result[np.isneginf(l)] = 0. > return result > > > Is there a better way of doing this that won't give me all these > warnings? Can I turn the warnings off? > > In [1]: log(0) Warning: divide by zero encountered in log Out[1]: -inf In [2]: olderr = np.seterr(divide='ignore') In [3]: log(0) Out[3]: -inf Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] new warning in 1.5.0: divide by zero encountered in log
On 9/8/2010 6:38 AM, John Reid wrote: > def safe_x_log_x(x): > "@return: x log(x) but replaces -inf with 0." > l = np.log(x) > result = x * l > result[np.isneginf(l)] = 0. > return result Assuming x is known to contain nonnegative floats: def safe_x_log_x(x): x = np.asarray(x) l = np.zeros_like(x) l[x>0] = np.log(x[x>0]) return x * l hth, Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta wrote:
> Thanks for your assistance,
> I was following this example:
> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
I didn't know,
If arrays are 1d, then numpy does the stacking for you., but it
doesn't work with column-vectors or 2d arrays.
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
array([[ 1., 1.],
[ 1., 1.]])
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
array([[ NaN, NaN, NaN],
[ NaN, NaN, NaN],
[ NaN, NaN, NaN]])
>
> I intend to use different arrays once I have resolved this problem.
> Trying your suggestion:
> ...
> print (rf1)
> print (np.corrcoef(rf1, rf1, rowvar=0))
> results in:
> [[-0.00641625]
> [-0.00498411]
> [-0.0038015 ]]
> [[ 1. 1.]
> [ 1. 1.]]
>
> which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values
variance would also be zero, and corrcoef wouldn't be defined if you
only have 1 observation per variable
>>> np.corrcoef(np.arange(3)[:,None])
array([[ NaN, NaN, NaN],
[ NaN, NaN, NaN],
[ NaN, NaN, NaN]])
>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
>>> np.arange(5)**3]), rowvar=0)
array([[ 1., 0.9589266 , 0.90588235],
[ 0.9589266 , 1., 0.98713033],
[ 0.90588235, 0.98713033, 1.]])
examples are in docstring for np.cov
> as for "stacking rf1 rf1 into a single array first" how would I do this?
>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
array([[0, 0],
[1, 1],
[2, 2]])
>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
[1, 1],
[2, 2]])
Josef
> Regards,
> Rui
>
> From: "josef.p...@gmail.com"
> To: Discussion of Numerical Python
> Sent: Wed, September 8, 2010 1:09:46 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>>
>> I'm getting this error, which must be a simple error in getting the result
>> from the cursor in the right shape, how do I get the cursor into the array
>> as a single dimension?:
>>
>> import numpy as np
>> import sqlite3
>> conn = sqlite3.connect("simpledb")
>> cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>> print (rf1)
>> print (np.corrcoef([rf1, rf1]))
>
> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>
> try
> np.corrcoef(rf1, rf1, rowvar=0) no [ ]
> or
> stacking rf1 rf1 into a single array first
>
> Josef
>
>>
>> result:
>> [[-0.00641625]
>> [-0.00498411]
>> [-0.0038015 ]]
>> Traceback (most recent call last):
>> File "C:\Documents and
>> Settings\rui\workspace\numpytest\src\numpytest.py",
>> line 38, in
>> print (np.corrcoef([rf1, rf1]))
>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 2003, in corrcoef
>> c = cov(x, y, rowvar, bias, ddof)
>> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 1953, in cov
>> return (dot(X, X.T.conj()) / fact).squeeze()
>> --
>> View this message in context:
>> http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ValueError: objects are not aligned
Thanks for your assistance,
I was following this example:
http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
I intend to use different arrays once I have resolved this problem.
Trying your suggestion:
...
print (rf1)
print (np.corrcoef(rf1, rf1, rowvar=0))
results in:
[[-0.00641625]
[-0.00498411]
[-0.0038015 ]]
[[ 1. 1.]
[ 1. 1.]]
which should have been a 3x3 matrix
as for "stacking rf1 rf1 into a single array first" how would I do this?
Regards,
Rui
From: "josef.p...@gmail.com"
To: Discussion of Numerical Python
Sent: Wed, September 8, 2010 1:09:46 PM
Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>
> I'm getting this error, which must be a simple error in getting the result
> from the cursor in the right shape, how do I get the cursor into the array
> as a single dimension?:
>
>import numpy as np
>import sqlite3
>conn = sqlite3.connect("simpledb")
>cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>print (rf1)
>print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try
np.corrcoef(rf1, rf1, rowvar=0) no [ ]
or
stacking rf1 rf1 into a single array first
Josef
>
> result:
> [[-0.00641625]
> [-0.00498411]
> [-0.0038015 ]]
> Traceback (most recent call last):
> File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py",
> line 38, in
>print (np.corrcoef([rf1, rf1]))
> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
> 2003, in corrcoef
>c = cov(x, y, rowvar, bias, ddof)
> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
> 1953, in cov
>return (dot(X, X.T.conj()) / fact).squeeze()
> --
> View this message in context:
>http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] vectorize doesn't like functools.partial
On Wed, Sep 8, 2010 at 7:44 AM, Neal Becker wrote: > If I try to use vectorize on the result of functools.partial, I seem to > get: > > ValueError: failed to determine the number of arguments for > > > Anything I can do about it? Set .nin (attribute of vectorized function, I think) directly with number of arguments (int) Josef > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] DataArray fixes
Fernando Perez writes: > Just let me know what you and the others (Keith, Skipper, Rob, etc) > think and I'm more than happy to open the doors to make the process > easier. Well, the pull requests I posted to your repository are for fairly trivial fixes (but some still need documentation and regression test updates). But some of the other features I posted on the issue tracker might need some previous discussion (I can recall the 'axis' vs 'axes' issue from the top of my head). Lluis -- "And it's much the same thing with knowledge, for whenever you learn something new, the whole world becomes that much richer." -- The Princess of Pure Reason, as told by Norton Juster in The Phantom Tollbooth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] vectorize doesn't like functools.partial
If I try to use vectorize on the result of functools.partial, I seem to get: ValueError: failed to determine the number of arguments for Anything I can do about it? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC wrote:
>
> I'm getting this error, which must be a simple error in getting the result
> from the cursor in the right shape, how do I get the cursor into the array
> as a single dimension?:
>
> import numpy as np
> import sqlite3
> conn = sqlite3.connect("simpledb")
> cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
> print (rf1)
> print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try
np.corrcoef(rf1, rf1, rowvar=0) no [ ]
or
stacking rf1 rf1 into a single array first
Josef
>
> result:
> [[-0.00641625]
> [-0.00498411]
> [-0.0038015 ]]
> Traceback (most recent call last):
> File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py",
> line 38, in
> print (np.corrcoef([rf1, rf1]))
> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
> 2003, in corrcoef
> c = cov(x, y, rowvar, bias, ddof)
> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
> 1953, in cov
> return (dot(X, X.T.conj()) / fact).squeeze()
> --
> View this message in context:
> http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] new warning in 1.5.0: divide by zero encountered in log
Hi, I recently upgraded to numpy 1.5.0 and now I get warnings when I take the logarithm of 0. In [5]: np.log(0.) Warning: divide by zero encountered in log Out[5]: -inf I want to evaluate x * log(x) where its value is defined as 0 when x=0 so I have the following function: def safe_x_log_x(x): "@return: x log(x) but replaces -inf with 0." l = np.log(x) result = x * l result[np.isneginf(l)] = 0. return result Is there a better way of doing this that won't give me all these warnings? Can I turn the warnings off? Thanks, John. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] ValueError: objects are not aligned
I'm getting this error, which must be a simple error in getting the result
from the cursor in the right shape, how do I get the cursor into the array
as a single dimension?:
import numpy as np
import sqlite3
conn = sqlite3.connect("simpledb")
cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
print (rf1)
print (np.corrcoef([rf1, rf1]))
result:
[[-0.00641625]
[-0.00498411]
[-0.0038015 ]]
Traceback (most recent call last):
File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py",
line 38, in
print (np.corrcoef([rf1, rf1]))
File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
2003, in corrcoef
c = cov(x, y, rowvar, bias, ddof)
File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
1953, in cov
return (dot(X, X.T.conj()) / fact).squeeze()
--
View this message in context:
http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] DataArray fixes
Hi Lluis, On Mon, Sep 6, 2010 at 12:17 PM, Lluís wrote: > I started writing some fixes for DataArray, but some others need discussion on > whether they should be implemented as I describe. > > To this extent, please see issues at http://github.com/fperez/datarray/issues > and comment on them, so that I'll know if I should implement these. I'm afraid I'm completely bandwidth-saturated for the next 2-3 weeks, but at the same time I have no intention whatsoever in becoming the bottleneck on all of this. Ultimately I don't see datarray as *my* code, but rather as code we as a group will refine to make numpy better. Keith Goodman has been also helping out, but I'm wondering if it would be good if I add a few more people with commit rights to that repo, so that development can move forward? Ultimately it's the people who do the coding who matter, and right now I'm too busy on the ipython side to do anything on this front at all. Just let me know what you and the others (Keith, Skipper, Rob, etc) think and I'm more than happy to open the doors to make the process easier. Once numpy settles in its github home, I hope we'll just move this repo from my personal hosting over to the numpy organization one, but I want to do that after numpy makes the transition to give them a chance to 'settle in'. Cheers, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
