Thanks again,
I was trying to follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53 but with the key difference being, that instead of the data for the three arrays being constructed from literals, that i'd be sourcing from a relational database. The data is quite simple, in that I have 199 recorded days for 8 different variables, but am just trying to do a proof-of-concept first with three, when I had this problem. The functionality I am seeking to reproduce is in Excel as a 3x3 grid: =CORREL(rf1,rf1) =CORREL(rf1,rf2) =CORREL(rf1,rf3) =CORREL(rf2,rf1) =CORREL(rf2,rf2) =CORREL(rf2,rf3) =CORREL(rf3,rf1) =CORREL(rf3,rf2) =CORREL(rf3,rf3) I tried to simplify and isolate the problem, and hence the confusion. So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of the 8), each 199 rows in length. From reading the lists and blogs, the way to put these into numpy seemed to be: rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64) rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64) Then to try and produce a correlation matrix: print (np.corrcoef([rf1, rf2, rf3])) But this is where I get the original error. From looking at column_stack, this doesn't seem like what I want to achieve, or am I looking at this the wrong way somehow?. Thanks again, Rui ________________________________ From: "[email protected]" <[email protected]> To: Discussion of Numerical Python <[email protected]> Sent: Wed, September 8, 2010 2:34:42 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <[email protected]> wrote: > Thanks for your assistance, > I was following this example: >http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53 >3 I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays. >>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) >>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]]) > > I intend to use different arrays once I have resolved this problem. > Trying your suggestion: > ... > print (rf1) > print (np.corrcoef(rf1, rf1, rowvar=0)) > results in: > [[-0.00641625] > [-0.00498411] > [-0.0038015 ]] > [[ 1. 1.] > [ 1. 1.]] > > which should have been a 3x3 matrix I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable >>> np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]]) >>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, >>> np.arange(5)**3]), >>>rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]]) examples are in docstring for np.cov > as for "stacking rf1 rf1 into a single array first" how would I do this? >>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) >>> np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]]) >>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) >>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) Josef > Regards, > Rui > ________________________________ > From: "[email protected]" <[email protected]> > To: Discussion of Numerical Python <[email protected]> > Sent: Wed, September 8, 2010 1:09:46 PM > Subject: Re: [Numpy-discussion] ValueError: objects are not aligned > > On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <[email protected]> wrote: >> >> I'm getting this error, which must be a simple error in getting the result >> from the cursor in the right shape, how do I get the cursor into the array >> as a single dimension?: >> >> import numpy as np >> import sqlite3 >> conn = sqlite3.connect("simpledb") >> cursor1 = conn.execute("select LogReturn from tblReturns limit 3") >> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) >> print (rf1) >> print (np.corrcoef([rf1, rf1])) > > numpy.corrcoef(x, y=None, rowvar=1, bias=0) > > try > np.corrcoef(rf1, rf1, rowvar=0) no [ ] > or > stacking rf1 rf1 into a single array first > > Josef > >> >> result: >> [[-0.00641625] >> [-0.00498411] >> [-0.0038015 ]] >> Traceback (most recent call last): >> File "C:\Documents and >> Settings\rui\workspace\numpytest\src\numpytest.py", >> line 38, in <module> >> print (np.corrcoef([rf1, rf1])) >> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line >> 2003, in corrcoef >> c = cov(x, y, rowvar, bias, ddof) >> File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line >> 1953, in cov >> return (dot(X, X.T.conj()) / fact).squeeze() >> -- >> View this message in context: >>http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html >>l >> Sent from the Numpy-discussion mailing list archive at Nabble.com. >> >> _______________________________________________ >> NumPy-Discussion mailing list >> [email protected] >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
