On Sat, May 15, 2010 at 12:24 AM, Jimmie Houchin <jlhouc...@gmail.com> wrote:
> Hello, I am really liking Numpy a lot. It is wonderful to be able to do
> the things that it does in a language as friendly as Python, and with
> the performance Numpy delivers over standard Python. Thanks.
>
> I am having a problem with creation of Numpy arrays with my generated
> dtypes. I am creating a dataset of a weeks worth of financial
> instruments data, in order to explore and test relationships with
> various Technical Analysis functions.
>
> My basic data is simply a list (or tuple) of lists (or tuples).
> ((startdate, bidopen, bidhigh, bidlow, bidclose, askopen, askhigh,
> asklow, askclose), ...)
>
> Nothing unusual. However I am creating arrays which have many, many more
> columns to allow storing the data generated from applying the functions
> to the original data.
>
> I have created two functions. One to dynamically create the dtype based
> on data I want to create for the exploration. And another to create the
> array and populate it with the initial data from a database.
>
> Code slightly modified, not tested.
>
> #examples
> taFunctions = (smva, wmva)
> inputColumns = (bidclose, ohlcavg)
>
> def createDType():
> """Will create a dtype based on the pattern for naming and the
> parameters of those items being stored in the array.
> """
> dttypes = [('startdate','object'),
> ('bidopen','f8'), ('bidhigh','f8'),
> ('bidlow','f8'), ('bidclose','f8'),
> ('askopen','f8'), ('askhigh','f8'),
> ('asklow','f8'), ('askclose','f8'),
> ('ocavg','f8'), ('hlavg','f8'), ('ohlavg','f8'),
> ('ohlcavg','f8'), ('direction','i1'), ('volatility', 'f8'),
> ('spread', 'f8'), ('pivot', 'S4')]
> for f in taFunctions:
> for i in inputColumns:
> dttypes.append((f+"-"+i,'f8'))
> dtminute = np.dtype(dttypes)
> return dtminute, dttypes
>
> def getArray(instrument, weekString=None):
> ...
> cur.execute(sql)
> weekData = cur.fetchall()
> wdata = []
> lst = []
> dtminute, dttypes = createDType()
> for i in dttypes:
> if i[1] == 'f8': lst.append(0.0)
> elif i[1] == 'i1': lst.append(0)
> else: lst.append('')
> for m in weekData:
> data = list(m)+lst[9:]
> wdata.append(data)
I think "data" here should be a tuple, i.e. tuple(data)
structured arrays expect tuples for each element/row
If this is not it, then you could provide a mini example of wdata with
just a few rows.
> return np.array(wdata,dtype=dtminute)
>
> The createDType() function works fine. The getArray() function fails with:
> ValueError: Setting void-array with object members using buffer.
cryptic exceptions messages in array construction usually means there
is some structure in the argument data that numpy doesn't understand,
I usually work with trial and error for a specific example
Josef
>
> However changing the getArray() function to this works just fine.
>
> def getArray(instrument, weekString=None):
> ...
> cur.execute(sql)
> weekData = cur.fetchall()
> arrayLength = len(weekData)
> lst = []
> dtminute, dttypes = createDType()
> for i in dttypes:
> if i[1] == 'f8': lst.append(0.0)
> elif i[1] == 'i1': lst.append(0)
> else: lst.append('')
> listLength = len(lst)
> weekArray = np.zeros(arrayLength, dtype=dtminute)
> for i in range(arrayLength):
> for j in range(listLength):
> if j < 9: weekArray[i][j] = weekData[i][j]
> else: weekArray[i][j] = lst[j]
> return weekArray
>
> After I finally worked out getArray number two I am back in business
> writing the rest of my app. But I banged my head on version number one
> for quite some time trying to figure out what I am doing wrong. I still
> don't know.
>
> I find no errors in my data length or types. I would thing that either
> would cause version two to fail also.
>
> In help in understanding is greatly appreciated.
>
> This is using Numpy 1.4.1.
>
> Thanks.
>
> Jimmie
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
