I'd like suggestions on how to work around
NotImplementedError: record arrays with columns with type description ``([('a',
'
<f8'), ('b', '<f8')],(3,))`` are not supported yet, sorry
I have multivariate time-series with a fixed number of time-steps stored as
numpy structured arrays. For example, two records with a field called "y" that
contains an array of shape (3,) with items having fields "a" and "b".
(¤¤¤ = >>> to get through Gmane's top-posting filter)
¤¤¤ x
rec.array([([(0.0, 1.0), (2.0, 3.0), (4.0, 5.0)],),
([(6.0, 7.0), (8.0, 9.0), (10.0, 11.0)],)],
dtype=[('y', [('a', '<f8'), ('b', '<f8')], 3)])
¤¤¤ x.y.a
array([[ 0., 2., 4.],
[ 6., 8., 10.]])
However, it seems PyTables cannot deal with a column that contains an array of
a structured dtype. Example:
import numpy as np
import tables as pt
dtype = [("y", [("a", float), ("b", float)], 3)]
x = np.arange(12.0).view(dtype, np.recarray)
x.y.a[1] # array([ 6., 8., 10.])
f = pt.openFile("testnest.h5", "w")
t = f.createTable(f.root, "t", x)
Traceback (most recent call last):
NotImplementedError: record arrays with columns with type description ``([('a',
'
<f8'), ('b', '<f8')],(3,))`` are not supported yet, sorry
What is the best workaround for this, given that I have a lot of existing code
based on the assumption that each record has a field called y that contains a
vector whose items have fields ("a", "b")?
One option is to store each record as a plain float array, keeping the dtype as
an extra attribute and converting every time I put data in or out.
¤¤¤ xf = x.view(float).reshape(len(x), -1)
¤¤¤ xf
array([[ 0., 1., 2., 3., 4., 5.],
[ 6., 7., 8., 9., 10., 11.]])
Another is to go from a record that has a field that
...contains a vector whose items have fields ("a", "b")
to
...has fields ("a", "b") whose items are vectors
This can be stored in PyTables. However, the data in each record will be
transposed (all time-points for each field will be contiguous).
Any suggestions are welcome.
Best regards,
Jon Olav Vik
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
