Hi Jason,
Upon further inspection, I think that this comes down to a spelling issue.
Really, what you want is a VL structure inside of a table. Tables are
themselves
VL structures and have been special cased to nest themselves. This could
be
changed to allow more general VL structures to be nested, but that would
require
a good amount of effort (that I sadly don't have time for now).
But really the only thing that you are doing when you nest a table within a
table
rather than nesting a vlarray in a table is that you are providing *extra*
information.
This isn't really a bad thing, and gets to what I mean by spelling.
Nesting a table in a table you have,
desc = {'A': Float64Col, 'B': {'C': Int32Col}}
Nesting a vlarry inside of a column would presumably be
desc = {'A': Float64Col, 'B': VLArrayCol(atom=Int32Col)}
Unfortunately, the VLArrayCol object doesn't exist. The reason it doesn't
I suspect is because of how you would spell the path to the nested data.
In nested tables you use the column names:
t['B/C']
In nested vlarrays, you'd have to use the index, which is weird.
t['B/0'], t['B/1'], t['B/2'], ...
Nested tables are everything you want, plus a tiny bit more in terms of
having to
name the dimension you want to be variable.
It should be noted that you end up with the same weird spelling issues in
numpy
structured arrays, which similarly disallow such funny business.
Of course, as Dav mentions, there are other options. Store your VLArrays
outside of
the table, then link to them in the table either through path or hard links,
etc.
At the end of the day I would still suggest the table in a table idea. It
isn't that much
extra work, and on the HDF5 level it is exactly what you want.
Be Well
Anthony
On Wed, Jun 22, 2011 at 6:26 PM, Jason Moore <moorepa...@gmail.com> wrote:
> Anthony,
>
> I read of the documentation and tried creating a table with a VLArray
> nested inside, but can't seem to figure out how to nest anything inside a
> table other than another table description.
>
> I found this question that is trying to do exactly what I want to do:
>
>
> http://stackoverflow.com/questions/5366099/in-pytables-how-to-create-nested-array-of-variable-length
>
> It is well written but has no answers.
>
> Any specific ideas on how to make this example work?
>
> Jason
>
>
> On Wed, Jun 22, 2011 at 2:57 PM, Anthony Scopatz <scop...@gmail.com>wrote:
>
>> Also note that whenever you are dealing with variable length stuctures,
>> chuncksize is very important.
>>
>>
>> On Wed, Jun 22, 2011 at 4:55 PM, Anthony Scopatz <scop...@gmail.com>wrote:
>>
>>> Hi Jason,
>>>
>>> You can try nesting variable length arrays inside of your table (
>>> http://www.pytables.org/docs/manual/ch04.html#VLArrayClassDescr). You
>>> might need to refer to nested tables to get an idea of how to do this (
>>> http://www.pytables.org/docs/manual/ch03.html#id332794). On the HDF5
>>> level, tables are just a convenient wrapper around variable length arrays of
>>> structs. Since you can nest tables, you should be able to nest VLArrays
>>> inside of tables.
>>>
>>> Let us know if you run into specific problems.
>>>
>>> Be Well
>>> Anthony
>>>
>>> On Wed, Jun 22, 2011 at 4:46 PM, Jason Moore <moorepa...@gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been attempting to use pyTables to store data from a
>>>> data acquisition system that I'm using for my experimental work. I
>>>> currently
>>>> make use of several pytables tables to store the data and really like the
>>>> querying ability that the table object provides. But I have an issue with
>>>> storing arrays in a table column. For each experiment that I do, I record
>>>> data for a different amount of time and potentially a different sampling
>>>> rate. I'd like the rows of the table to correspond with each experiment and
>>>> the columns with the various data signals that I collect. But it seems that
>>>> a pytables table object requires every array in a column to be of the same
>>>> shape. Right now I choose an arbitrarily large shape in my table definition
>>>> and I'm awkwardly padding the data signals before I store them in the
>>>> table.
>>>> Are there other ways to do this?
>>>>
>>>> Jason
>>>>
>>>> --
>>>> http://mae.ucdavis.edu/~biosport/jkm/
>>>> Sports Biomechanics Lab <http://biosport.ucdavis.edu>, UC Davis
>>>> Davis Bike Collective <http://www.davisbikecollective.org> Minister,
>>>> Davis, CA
>>>> BikeDavis.info
>>>> Office: +01 530-752-2163
>>>> Lab: +01 530-752-2235
>>>> Home: +01 530-753-0794
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Simplify data backup and recovery for your virtual environment with
>>>> vRanger.
>>>> Installation's a snap, and flexible recovery options mean your data is
>>>> safe,
>>>> secure and there when you need it. Data protection magic?
>>>> Nope - It's vRanger. Get your free trial download today.
>>>> http://p.sf.net/sfu/quest-sfdev2dev
>>>> _______________________________________________
>>>> Pytables-users mailing list
>>>> Pytables-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>>
>>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Simplify data backup and recovery for your virtual environment with
>> vRanger.
>> Installation's a snap, and flexible recovery options mean your data is
>> safe,
>> secure and there when you need it. Data protection magic?
>> Nope - It's vRanger. Get your free trial download today.
>> http://p.sf.net/sfu/quest-sfdev2dev
>> _______________________________________________
>> Pytables-users mailing list
>> Pytables-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
> --
> http://mae.ucdavis.edu/~biosport/jkm/
> Sports Biomechanics Lab <http://biosport.ucdavis.edu>, UC Davis
> Davis Bike Collective <http://www.davisbikecollective.org> Minister,
> Davis, CA
> BikeDavis.info
> Office: +01 530-752-2163
> Lab: +01 530-752-2235
> Home: +01 530-753-0794
>
>
>
> ------------------------------------------------------------------------------
> Simplify data backup and recovery for your virtual environment with
> vRanger.
> Installation's a snap, and flexible recovery options mean your data is
> safe,
> secure and there when you need it. Data protection magic?
> Nope - It's vRanger. Get your free trial download today.
> http://p.sf.net/sfu/quest-sfdev2dev
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
