Hi Francesc,
I can confirm that it does work with an older hdf5 version. I ran the
example that I attached earlier (problem.py) on my home machine, which
is still running an older version, and it does create the table.
gabr...@gabriel-desktop:~/usbdisk/programs$ python problem.py
1.6.6
2.0.4
gabr...@gabriel-desktop:~/usbdisk/programs$ ptdump -v test.h5
/ (RootGroup) ''
/test (Table(0L,)) ''
description := {
"signal": Float32Col(shape=(32, 3891), dflt=0.0, pos=0)}
byteorder := 'little'
chunkshape := (1L,)
gabr...@gabriel-desktop:~/usbdisk/programs$
As you can see hdf5 version is 1.6.6 (this is the standard hdf5 package
on Ubuntu). It is funny that it did not work on your machine with 1.6.5.
In any case, it is clear that I need to rethink how I am going store my
3D arrays.
Cheers, Gabriel
On Fri, 2009-02-20 at 22:03 +0100, Gabriel Beckers wrote:
> OK, thanks. I think in a previous version this was possible. Or am I
> getting something wrong? I have many files with tables of 1002 rows,
> with one column that holds a 32 x 3488 float32 matrix, for example:
>
> /bird1/IJ_5000/erp/lfp (Table(1002L,)) ''
> description := {
> "signal": Float32Col(shape=(32, 3488), dflt=0.0, pos=0)}
> byteorder := 'little'
> chunkshape := (1L,)
>
> I admit that it is a weird thing to do, but I didn't know a better
> solution.
>
> Here is my application:
>
> I store long 32-channel recordings (say 1.5 hours at 14000 Hz) of brain
> activity in a PyTables file. In this recording there are relatively
> short episodes (events, n=1002) where I want to study brain activity,
> across the 32 channels. Ideally I would 'cut out' these 1002 events from
> the recording (say 32 x 3488 samples per event) and store the result in
> a big 1002 x 32 x 3488 array for further analyses. This is no problem of
> course, but for my analyses I very often I need to make selections of
> events that are not sequential. Say, event numbers [3, 8, 23, 57, 576,
> 578]. But usually the sets are much bigger. I cannot access those with
> normal slicing in Array types. So as an alternative I stored the 32 x
> 3488 events in a 1002-row Table. Tables have a readCoordinates method
> that does exactly what I want. If there is a better solution I would be
> glad to learn about it.
>
> Many thanks, cheers,
> Gabriel
>
>
> On Fri, 2009-02-20 at 17:47 +0100, Francesc Alted wrote:
> > This seems a limitation in HDF5 itself, I'm afraid. Using HDF5 1.6.5
> > gives a little better explanation about this:
> >
> > #005: ../../../src/H5O.c line 2204 in H5O_new_mesg(): object header
> > message is too large (16k max)
> >
> > In fact, I was able to create a compound type with up to (16k - 3)
> > components. However, you are trying with one with 122k components
> (!).
> > I think that defining such a large data types is really
> > counter-productive, mainly because of performance reasons, so you
> > should avoid them.
> >
> > What are you trying to do? Perhaps using an EArray is your best bet
> for
> > this case.
> >
> > Cheers,
> >
>
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
