Thanks a lot for your very quick answer, that helped a lot and solved the problem!
I wanted now to use variable length datatypes. To test, I ran the example C code
from HDF5 (h5ex_t_vlen.c). When I try to open the produced file h5ex_t_vlen.h5
with pytables, I get a very similar warning as in the other example (see below).
Is it not possible to use variable length datatypes with pytables?
Unfortunately, I will need at least some similar construct. The data I
want to write
into hdf5 tables comes in as events. Each event consists of a number
of hits (which
is a c-struct), and each event has a different number of hits (1 to
64). Is it possible
to read such kind of data with pytables?
Thanks a lot again!
Best regards,
Michael
f=openFile("h5ex_t_vlen.h5")
In [3]: f.root
/usr/local/lib/python2.6/dist-packages/tables/group.py:379:
UserWarning: leaf ``/DS1`` is of an unsupported type; it will become
an ``UnImplemented`` node
% self._g_join(childName))
Out[3]:
/ (RootGroup) ''
children := ['DS1' (UnImplemented)]
In [4]: f.root.DS1
Out[4]:
/DS1 (UnImplemented(2,)) ''
NOTE: <The UnImplemented object represents a PyTables unimplemented
dataset present in the 'h5ex_t_vlen.h5' HDF5 file. If you
want to see this
kind of HDF5 dataset implemented in PyTables, please contact the
developers.>
2011/3/9 Francesc Alted <fal...@pytables.org>:
> A Wednesday 09 March 2011 11:08:28 Michael Rissi escrigué:
>> Hello Pytablers,
>>
>> In our readout software for a PET scanner, I write the data to an
>> HDF5 array of compound datatypes (in C++).
>> When I want to read this table in ipython with pytables, I get the
>> message:
>>
>>
>> In [11]: import tables
>>
>> In [11]: f=tables.openFile("Events_0012.h5")
>>
>> In [12]: f.root.dset
>> /usr/local/lib/python2.6/dist-packages/tables/path.py:102:
>> NaturalNameWarning: object name is not a valid Python identifier:
>> 'ToT '; it does not match the pattern ``^[a-zA-Z_][a-zA-Z0-9_]*$``;
>> you will not be able to use natural naming to access this object;
>> using ``getattr()`` will still work, though
>> NaturalNameWarning )
>> /usr/local/lib/python2.6/dist-packages/tables/group.py:1215:
>> UserWarning: problems loading leaf ``/dset``::
>>
>> table ``/dset``, column ``hits``: data type not understood
>>
>> The leaf will become an ``UnImplemented`` node.
>> % (self._g_join(childName), exc))
>> Out[12]:
>> /dset (UnImplemented(16384,)) ''
>> NOTE: <The UnImplemented object represents a PyTables unimplemented
>> dataset present in the 'Events_0012.h5' HDF5 file. If you
>> want to see this
>> kind of HDF5 dataset implemented in PyTables, please contact
>> the developers.>
>>
>>
>> I also attached the C++ code which writes the table.
>> Should I write the HDF5 file in a different way in order to read the
>> data with pytables?
>
> I've been looking at your table description, and the ``hits`` column is
> an array of compound types, and this is currently unsupported in
> PyTables.
>
> What you can do is to create the fields of the array in the ``hits``
> column at the top-level table description. Or, if you still want to set
> these fields apart, you can create them as nested fields of ``hits``
> (that is, ``hits`` will become a compound of compounds), which is fully
> supported.
>
> Hope this helps,
>
> --
> Francesc Alted
>
> ------------------------------------------------------------------------------
> Colocation vs. Managed Hosting
> A question and answer guide to determining the best fit
> for your organization - today and in the future.
> http://p.sf.net/sfu/internap-sfd2d
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
--
Dr. Michael Rissi
room ø385A
Sem Sælandsvei 24
N-0316 Oslo
Universitetet i Oslo
NORWAY
/************************************************************
This example shows how to read and write variable-length
datatypes to a dataset. The program first writes two
variable-length integer arrays to a dataset then closes
the file. Next, it reopens the file, reads back the data,
and outputs it to the screen.
This file is intended for use with HDF5 Library version 1.6
************************************************************/
#include "hdf5.h"
#include <stdio.h>
#include <stdlib.h>
#define FILE "h5ex_t_vlen.h5"
#define DATASET "DS1"
#define LEN0 3
#define LEN1 12
int
main (void)
{
hid_t file, filetype, memtype, space, dset;
/* Handles */
herr_t status;
hvl_t wdata[2], /* Array of vlen structures */
*rdata; /* Pointer to vlen structures */
hsize_t dims[1] = {2};
int *ptr,
ndims,
i, j;
/*
* Initialize variable-length data. wdata[0] is a countdown of
* length LEN0, wdata[1] is a Fibonacci sequence of length LEN1.
*/
wdata[0].len = LEN0;
ptr = (int *) malloc (wdata[0].len * sizeof (int));
for (i=0; i<wdata[0].len; i++)
ptr[i] = wdata[0].len - i; /* 3 2 1 */
wdata[0].p = (void *) ptr;
wdata[1].len = LEN1;
ptr = (int *) malloc (wdata[1].len * sizeof (int));
ptr[0] = 1;
ptr[1] = 1;
for (i=2; i<wdata[1].len; i++)
ptr[i] = ptr[i-1] + ptr[i-2]; /* 1 1 2 3 5 8 etc. */
wdata[1].p = (void *) ptr;
/*
* Create a new file using the default properties.
*/
file = H5Fcreate (FILE, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);
/*
* Create variable-length datatype for file and memory.
*/
filetype = H5Tvlen_create (H5T_STD_I32LE);
memtype = H5Tvlen_create (H5T_NATIVE_INT);
/*
* Create dataspace. Setting maximum size to NULL sets the maximum
* size to be the current size.
*/
space = H5Screate_simple (1, dims, NULL);
/*
* Create the dataset and write the variable-length data to it.
*/
dset = H5Dcreate (file, DATASET, filetype, space, H5P_DEFAULT);
status = H5Dwrite (dset, memtype, H5S_ALL, H5S_ALL, H5P_DEFAULT, wdata);
/*
* Close and release resources. Note the use of H5Dvlen_reclaim
* removes the need to manually free() the previously malloc'ed
* data.
*/
status = H5Dvlen_reclaim (memtype, space, H5P_DEFAULT, wdata);
status = H5Dclose (dset);
status = H5Sclose (space);
status = H5Tclose (filetype);
status = H5Tclose (memtype);
status = H5Fclose (file);
/*
* Now we begin the read section of this example. Here we assume
* the dataset has the same name and rank, but can have any size.
* Therefore we must allocate a new array to read in data using
* malloc().
*/
/*
* Open file and dataset.
*/
file = H5Fopen (FILE, H5F_ACC_RDONLY, H5P_DEFAULT);
dset = H5Dopen (file, DATASET);
/*
* Get dataspace and allocate memory for array of vlen structures.
* This does not actually allocate memory for the vlen data, that
* will be done by the library.
*/
space = H5Dget_space (dset);
ndims = H5Sget_simple_extent_dims (space, dims, NULL);
rdata = (hvl_t *) malloc (dims[0] * sizeof (hvl_t));
/*
* Create the memory datatype.
*/
memtype = H5Tvlen_create (H5T_NATIVE_INT);
/*
* Read the data.
*/
status = H5Dread (dset, memtype, H5S_ALL, H5S_ALL, H5P_DEFAULT, rdata);
/*
* Output the variable-length data to the screen.
*/
for (i=0; i<dims[0]; i++) {
printf ("%s[%u]:\n {",DATASET,i);
ptr = rdata[i].p;
for (j=0; j<rdata[i].len; j++) {
printf (" %d", ptr[j]);
if ( (j+1) < rdata[i].len )
printf (",");
}
printf (" }\n");
}
/*
* Close and release resources. Note we must still free the
* top-level pointer "rdata", as H5Dvlen_reclaim only frees the
* actual variable-length data, and not the structures themselves.
*/
status = H5Dvlen_reclaim (memtype, space, H5P_DEFAULT, rdata);
free (rdata);
status = H5Dclose (dset);
status = H5Sclose (space);
status = H5Tclose (memtype);
status = H5Fclose (file);
return 0;
}
h5ex_t_vlen.h5
Description: HDF file
------------------------------------------------------------------------------ Colocation vs. Managed Hosting A question and answer guide to determining the best fit for your organization - today and in the future. http://p.sf.net/sfu/internap-sfd2d
_______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
