Daniele Nicolodi wrote:
Dear John,
thank you very much for your hints. Partial reads and absolute
performances are not critical for my application. I think I'll give
packet tables a try. On the other hand, easily accessing the data from
Python and Matlab is a requirement.
I don't have experience with either, but in my attempts to interface my
own work with Octave, I found that it could *only* use scalar data
types, and I suspect Matlab may have the same restriction. Pytables is
perfectly capable of using compound types, however.
Are packet tables (or the underlying structures) accessible in an easy
way from Matlab? Python has the h5py library, which, if it is not
already capable of handling packet tables, should be not too difficult
to extend.
Packet tables are ultimately like any other dataset, as Scott Mitchell
indicated. Packet tables are more just a name for a high-level API that
makes it simple to build extensible datasets (obviously this can be done
in the low-level libraries that the H5PT interface is built on, but H5PT
makes the implementation easier). Unless, for some reason, the matlab
interface can't read datasets that have current dimensions != maximum
dimensions, I don't see why any interface wouldn't be able to read them
vs. an otherwise identical dataset.
A while back I investigated the kind of data structure used by PyTables,
which offers the kind of functionality I'm interested in. From my
aproximative knowledge of HDF5, I understood that it uses a "custom"
version of HDF5 Tables which support chunked writes. However I need
compatibility with Matlab and the ability of writing those files from a
C library. How such a structure compares with packed tables?
Thank you. Cheers,
I lack sufficient knowledge to answer that question, but I believe the
pytables author is active on this mailing list, so maybe he'll chime in
with pytables details. That said, with the exception of the scalar vs.
compound data type issue I'd already mentioned, I have a hard time
seeing how an HDF5 file written in one piece of software wouldn't be
readable in another. If the dataset is chunked vs. continuous, that
should be completely transparent to the reading application. I believe
that packet tables are chunked by definition as the library requires
that extensible datasets be chunked to allow for the extension.
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
