Hi Martin,
On Fri, Apr 08, 2011 at 10:09:27AM +0100, Martin Galpin wrote:
> I am writing an packet table from C with a compound dataset in the form of:
>
> // C
> struct A
> {
> int x;
> int y;
> };
>
> struct Container
> {
> int offset;
> struct A element;
> }
Am I correct in assuming that you write that sructure out as a
"blob", i.e. not each element of the structure (and then the
elements of the substructure), probably using the sizeof()
the structure? If not please just forget about all of the
rest;-)
If that's the case then you're in trouble for several rea-
sons already when you just use C and don't even throw C#
into the mix. First of all already an 'int' can have dif-
ferent sizes on different machines, so the structure is
unreadable e.g. on a 64-bit machine when written on a
32-bit machine. Same holds for other types. Moreover some
machines store an int with the most-significant byte first,
others with the last-significant byte first etc. (that's
often called the "endian-ness" of a certain architecture).
HDF5 goes a long way to keep you out of trouble when you write
and read back again normal types like int's or double's by
storing the exact format the int's and double's are written
into the file and converts them to the format expected on the
machine the data are read in as necessary.
And then there is "structure padding". The compiler is free
to insert "padding bytes" between the elements of the struc-
ture (and after the last element). Those are needed to ensure
that the members are "properly aligned" in memory - at least
on some architectures e.g. int's can't start at arbitrary
addresses, so padding bytes can be required to get a member
to be aligned to a valid address. Therefor the sizeof() of a
structure is often larger then the sum of sizeof()'s of its
members.
The amount of padding needed can be different on different
architectures, but even on the same machine different com-
pilers are free to insert different amounts. So already a
structure, written out as a "blob" by a program compiled with
compiler A, may not readable with a program compiled using
compiler B (or, theoretically, even a different version of
compiler A).
> I am later reading the same dataset into .NET using HDF5DotNet. However, I
> cannot read the elements if the .NET structs have the same form (with a
> nested struct). However, it works correctly if I define a .NET struct as:
>
> // C# - works
> public struct Container
> {
> public int offset;
> public int x;
> public int y;
> }
By pure chance the layout of the elements in the C structure,
the sizeof()'s the elements and the endian-ness of the data
seem to be identical to what's used in this C# structure.
If you would try to copy the HDF file to another archtecture
(e.g. a 32-bit machine instead of a 64-bit machine or a small-
endian instead of a big-endian machine) and read it there you
will rather likely find that it doesn't work at all anymore.
> // C# - does not work
> public struct A
> {
> public int x;
> public int y;
> }
>
> public struct Container
> {
> public int offset;
> public A element;
> }
>
> Does anybody know any way to use the same form in C# (nested structs) as
> with C?
It's impossible to know since the layout of a C structure isn't
well defined. What you have to do is "serialize" the structure,
i.e. split it up into its compnents and write them out each
on its own. When reading it back in you then have to re-assemb-
le the components into a structure on th target machine.
But HDF5 can also help you with this when you use the "compound"
data type (H5T_COMPOUND). You have to tell it about the exact
layout and contents of your structure. That allows HDF5 to
"serialize" the structure for you (and re-assemble it when
reading it in). An introduction can be found e.g. at
http://www.hdfgroup.org/HDF5/Tutor/compound.html
And since a compound data type can even itself contain a com-
pound data type even writing out a structure that contains a
structure should work.
Best regards, Jens
--
\ Jens Thoms Toerring ________ [email protected]
\_______________________________ http://toerring.de
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
