Hi Quincey,
Quincey Koziol wrote:
Hi Mathis,
On Feb 18, 2010, at 10:13 AM, Quincey Koziol wrote:
Hi Mathis,
On Feb 17, 2010, at 6:55 AM, Mathis Rosenhauer wrote:
Hello,
I have a strange error where HDF5 files written with NetCDF4 and HDF5-1.8.4 on a big endian machine
(Power6) could not be read on little endian and vice versa. This isn't a NetCDF issue since pure
HDF5 files written with the attached program show the same behaviour. It turned out that the
"Fractal Heap ID for Managed Objects" in the "Version 2 B-tree, Type 8 Record
Layout" isn't correctly written on big endian. The ID is constructed from version, offset, and
length with the macro
#define H5HF_MAN_ID_ENCODE(i, h, o, l) \
*(i) = H5HF_ID_VERS_CURR | H5HF_ID_TYPE_MAN; \
(i)++; \
UINT64ENCODE_VAR((i), (o), (h)->heap_off_size); \
UINT64ENCODE_VAR((i), (l), (h)->heap_len_size)
so offset and length are already encoded. Prior to actually writing the ID, it
is encoded again, though:
In H5A_dense_btree2_name_encode():
UINT64ENCODE(raw, nrecord->id);
And this breaks on big endian. Just copying nrecord->id to raw works for me but
I don't know if this covers all use cases of H5A_dense_btree2_name_encode().
Hmm, this may be related, but probably isn't the root cause of this
problem, since there's a corresponding decode macro for the ID in
H5A_dense_btree2_name_decode(). (And the H5HF_MAN_ID_DECODE() macro looks
correct also).
On second thought, I think you are correct, this is the problem. Sorry
for my earlier response - we are working on putting a fix into the library
right now.
It works perfetly right now if you stay on the same architecture because
the de/encoding is symmetric ;). One problem with a fix will be that you
won't be able to read old files from BE machines with newer hdf5
releases. People here who are preparing the CMIP5 runs (they discovered
the problem) tell me that they don't yet have hdf5 files they want to
keep. But that's probably an exception. So it might be good to have a
tool to fix the old files.
Cheers,
Mathis
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
