Hi John,
On May 4, 2011, at 2:10 AM, Biddiscombe, John A. wrote:
> If I know that I want to attach a couple of arrays as attributes, and these
> arrays may be say 1024 elements of double precision max. (say 8kB). The
> datasets themselves might be xGB.
>
> 1) Can I tell hdf5 to reserve a minimum size in the object header so that I
> know in advance that the attributes will fit (or does the object header only
> contain pointers to other structures anyway)
This feature isn't available through the public API currently. (I have
thought about exposing the internal part of the library that would allow it,
but it seemed like it would only be used by a very small portion of the user
base)
> 2) Is there any performance advantage in terms of fewer metadata transaction
> to using attributes over datasets in a parallel IO context. (file is written
> in parallel, only rank zero needs to actually write the attributes)
Hmm, as usual, it depends. :-) Are all the processes going to be
accessing the attribute? If not, you could create an attribute with an object
reference to an auxiliary dataset, and then read that in dataset in when needed.
Quincey
> Question motivated by ....
>
> From the docs ...
> "The HDF5 format and I/O library are designed with the assumption that
> attributes are small datasets. They are always stored in the object header of
> the object they are attached to. Because of this, large datasets should not
> be stored as attributes. How large is "large" is not defined by the library
> and is up to the user's interpretation. (Large datasets with metadata can be
> stored as supplemental datasets in a group with the primary dataset.)"
>
> Thanks
>
> JB
>
> --
> John Biddiscombe, email:biddisco @ cscs.ch
> http://www.cscs.ch/
> CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07
> Via Cantonale, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82
>
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
