If I know that I want to attach a couple of arrays as attributes, and these arrays may be say 1024 elements of double precision max. (say 8kB). The datasets themselves might be xGB.
1) Can I tell hdf5 to reserve a minimum size in the object header so that I know in advance that the attributes will fit (or does the object header only contain pointers to other structures anyway) 2) Is there any performance advantage in terms of fewer metadata transaction to using attributes over datasets in a parallel IO context. (file is written in parallel, only rank zero needs to actually write the attributes) Question motivated by .... >From the docs ... "The HDF5 format and I/O library are designed with the assumption that attributes are small datasets. They are always stored in the object header of the object they are attached to. Because of this, large datasets should not be stored as attributes. How large is "large" is not defined by the library and is up to the user's interpretation. (Large datasets with metadata can be stored as supplemental datasets in a group with the primary dataset.)" Thanks JB -- John Biddiscombe, email:biddisco @ cscs.ch http://www.cscs.ch/ CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07 Via Cantonale, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
