Hi, I was not aware of the reference of being a type of variable length.
So I assume it is the path string leading to the varying length? Naiv spoken I could imagine of having some hdf internal mapping of reference paths to an unique - fixed size - id to store a reference path only once. And in the references dataset the id would be used instead with getting the compression to work there then ... ? In principle this should be the same as you propose to implement on application layer, thanks for that. I will go this way then. Unfortunately this will double application calls (instead of internal ones) for each read/write of data ... :-( Best regards, Johannes Stamminger On Di, 2010-04-20 at 08:50 -0500, Elena Pourmal wrote: > Hi Johannes, > > As with other variable type data references cannot be compressed since > internally they are stored in the heaps while dataset itself stores pointers > to the data in a file. If size is a problem, you may come up with your own > set of indices stored in a dataset (for example, reference to a hyperslab can > be stored as 2 n-dim vectors of corner coordinates, or something like this) > and use compression. > > Elena > > On Apr 20, 2010, at 2:25 AM, Stamminger, Johannes wrote: > > > Hi, > > > > just a question related to this topic concerning the filesize: last week > > I played around with refernces a little bit. I created a dataset of > > 1-dim packet table of references. With 1,1 Mio references to hyperslab > > regions of other packet tables within the same file. And this increased > > the hdf file size dramatically. Seems to me as if compression does not > > work (well) for such refs ... or would you expect different result and I > > missed something there? > > > > Best regards, > > Johannes Stamminger
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
