Mathieu, (simple) HDF5 datasets are logically dense arrays. With chunking you can mimic sparse storage, but you loose track of where the non-zero entries are. If you need that kind of information, you have to keep track of that with, for example, a region reference or emulate Compressed Row Storage or other sparse format directly with more than one dataset.
Best, G. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: Friday, May 11, 2012 5:24 AM To: [email protected] Subject: Re: [Hdf-forum] Read All Available Chunk in a Dataset. Thank you ! So as i understand there is no way to automatically select availlable data without knowing where it is. WHen i read the data, i normally dont know wich part of the dataset is full of data and wich part is not, so i will have to store an hyperslab wich describe where the available data is. Thanks for your help Mathieu Quoting Gerd Heber <[email protected]>: > That's why I asked you to define the smallest hyperslab or union of > hyperslabs, or point selections that cover you chunks and read just > those. The library will locate just those chunks (tree traversal, not > linear) and deal with them. > Have a look at section 7.4. Dataspaces and Data Transfer in the user > guide! > > Best, G. > > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] > On Behalf Of Mathieu westphal > Sent: Thursday, May 10, 2012 8:15 AM > To: HDF Users Discussion List > Subject: Re: [Hdf-forum] Read All Available Chunk in a Dataset. > > > > On 05/10/2012 02:01 PM, Gerd Heber wrote: >> Mathieu, don't worry about the chunks. Just read/write what you need >> in terms of the dataspace (hyperslab, point selection...) and the >> library will determine for you which chunks are touched >> (read/written) by the operation. >> It'll just return fill values for un-allocated/partially filled chunks. > Yep you're right! But the time to read the data is linear with the > size of all the Dataset, not only of the size of available data ! > My real dataset is really sparse ! i need to read all available value > without taking time to read unavailable value. > Is there anyway to select only real value? > > Anyway how to know how many value are really read from dataset? > > Thanks > > Mathieu > >> >> Best, G. >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] >> On Behalf Of Mathieu westphal >> Sent: Thursday, May 10, 2012 6:46 AM >> To: [email protected] >> Subject: [Hdf-forum] Read All Available Chunk in a Dataset. >> >> I have a dataset of size 16*16 >> I write Chunk of Data of size 4*4. >> >> Some data aren't available so i wrote only 3 chunk on my dataset. >> >> How to read only the 3 available chunk? >> >> Thanks >> >> Mathieu >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >> >> >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
