Hello Mathieu,
AFAIK HDF5 will always reorder the data. I think this is what most
users want as they don't care/know how the data are chunked. Note that
the main purpose of chunking is to have reasonably fast access
independent of the order the array's data are traversed. The chunk shape
should be such that it serves the expected access patterns reasonably
well.
Using chunking for a sparse data array is very valid.
However, if you don't want the data to be reordered, you have to access
them chunk-wise. Thus define your hyperslab such that it matches the
chunk boundaries.
Getting data in Z-major order can also be achieved by defining your
hyperslabs correctly.
Note that HDF5 has a lot of overhead when using many calls with small
hyperslabs.
Cheers,
Ger
>>> <mathieu.westp...@obs.ujf-grenoble.fr> 5/15/2012 12:22 PM >>>
Hello
I have a chunked dataset of size 20 20 10.
Chunk size are : 10 10 1.
le'ts say i read an hyperslab of data defined by:
start={0 0 0}
count={4 4 4}
I read it into a 1-D array.
and i get
X Y Z
0 0 0
0 0 1
0 0 2
0 0 3
0 0 4
0 1 0
0 1 1
0 1 2
0 1 3
0 1 4
0 2 0
0 2 1
...
0 4 3
0 4 4
1 0 0
1 0 1
...
If i understand well, a chunked dataset is read chunk by chunk, so i
cannot understand how i can obtain this kind of order without
reordering completelly the data. A unique chunk cannot contain two
diferent Z..
So,
Is this normal? do HDF5 reorder data (wasting time and ressources )?
Is there anyway to control this order? (not row-major, but let's say
Z-major..)
Thanks for helping.
Mathieu
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
