Hello, Would it be possible for you to send us an example that demonstrates the problem? Could you please also send those two files to [email protected]?
It will also help if we know how many datasets you have in a data group when you see such behavior. Which version of the gzip library are you using? Which OS and compiler? Have you tried your application with the latest HDF5? Thank you! Elena ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elena Pourmal The HDF Group http://hdfgroup.org 1800 So. Oak St., Suite 203, Champaign IL 61820 217.531.6112 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On Nov 13, 2012, at 9:59 AM, ylegoc wrote: > Our instrument control software uses hdf5 files to store neutron acquisition > data files. > When the size of the "data" group is growing, we have random compressions. > Sometimes the dataset is compressed, sometimes not. Here is the dump of two > files containing the same dataset but with different resulting compression: > > Bad file : > > HDF5 "000028.nxs" { > GROUP "/" { > ATTRIBUTE "HDF5_Version" { > DATATYPE H5T_STRING { > STRSIZE 5; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > GROUP "entry0" { > ATTRIBUTE "NX_class" { > DATATYPE H5T_STRING { > STRSIZE 7; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > GROUP "data" { > ATTRIBUTE "NX_class" { > DATATYPE H5T_STRING { > STRSIZE 6; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > DATASET "data" { > DATATYPE H5T_STD_I32LE > DATASPACE SIMPLE { ( 384, 256, 1024 ) / ( 384, 256, 1024 ) } > STORAGE_LAYOUT { > CHUNKED ( 384, 256, 1024 ) > SIZE 402653184 (1.000:1 COMPRESSION) > } > FILTERS { > COMPRESSION DEFLATE { LEVEL 6 } > } > FILLVALUE { > FILL_TIME H5D_FILL_TIME_IFSET > VALUE 0 > } > ALLOCATION_TIME { > H5D_ALLOC_TIME_INCR > } > ATTRIBUTE "signal" { > DATATYPE H5T_STD_I32LE > DATASPACE SCALAR > } > } > } > > Correct file : > > HDF5 "000029.nxs" { > GROUP "/" { > ATTRIBUTE "HDF5_Version" { > DATATYPE H5T_STRING { > STRSIZE 5; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > GROUP "entry0" { > ATTRIBUTE "NX_class" { > DATATYPE H5T_STRING { > STRSIZE 7; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > GROUP "data" { > ATTRIBUTE "NX_class" { > DATATYPE H5T_STRING { > STRSIZE 6; > STRPAD H5T_STR_NULLTERM; > CSET H5T_CSET_ASCII; > CTYPE H5T_C_S1; > } > DATASPACE SCALAR > } > DATASET "data" { > DATATYPE H5T_STD_I32LE > DATASPACE SIMPLE { ( 384, 256, 1024 ) / ( 384, 256, 1024 ) } > STORAGE_LAYOUT { > CHUNKED ( 384, 256, 1024 ) > SIZE 139221680 (2.892:1 COMPRESSION) > } > FILTERS { > COMPRESSION DEFLATE { LEVEL 6 } > } > FILLVALUE { > FILL_TIME H5D_FILL_TIME_IFSET > VALUE 0 > } > ALLOCATION_TIME { > H5D_ALLOC_TIME_INCR > } > ATTRIBUTE "signal" { > DATATYPE H5T_STD_I32LE > DATASPACE SCALAR > } > } > } > > compression type : NX_COMP_LZW > hdf5 version 1.8.3 called by the Nexus library 4.3.0 > > Are there explanations for such random behaviour? Some solutions? > > > > > -- > View this message in context: > http://hdf-forum.184993.n3.nabble.com/hdf5-compression-problem-tp4025575.html > Sent from the hdf-forum mailing list archive at Nabble.com. > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
