Thanks, Allen!
If uninitialized data presents a problem for h5diff/h5dump, perhaps they could
indicate the data are initialized? We'll fix our code, but it would be nice to
have the h5 tools indicate this, rather than returning a random value.
Dave
On Nov 6, 2012, at 12:11 PM, Allen D Byrne wrote:
> The issue is uninitialized data in the dataset. Only the major
> index(1,0,0-2,0,0-3,0,0-...) has data.
>
> H5dump uses malloc to create the read buffer and the initial state of that
> buffer is platform dependent. By changing this to a calloc call, the issue is
> eliminated.
>
> The question becomes how should h5dump behave?
>
> Allen
>
> On Monday, October 29, 2012 12:18:47 PM Dave Wade-Stein wrote:
>> So it appears this problem with h5diff extends to h5dump as well.
>>
>> I have an h5 file, which, when h5dump'ed on Win32 and Linux, produces
>> different output.
>>
>> Incorrect output is shown in bold.
>>
>> Linux:
>> [dws@oxygen02 history]$ h5dump trajectory
>> ptclTrackTagFromAbsorber2s_History.h5 ...
>> DATASET "trajectory" {
>> DATATYPE H5T_IEEE_F64LE
>> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
>> H5S_UNLIMITED ) } DATA {
>> (0,0,0): 1.06204e-05, -4.79586e-07, 1.00833e+12, 1.87949e+08, 0, 8, 0,
>> (0,1,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,2,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,0,0): 1.07179e-05, -4.79568e-07, 1.01124e+12, 1.89595e+08, 0, 8, 0,
>> (1,1,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,2,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,3,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,10,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,11,0): 0, 0, 0, 0, 0, 0, 0,
>> ...
>>
>> Win32:
>> ...
>> DATASET "trajectory" {
>> DATATYPE H5T_IEEE_F64LE
>> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
>> H5S_UNLIMITED ) } DATA {
>> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0,
>> 8, (0,0,6): 0,
>> (0,1,0): 1.71412e+214, 1.73429e+185, 1.75754e+136, 3.44266e+218,
>> (0,1,4): 5.73569e+169, 0, 1.34682e-308,
>> (0,2,0): 1.91002e-308, 0, 0, 0, 0, 0, 0,
>> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
>> ...
>>
>> Oddly, h5dump produces still different output with the -d option. Here is
>> the output of h5dump -d trajectory, run on Win32:
>>
>> HDF5 "ptclTrackTagFromAbsorber2s_History.h5" {
>> DATASET "trajectory" {
>> DATATYPE H5T_IEEE_F64LE
>> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
>> H5S_UNLIMITED ) } DATA {
>> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0, 8,
>> 0, (0,1,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,2,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,0,0): 1.07179e-005, -4.79568e-007, 1.01124e+012, 1.89595e+008, 0, 8,
>> 0, (1,1,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,2,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,3,0): 1.37758e-308, 8.34708e-309, 4.17213e-308, 1.043e-308, 0, 0, 0,
>> (1,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,10,0): 0, 0, 0, 0, 0, 0, 0,
>> (1,11,0): 0, 0, 0, 0, 0, 0, 0,
>>
>> I've tried versions 1.8.5 through 1.8.9, and they are all wrong, but in
>> different ways!
>>
>> Here is the output from 1.8.5-patch1, run on the same file on Win32:
>>
>> DATASET "trajectory" {
>> DATATYPE H5T_IEEE_F64LE
>> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
>> H5S_UNLIMITED ) } DATA {
>> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0,
>> 8, (0,0,6): 0,
>> (0,1,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,2,0): 0, 0, 1.32116e-308, 1.38263e-304, 0, 0, 0,
>> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
>> (0,10,0): 0, 0, 1.32228e-308, 1.38263e-304, 0, 0, 0,
>> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
>>
>> This is an anomalous case for us, as it only appears on this one file, out
>> of 6200+ in our test suite. We can work around it for now, but it would be
>> great to know what it is we are doing when we create this file that is
>> causing the problem. It's small enough (144K) that I figured I'd just
>> attached it so that anyone who wants to look at the file may do so.
>>
>> Thanks!
>> Dave
>>
>>> I'm getting some output from h5diff which seems wrong.
>>>
>>> [dws@oxygen02 history]$ /usr/local/contrib/hdf5-1.8.8-par/bin/h5diff -v -p
>>> 1e-30 ptclTrackTagFromAbsorber1s_HistoryOld.h5
>>> /scr1_oxygen/dws/jenkins/workspace/vorpalall-build/qarnodes/oxygen/vptest
>>> s/vpresults-oxygen/history/ptclTrackTagFromAbsorber1s/ptclTrackTagFromAbso
>>> rber1s_HistoryOld.h5 /trajectory dataset: </trajectory> and </trajectory>
>>> size: [339x7x6] [339x7x6]
>>> position trajectory trajectory difference relative
>>> ------------------------------------------------------------------------
>>> [ 0 1 0 ] 1.678294506e+243 0 1.678294506e+243 1
>>> [ 0 1 1 ] 9.047791808e+164 0 9.047791808e+164 1
>>> [ 0 1 2 ] 4.78541875e+175 0 4.78541875e+175 1
>>> [ 0 1 3 ] 4.440349806e-38 0 4.440349806e-38 1
>>> [ 0 1 4 ] 4.545801862e+97 0 4.545801862e+97 1
>>> [ 0 1 5 ] 8.535925984e-307 0 8.535925984e-307 1
>>> [ 0 2 0 ] 3.952525167e-322 0 3.952525167e-322 1
>>> [ 0 2 1 ] 3.119382268e-319 0 3.119382268e-319 1
>>> [ 0 2 2 ] 9.698666729e-317 0 9.698666729e-317 1
>>> [ 0 2 3 ] 6.920039978e-310 0 6.920039978e-310 1
>>> [ 0 2 4 ] 4.193339982e+228 0 4.193339982e+228 1
>>> [ 0 2 5 ] 1.129002081e+277 0 1.129002081e+277 1
>>> [ 0 3 0 ] 1.355073239e+248 0 1.355073239e+248 1
>>> [ 0 3 2 ] 3.162020133e-322 0 3.162020133e-322 1
>>> [ 0 3 3 ] 3.162020133e-322 0 3.162020133e-322 1
>>> [ 0 3 5 ] 2.60988666e+180 0 2.60988666e+180 1
>>> [ 0 4 0 ] 4.193339982e+228 0 4.193339982e+228 1
>>> [ 0 4 1 ] 1.129002081e+277 0 1.129002081e+277 1
>>> [ 0 4 2 ] 1.355073239e+248 0 1.355073239e+248 1
>>> [ 0 4 5 ] 3.113058228e-319 0 3.113058228e-319 1
>>> [ 0 5 0 ] 4.824065618e+228 0 4.824065618e+228 1
>>> [ 0 5 1 ] 2.60988666e+180 0 2.60988666e+180 1
>>> [ 0 5 2 ] 4.193339982e+228 0 4.193339982e+228 1
>>> [ 0 5 3 ] 1.129002081e+277 0 1.129002081e+277 1
>>> [ 0 5 4 ] 1.355073239e+248 0 1.355073239e+248 1
>>>
>>> The diffs says that element (0,1,0) of the /trajectory dataset is
>>> 1.678294506e+243 (in bold above), but when I use h5dump to look at the
>>> dataset, I see 0s:
>>>
>>> ...
>>>
>>> DATASET "trajectory" {
>>>
>>> DATATYPE H5T_IEEE_F64LE
>>> DATASPACE SIMPLE { ( 339, 7, 6 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
>>> H5S_UNLIMITED ) } DATA {
>>> (0,0,0): 1.09772e-05, 1.007e+12, 0, 0, 8, 0,
>>> (0,1,0): 0, 0, 0, 0, 0, 0,
>>> (0,2,0): 0, 0, 0, 0, 0, 0,
>>> (0,3,0): 0, 0, 0, 0, 0, 0,
>>> (0,4,0): 0, 0, 0, 0, 0, 0,
>>> (0,5,0): 0, 0, 0, 0, 0, 0,
>>> (0,6,0): 0, 0, 0, 0, 0, 0,
>>> (1,0,0): 1.10752e-05, 1.00993e+12, 0, 0, 8, 0,
>>> (1,1,0): 0, 0, 0, 0, 0, 0,
>>> (1,2,0): 0, 0, 0, 0, 0, 0,
>>> (1,3,0): 0, 0, 0, 0, 0, 0,
>>> (1,4,0): 0, 0, 0, 0, 0, 0,
>>> (1,5,0): 0, 0, 0, 0, 0, 0,
>>> (1,6,0): 0, 0, 0, 0, 0, 0,
>>>
>>> ...
>>>
>>> Any ideas what's going on here. The datatype in both files is
>>> H5T_IEEE_F64LE, so I know this is not a datatype comparison issue.
>>>
>>> Thanks,
>>> Dave
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
