The tools lead decided that h5dump (h5diff) should use calloc calls. That
solution is now in process.
Allen
On Tuesday, November 06, 2012 12:50:33 PM Dave Wade-Stein wrote:
> Thanks, Allen!
>
> If uninitialized data presents a problem for h5diff/h5dump, perhaps they
> could indicate the data are initialized? We'll fix our code, but it would
> be nice to have the h5 tools indicate this, rather than returning a random
> value.
>
> Dave
>
> On Nov 6, 2012, at 12:11 PM, Allen D Byrne wrote:
> > The issue is uninitialized data in the dataset. Only the major
> > index(1,0,0-2,0,0-3,0,0-...) has data.
> >
> > H5dump uses malloc to create the read buffer and the initial state of that
> > buffer is platform dependent. By changing this to a calloc call, the issue
> > is eliminated.
> >
> > The question becomes how should h5dump behave?
> >
> > Allen
> >
> > On Monday, October 29, 2012 12:18:47 PM Dave Wade-Stein wrote:
> >> So it appears this problem with h5diff extends to h5dump as well.
> >>
> >> I have an h5 file, which, when h5dump'ed on Win32 and Linux, produces
> >> different output.
> >>
> >> Incorrect output is shown in bold.
> >>
> >> Linux:
> >> [dws@oxygen02 history]$ h5dump trajectory
> >> ptclTrackTagFromAbsorber2s_History.h5 ...
> >>
> >> DATASET "trajectory" {
> >>
> >> DATATYPE H5T_IEEE_F64LE
> >> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
> >>
> >> H5S_UNLIMITED ) } DATA {
> >>
> >> (0,0,0): 1.06204e-05, -4.79586e-07, 1.00833e+12, 1.87949e+08, 0, 8,
> >> 0,
> >>
> >> (0,1,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> (0,2,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,0,0): 1.07179e-05, -4.79568e-07, 1.01124e+12, 1.89595e+08, 0, 8,
> >> 0,
> >>
> >> (1,1,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> (1,2,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,3,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,10,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,11,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> ...
> >>
> >> Win32:
> >> ...
> >>
> >> DATASET "trajectory" {
> >>
> >> DATATYPE H5T_IEEE_F64LE
> >> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
> >>
> >> H5S_UNLIMITED ) } DATA {
> >>
> >> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0,
> >>
> >> 8, (0,0,6): 0,
> >>
> >> (0,1,0): 1.71412e+214, 1.73429e+185, 1.75754e+136, 3.44266e+218,
> >> (0,1,4): 5.73569e+169, 0, 1.34682e-308,
> >> (0,2,0): 1.91002e-308, 0, 0, 0, 0, 0, 0,
> >> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> ...
> >>
> >> Oddly, h5dump produces still different output with the -d option. Here is
> >> the output of h5dump -d trajectory, run on Win32:
> >>
> >> HDF5 "ptclTrackTagFromAbsorber2s_History.h5" {
> >> DATASET "trajectory" {
> >>
> >> DATATYPE H5T_IEEE_F64LE
> >> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
> >>
> >> H5S_UNLIMITED ) } DATA {
> >>
> >> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0, 8,
> >>
> >> 0, (0,1,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> (0,2,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,10,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,0,0): 1.07179e-005, -4.79568e-007, 1.01124e+012, 1.89595e+008, 0, 8,
> >>
> >> 0, (1,1,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> (1,2,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,3,0): 1.37758e-308, 8.34708e-309, 4.17213e-308, 1.043e-308, 0, 0, 0,
> >> (1,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,10,0): 0, 0, 0, 0, 0, 0, 0,
> >> (1,11,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> I've tried versions 1.8.5 through 1.8.9, and they are all wrong, but in
> >> different ways!
> >>
> >> Here is the output from 1.8.5-patch1, run on the same file on Win32:
> >> DATASET "trajectory" {
> >>
> >> DATATYPE H5T_IEEE_F64LE
> >> DATASPACE SIMPLE { ( 218, 12, 7 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
> >>
> >> H5S_UNLIMITED ) } DATA {
> >>
> >> (0,0,0): 1.06204e-005, -4.79586e-007, 1.00833e+012, 1.87949e+008, 0,
> >>
> >> 8, (0,0,6): 0,
> >>
> >> (0,1,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,2,0): 0, 0, 1.32116e-308, 1.38263e-304, 0, 0, 0,
> >> (0,3,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,4,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,5,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,6,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,7,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,8,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,9,0): 0, 0, 0, 0, 0, 0, 0,
> >> (0,10,0): 0, 0, 1.32228e-308, 1.38263e-304, 0, 0, 0,
> >> (0,11,0): 0, 0, 0, 0, 0, 0, 0,
> >>
> >> This is an anomalous case for us, as it only appears on this one file,
> >> out
> >> of 6200+ in our test suite. We can work around it for now, but it would
> >> be
> >> great to know what it is we are doing when we create this file that is
> >> causing the problem. It's small enough (144K) that I figured I'd just
> >> attached it so that anyone who wants to look at the file may do so.
> >>
> >> Thanks!
> >> Dave
> >>
> >>> I'm getting some output from h5diff which seems wrong.
> >>>
> >>> [dws@oxygen02 history]$ /usr/local/contrib/hdf5-1.8.8-par/bin/h5diff -v
> >>> -p
> >>> 1e-30 ptclTrackTagFromAbsorber1s_HistoryOld.h5
> >>> /scr1_oxygen/dws/jenkins/workspace/vorpalall-build/qarnodes/oxygen/vptes
> >>> t
> >>> s/vpresults-oxygen/history/ptclTrackTagFromAbsorber1s/ptclTrackTagFromAb
> >>> so
> >>> rber1s_HistoryOld.h5 /trajectory dataset: </trajectory> and
> >>> </trajectory>
> >>> size: [339x7x6] [339x7x6]
> >>> position trajectory trajectory difference relative
> >>> ------------------------------------------------------------------------
> >>> [ 0 1 0 ] 1.678294506e+243 0 1.678294506e+243 1
> >>> [ 0 1 1 ] 9.047791808e+164 0 9.047791808e+164 1
> >>> [ 0 1 2 ] 4.78541875e+175 0 4.78541875e+175 1
> >>> [ 0 1 3 ] 4.440349806e-38 0 4.440349806e-38 1
> >>> [ 0 1 4 ] 4.545801862e+97 0 4.545801862e+97 1
> >>> [ 0 1 5 ] 8.535925984e-307 0 8.535925984e-307 1
> >>> [ 0 2 0 ] 3.952525167e-322 0 3.952525167e-322 1
> >>> [ 0 2 1 ] 3.119382268e-319 0 3.119382268e-319 1
> >>> [ 0 2 2 ] 9.698666729e-317 0 9.698666729e-317 1
> >>> [ 0 2 3 ] 6.920039978e-310 0 6.920039978e-310 1
> >>> [ 0 2 4 ] 4.193339982e+228 0 4.193339982e+228 1
> >>> [ 0 2 5 ] 1.129002081e+277 0 1.129002081e+277 1
> >>> [ 0 3 0 ] 1.355073239e+248 0 1.355073239e+248 1
> >>> [ 0 3 2 ] 3.162020133e-322 0 3.162020133e-322 1
> >>> [ 0 3 3 ] 3.162020133e-322 0 3.162020133e-322 1
> >>> [ 0 3 5 ] 2.60988666e+180 0 2.60988666e+180 1
> >>> [ 0 4 0 ] 4.193339982e+228 0 4.193339982e+228 1
> >>> [ 0 4 1 ] 1.129002081e+277 0 1.129002081e+277 1
> >>> [ 0 4 2 ] 1.355073239e+248 0 1.355073239e+248 1
> >>> [ 0 4 5 ] 3.113058228e-319 0 3.113058228e-319 1
> >>> [ 0 5 0 ] 4.824065618e+228 0 4.824065618e+228 1
> >>> [ 0 5 1 ] 2.60988666e+180 0 2.60988666e+180 1
> >>> [ 0 5 2 ] 4.193339982e+228 0 4.193339982e+228 1
> >>> [ 0 5 3 ] 1.129002081e+277 0 1.129002081e+277 1
> >>> [ 0 5 4 ] 1.355073239e+248 0 1.355073239e+248 1
> >>>
> >>> The diffs says that element (0,1,0) of the /trajectory dataset is
> >>> 1.678294506e+243 (in bold above), but when I use h5dump to look at the
> >>> dataset, I see 0s:
> >>>
> >>> ...
> >>>
> >>> DATASET "trajectory" {
> >>>
> >>> DATATYPE H5T_IEEE_F64LE
> >>> DATASPACE SIMPLE { ( 339, 7, 6 ) / ( H5S_UNLIMITED, H5S_UNLIMITED,
> >>> H5S_UNLIMITED ) } DATA {
> >>> (0,0,0): 1.09772e-05, 1.007e+12, 0, 0, 8, 0,
> >>> (0,1,0): 0, 0, 0, 0, 0, 0,
> >>> (0,2,0): 0, 0, 0, 0, 0, 0,
> >>> (0,3,0): 0, 0, 0, 0, 0, 0,
> >>> (0,4,0): 0, 0, 0, 0, 0, 0,
> >>> (0,5,0): 0, 0, 0, 0, 0, 0,
> >>> (0,6,0): 0, 0, 0, 0, 0, 0,
> >>> (1,0,0): 1.10752e-05, 1.00993e+12, 0, 0, 8, 0,
> >>> (1,1,0): 0, 0, 0, 0, 0, 0,
> >>> (1,2,0): 0, 0, 0, 0, 0, 0,
> >>> (1,3,0): 0, 0, 0, 0, 0, 0,
> >>> (1,4,0): 0, 0, 0, 0, 0, 0,
> >>> (1,5,0): 0, 0, 0, 0, 0, 0,
> >>> (1,6,0): 0, 0, 0, 0, 0, 0,
> >>>
> >>> ...
> >>>
> >>> Any ideas what's going on here. The datatype in both files is
> >>> H5T_IEEE_F64LE, so I know this is not a datatype comparison issue.
> >>>
> >>> Thanks,
> >>> Dave
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
