Hi Tom,
Looks like you are working with a thread-safe build of HDF5, which is
unnecessary for the command-line tools. You could rebuild the HDF5
distribution (I would suggest moving up to 1.8.8 or the 1.8.9 prerelease)
without the thread-safe configure flag, and that should get rid of the mutex
issues.
Quincey
On Apr 25, 2012, at 9:28 AM, tom fogal wrote:
> I am getting really awful performance using 'h5dump' to dump a scalar field
> as a binary file. It takes literally hours, whereas an h5copy -f ref takes
> just under 2 minutes, and a simple 'cp' is a little bit quicker than that.
>
> My file header is reproduced below [1]. I am using
>
> h5dump -b LE -d /C00 -o outfile.raw infile.h5
>
> to convert. For comparison, 'h5copy' is run thusly:
>
> h5copy -s C00 -d C00 -i infile.h5 -o ./testing.h5 -v -f ref
>
> While running, h5dump pegs a core at 98+% CPU usage.
>
> I've tried attaching gdb to the process while it's running, so that I can
> obtain some poor-man's profiling. One popular stacktrace is appended below
> [2]. I guess it's locking and unlocking a mutex constantly? Other traces I
> have seen multiple times: H5I_object_verify called from H5Tequal;
> __pthread_setcancelstate from H5TS_cancel_count_inc from H5Tequal;
> __pthread_mutex_lock from H5TS_mutex_lock from H5open; H5T_cmp from H5Tequal
> (rarely).
>
> If locking is indeed the problem, can I disable it at runtime somehow? These
> files are only being accessed by one process at a time, h5dump isn't even
> multithreaded anyway, and furthermore the access is purely read-only.
>
> I am using HDF5 1.8.4. Please enlighten me as to how I can get reasonable
> performance out of these files.
>
> Thanks,
>
> -tom
>
> [1]
> $ h5dump -p -H TS_2011_12_26/TS_C00_0_16.h5
> HDF5 "TS_C00_0_16.h5" {
> GROUP "/" {
> DATASET "C00" {
> DATATYPE H5T_STD_U16LE
> DATASPACE SIMPLE { ( 301, 2550, 2550 ) / ( 301, 2550, 2550 ) }
> STORAGE_LAYOUT {
> CONTIGUOUS
> SIZE 3914505000
> OFFSET 1400
> }
> FILTERS {
> NONE
> }
> FILLVALUE {
> FILL_TIME H5D_FILL_TIME_IFSET
> VALUE 0
> }
> ALLOCATION_TIME {
> H5D_ALLOC_TIME_LATE
> }
> }
> }
> }
>
> [2]
> (gdb) bt
> #0 __pthread_mutex_lock (mutex=0x7ff2d1e7fac8) at pthread_mutex_lock.c:47
> #1 0x00007ff2d1bf67d6 in H5TS_mutex_unlock () from /usr/lib/libhdf5.so.6
> #2 0x00007ff2d19105b8 in H5open () from /usr/lib/libhdf5.so.6
> #3 0x0000000000420501 in ?? ()
> #4 0x000000000041fbf6 in ?? ()
> #5 0x0000000000416e7f in ?? ()
> #6 0x000000000041c856 in ?? ()
> #7 0x000000000041cea9 in ?? ()
> #8 0x000000000040abaf in ?? ()
> #9 0x000000000040a20d in ?? ()
> #10 0x000000000040d3c6 in ?? ()
> #11 0x000000000040f387 in ?? ()
> #12 0x00007ff2d156530d in __libc_start_main (main=0x40eae4, argc=8,
> ubp_av=0x7fffa529e648, init=<optimized out>, fini=<optimized out>,
> rtld_fini=<optimized out>, stack_end=0x7fffa529e638) at libc-start.c:226
> #13 0x0000000000405349 in ?? ()
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
