Hi Mark,
On Dec 3, 2010, at 4:00 PM, Mark Chaisson wrote:
> Hi, this is my first post to hdf-forum so apologies in advance for anything
> that is redundant that I've missed in previous posts. I have a process that
> is running for several hours opening and reading from many hdf files. The
> heap memory is constantly growing, and because all memory is returned upon
> exit, I suspect it is from heap fragmentation.
>
> Running my code using valgrind and massif to profile heap usage, it shows a
> large amount of memory allocated simply for getNumAttrs. Furthermore, many
> calls to read data are also accompanied by malloc's. When I run similar
> code that reads a txt version of the input, the memory does not climb as high.
>
> Are there any settings to get hdf to reuse buffers, or perform fewer mallocs?
Which version of HDF5 are you using? If you'd like to use valgrind to
analyze the memory use in the HDF5 library, you may want to use the
"--enable-using-memchecker" option for configure, to avoid the internal free
space memory allocator and call malloc() directly all the time.
Quincey
> thanks,
> -mark
>
> ->46.33% (21,936,989B) 0x515FA6B: H5FL_malloc (H5FL.c:203)
> | ->43.55% (20,622,405B) 0x5160049: H5FL_blk_malloc (H5FL.c:889)
> | | ->38.49% (18,226,752B) 0x50FA556: H5B_load (H5Bcache.c:130)
> | | | ->38.49% (18,226,752B) 0x510D418: H5C_protect (H5C.c:8175)
> | | | ->38.49% (18,226,752B) 0x50F4CFF: H5AC_protect (H5AC.c:1587)
> | | | ->38.37% (18,171,584B) 0x50F6407: H5B_get_info_helper (H5B.c:1989)
> | | | | ->38.37% (18,171,584B) 0x50F65DF: H5B_get_info_helper (H5B.c:2008)
> | | | | ->37.53% (17,771,008B) 0x50F65DF: H5B_get_info_helper
> (H5B.c:2008)
> | | | | | ->37.53% (17,771,008B) 0x50F7397: H5B_get_info (H5B.c:2058)
> | | | | | ->37.53% (17,771,008B) 0x51157C3: H5D_btree_idx_size
> (H5Dbtree.c:1321)
> | | | | | ->37.53% (17,771,008B) 0x5115E97: H5D_chunk_bh_info
> (H5Dchunk.c:4702)
> | | | | | ->37.53% (17,771,008B) 0x512D465: H5O_dset_bh_info
> (H5Doh.c:404)
> | | | | | ->37.53% (17,771,008B) 0x51B8C63: H5O_get_info
> (H5O.c:2827)
> | | | | | ->37.53% (17,771,008B) 0x5177CB1: H5G_loc_info_cb
> (H5Gloc.c:750)
> | | | | | ->37.53% (17,771,008B) 0x5184541:
> H5G_traverse_real (H5Gtraverse.c:886)
> | | | | | ->37.53% (17,771,008B) 0x5184ABB: H5G_traverse
> (H5Gtraverse.c:963)
> | | | | | ->37.53% (17,771,008B) 0x51777AA:
> H5G_loc_info (H5Gloc.c:795)
> | | | | | ->37.53% (17,771,008B) 0x51BCBC3:
> H5Oget_info (H5O.c:612)
> | | | | | ->37.53% (17,771,008B) 0x4E4F317:
> H5::H5Object::getNumAttrs() const (H5Object.cpp:248)
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
