Barbara,
I've place a datafile A2003061013500.L2_LAC_AT110 on the ftp server as
well as a routine compress_memleak_multifiles.c that I hope will
demonstrate the problem.
It is actually a netcdf4 file but the HDF5 library is called to do the
reads and the example program uses only HDF5 calls.
It simulates multiple files by opening the same one five times. The
allocated memory leak is about 15M.
We are using hdf5-1.8.10-patch1.
Joel
On Fri, 28 Feb 2014, Barbara Jones wrote:
Hi Joel,
Which version of HDF5 are you using? Would you be able to send me a small
program or code snippet and a sample file to reproduce the issue? A similar
issue came up a while back (pre-HDF5-1.8.5) with a netCDF-4 user, but we never
determined what the issue was.
You can place the files on our local ftp server. Let me know (at
[email protected]) when they are there.
Here are the instructions:
---------------
ftp ftp.hdfgroup.uiuc.edu
Login as "anonymous"
Go to the pub/incoming/help/ directory:
cd pub/incoming/help
Type 'pwd' to see if you are in that location. (You will not be able to see the
contents of
the directory.):
pwd
Type 'bin' to put in binary mode:
bin
Place file in directory after you are in binary mode:
put xxx
Get out of ftp:
quit
--------------
Thanks!
-Barbara
=================
Barbara Jones
The HDF Helpdesk
The HDF Group
[email protected]
=================
-----Original Message-----
From: Hdf-forum [mailto:[email protected]] On Behalf Of Joel
Gales
Sent: Sunday, February 23, 2014 2:01 PM
To: [email protected]
Cc: Bryan Franz; sean bailey
Subject: [Hdf-forum] memory leak reading compressed files
I am reading a large number (186) of HDF5 compressed datafiles.
Eventually the allocated memory exceeds 20g and the program fails.
I've traced the increasing memory allocation to the H5Z_filter_deflate()
function in H5Zdeflate.c, specifically the lines:
nalloc *= 2;
if(NULL == (new_outbuf = H5MM_realloc(outbuf, nalloc))) {
(void)inflateEnd(&z_strm);
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, 0, "memory allocation failed for
deflate uncompression")
It appears as if this memory is never freed causing the memory leak.
Is this the desired behaviour or could there be a bug that prevents this memory
from being freed?
=========================
Joel Gales
Futuretech Corp.
SIMBIOS Code 616.0
Phone: (301) 286-1403
FAX: (301) 286-0268
Bin Globally Map Locally
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
=========================
Joel Gales
Futuretech Corp.
SIMBIOS Code 616.0
Phone: (301) 286-1403
FAX: (301) 286-0268
Bin Globally Map Locally
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
