Hi Joel, Which version of HDF5 are you using? Would you be able to send me a small program or code snippet and a sample file to reproduce the issue? A similar issue came up a while back (pre-HDF5-1.8.5) with a netCDF-4 user, but we never determined what the issue was.
You can place the files on our local ftp server. Let me know (at [email protected]) when they are there. Here are the instructions: --------------- ftp ftp.hdfgroup.uiuc.edu Login as "anonymous" Go to the pub/incoming/help/ directory: cd pub/incoming/help Type 'pwd' to see if you are in that location. (You will not be able to see the contents of the directory.): pwd Type 'bin' to put in binary mode: bin Place file in directory after you are in binary mode: put xxx Get out of ftp: quit -------------- Thanks! -Barbara ================= Barbara Jones The HDF Helpdesk The HDF Group [email protected] ================= -----Original Message----- From: Hdf-forum [mailto:[email protected]] On Behalf Of Joel Gales Sent: Sunday, February 23, 2014 2:01 PM To: [email protected] Cc: Bryan Franz; sean bailey Subject: [Hdf-forum] memory leak reading compressed files I am reading a large number (186) of HDF5 compressed datafiles. Eventually the allocated memory exceeds 20g and the program fails. I've traced the increasing memory allocation to the H5Z_filter_deflate() function in H5Zdeflate.c, specifically the lines: nalloc *= 2; if(NULL == (new_outbuf = H5MM_realloc(outbuf, nalloc))) { (void)inflateEnd(&z_strm); HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, 0, "memory allocation failed for deflate uncompression") It appears as if this memory is never freed causing the memory leak. Is this the desired behaviour or could there be a bug that prevents this memory from being freed? ========================= Joel Gales Futuretech Corp. SIMBIOS Code 616.0 Phone: (301) 286-1403 FAX: (301) 286-0268 Bin Globally Map Locally _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
