Hi John,
On Mar 11, 2011, at 9:56 AM, John Knutson wrote:
> I have a program that is a lot more complicated than I like (sadly, there's
> not a lot I can do about that), that I've recently converted to using HDF as
> its data store (from flat files). It also uses WebSphere MQ middleware to
> publish the results from the processing it does, so there's a bit of a loop.
> Program 1 generates data and publishes it, program 2 gets that published data
> and writes it to an HDF file, which program 1 in turn will occasionally read.
>
> There's a lot of data involved, so using an internal store isn't really an
> option, and the HDF structure gives the advantage of being able to quickly
> find exactly the data that's needed at the time. Other programs also use the
> data being published by program 1, so removing the publication wouldn't be an
> option.
>
> Anyway, hopefully that's sufficient background. The problem I'm seeing at
> the moment is that program 1 eventually will try to read some data from the
> HDF file that will fail in:
> H5Z_filter_deflate(): inflate() failed
> major: Data filters
> minor: Unable to initialize object
>
> This continues until memory runs out.
>
> The fact that 1) a read is failing when all other programs have not exhibited
> this problem since I modified program 2 to be very diligent about flushing
> its data to disc regularly, and 2) memory is consumed on repeated calls to
> the function in program 1 that's supposed to read the HDF file until it runs
> out; makes it look like I have the file open and the HDF library is caching
> the file's superblock, which is causing it to fail to read the data.
>
> I'm definitely closing the HDF5 object handles used in that function and in
> all the other functions I've checked so far, so my question is this: how can
> I get the library to give me a list of open hids? I've looked through the
> library API documentation and found H5Inmembers, which appears to provide the
> ability to determine *how many* identifiers of a given type are open, and I
> found H5Fget_obj_ids which appears to give one the ability to find all open
> object IDs given an open file ID, but I don't see a way to start from nothing
> and get open file IDs, object, group, link, etc. IDs.
>
> Any help would be appreciated :-)
If you pass in H5F_OBJ_ALL for the file_id parameter (and H5F_OBJ_ALL
for the types parameter) of H5Fget_obj_ids(), it will give you a list of all
the open file IDs, along with the list of other open IDs.
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
