Hi Mark,
On Nov 9, 2010, at 11:01 AM, Mark Miller wrote:
> Hi Quincey,
>
> Thanks for the follow-up. One question; what are the
> functions/properties by which a plugin might wind up ENABLING caching of
> opened file handles?
There's only the new one in the RFC: H5Pset_elink_file_cache_size()
And that routine is only [currently] planned to enable caching of files opened
through external links.
Quincey
> Mark
>
> On Tue, 2010-11-09 at 05:06 -0800, Quincey Koziol wrote:
>> Hi Mark,
>>
>> On Nov 8, 2010, at 9:12 AM, Mark Miller wrote:
>>
>>> On the topic of caching opened files (external or otherwise), I have a
>>> question.
>>>
>>> In VisIt, we have many different 'flavors' of HDF5 files we read. A
>>> 'flavor' is a particular structure (groups/dataset/attribute names) of
>>> HDF5 file. We have a separate reader plugin for each flavor.
>>>
>>> We don't always know ahead of time which plugin should be used to open a
>>> given HDF5 file. So, we can wind up iterating over several, each
>>> attempting some HDF5 operations on the file and, perhaps, throwing
>>> exceptions (VisIt exceptions not HDF5 exceptions) and then moving onto
>>> the next HDF5 reader plugin to try.
>>>
>>> Here is my question. If one reader plugin calls H5Fopen and that
>>> succeeds but later in the plugin's attempt to read the file, it throws a
>>> VisIt exception to move onto the next plugin, what if it also does NOT
>>> call H5Fclose? Will that potentially negatively affect the next reader
>>> plugin which will also use H5Fopen to open that same file? Also, how
>>> does fact that these plugins are operating as shared libs effect things,
>>> in particular, any HDF5 library wide global data such as the list of
>>> cached files it has open?
>>
>> Well, this document describes a method of caching files opened through
>> external links, not files opened directly with H5Fopen(). Also, by default,
>> the caching is disabled, so applications won't get a surprise. And, it's
>> enabled on a per-file basis, not a global basis, again to avoid surprises
>> for applications. So, I don't think it'll affect the situation you describe
>> above with VisIt. Of course, you should close the files you open with
>> H5Fopen() or else they will remain open and chew up resources, etc. :-) I
>> don't think the shared library aspect will have any effect.
>>
>> Quincey
>>
>>> Mark
>>>
>>> On Mon, 2010-11-08 at 07:05, Quincey Koziol wrote:
>>>> Hi all,
>>>> I've post a new RFC describing a feature we are planning to add to the
>>>> HDF5 library that will cache open files from external links:
>>>>
>>>> http://BLOCKEDBLOCKEDwww.BLOCKEDBLOCKEDhdfgroup.uiuc.edu/RFC/HDF5/ExternalLinkFileAccessProperty/CacheExternalLinkFileOpens.pdf
>>>>
>>>> If you are using external links and would like to comment on this
>>>> performance enhancement, please do so.
>>>>
>>>> Thanks,
>>>> Quincey
>>>>
>>>>
>>>> _______________________________________________
>>>> Hdf-forum is for HDF software users discussion.
>>>> [email protected]
>>>> http://BLOCKEDBLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>> --
>>> Mark C. Miller, Lawrence Livermore National Laboratory
>>> ================!!LLNL BUSINESS ONLY!!================
>>> [email protected] urgent: [email protected]
>>> T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
>>>
>>>
>>> _______________________________________________
>>> Hdf-forum is for HDF software users discussion.
>>> [email protected]
>>> http://BLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>> http://BLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>
> --
> Mark C. Miller, Lawrence Livermore National Laboratory
> ================!!LLNL BUSINESS ONLY!!================
> [email protected] urgent: [email protected]
> T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
