Hi Quincey, Thanks for the follow-up. One question; what are the functions/properties by which a plugin might wind up ENABLING caching of opened file handles?
Mark On Tue, 2010-11-09 at 05:06 -0800, Quincey Koziol wrote: > Hi Mark, > > On Nov 8, 2010, at 9:12 AM, Mark Miller wrote: > > > On the topic of caching opened files (external or otherwise), I have a > > question. > > > > In VisIt, we have many different 'flavors' of HDF5 files we read. A > > 'flavor' is a particular structure (groups/dataset/attribute names) of > > HDF5 file. We have a separate reader plugin for each flavor. > > > > We don't always know ahead of time which plugin should be used to open a > > given HDF5 file. So, we can wind up iterating over several, each > > attempting some HDF5 operations on the file and, perhaps, throwing > > exceptions (VisIt exceptions not HDF5 exceptions) and then moving onto > > the next HDF5 reader plugin to try. > > > > Here is my question. If one reader plugin calls H5Fopen and that > > succeeds but later in the plugin's attempt to read the file, it throws a > > VisIt exception to move onto the next plugin, what if it also does NOT > > call H5Fclose? Will that potentially negatively affect the next reader > > plugin which will also use H5Fopen to open that same file? Also, how > > does fact that these plugins are operating as shared libs effect things, > > in particular, any HDF5 library wide global data such as the list of > > cached files it has open? > > Well, this document describes a method of caching files opened through > external links, not files opened directly with H5Fopen(). Also, by default, > the caching is disabled, so applications won't get a surprise. And, it's > enabled on a per-file basis, not a global basis, again to avoid surprises for > applications. So, I don't think it'll affect the situation you describe > above with VisIt. Of course, you should close the files you open with > H5Fopen() or else they will remain open and chew up resources, etc. :-) I > don't think the shared library aspect will have any effect. > > Quincey > > > Mark > > > > On Mon, 2010-11-08 at 07:05, Quincey Koziol wrote: > >> Hi all, > >> I've post a new RFC describing a feature we are planning to add to the > >> HDF5 library that will cache open files from external links: > >> > >> http://BLOCKEDBLOCKEDwww.BLOCKEDBLOCKEDhdfgroup.uiuc.edu/RFC/HDF5/ExternalLinkFileAccessProperty/CacheExternalLinkFileOpens.pdf > >> > >> If you are using external links and would like to comment on this > >> performance enhancement, please do so. > >> > >> Thanks, > >> Quincey > >> > >> > >> _______________________________________________ > >> Hdf-forum is for HDF software users discussion. > >> [email protected] > >> http://BLOCKEDBLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > -- > > Mark C. Miller, Lawrence Livermore National Laboratory > > ================!!LLNL BUSINESS ONLY!!================ > > [email protected] urgent: [email protected] > > T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511 > > > > > > _______________________________________________ > > Hdf-forum is for HDF software users discussion. > > [email protected] > > http://BLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://BLOCKEDmail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > -- Mark C. Miller, Lawrence Livermore National Laboratory ================!!LLNL BUSINESS ONLY!!================ [email protected] urgent: [email protected] T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
