On Wed, 28 May 2003, Thomas Radke wrote: > Richard Guenther wrote: > > > > On Mon, 26 May 2003, Ireneusz SZCZESNIAK wrote: > > > > > Hello, > > > > > > I am sending this letter to seek your advice. We are developing a > > > package called dxhdf5 (www-beams.colorado.edu/dxhdf5), used to import > > > HDF5 data into OpenDX, and now we are investigating on how to extend > > > our modules. > > > > > > We would appreciate any comments from you. > > > > Try to make it less dependend on file/dataset structure. Just traverse > > the file for groups/datasets and output a string list of available > > datasets as one output, selected datasets as another output. Try to > > preserve all attributes at the root group and the datasets. > > > > I tried to do this but got distracted to add some special support for > > my personal datasets. But you can get it at > > > > http://www.tat.physik.uni-tuebingen.de/~rguenth/pooma/hdf5.tar.gz > > > > Richard. > > > > -- > > Richard Guenther <richard dot guenther at uni-tuebingen dot de> > > WWW: http://www.tat.physik.uni-tuebingen.de/~rguenth/ > > Hi Ireneusz & Richard, > > I'm glad to see other people developing HDF5 readers for OpenDX. > It would be nice to combine all the activities to provide a single > import module which then could be made part of the main distribution.
Yes, thats desperately needed. > Of course, this module should be general enough to fit peoples' needs, > and that hightly depends on the HDF5 file layout used by the various > applications which generate HDF5 datafiles. > For plain unigrid data, I think, there exists a de-facto standard that > each dataset should have at least an "origin" and a "delta" attribute > which will be used to create the positions and connections of the > imported DX field. Which is not necessary, as you can extract the required attributes, create positions and replace them as needed in your dx program. > I tried both the "ImportHDF5Field" and "hdf5" reader modules on some > example HDF5 output from our application code (http://www.cactuscode.org). > The first module worked just fine, the second produced HDF5 error messages > - maybe because your module expects a different layout, Richard ? Like I said - I started with "generic" code but "this needs to work now" introduced some specialities which are not checked. So my reader is not generic at the moment, but contains everything needed for genericity. > Like Richard, I find it a bit inconvenient for files with multiple datasets > that a user has to specify each dataset to import explicitely by name. > One could output the list of dataset names and then let the user pick > one (or more) out of this list. > Our ImportHDF5 module takes an index of the dataset to read instead, > ranging from 0 to the total number of datasets found in the file (this > information is output on a separate tab). > Our datafiles usually contain a time series of the same variable, with > each dataset having a "time" attribute attached to it. If such an attri- > bute was found during the browsing then the import module would also > sort the datasets by their time values. Feeding the "max" input tab > of a Sequencer module with the "total number of datasets" information, > and connecting the sequencer output tab back to ImportHDF5, it is thus > trivial to play an animation of all available timesteps. > To make the dataset selection via indices more general, an import module > could receive a (list of) attribute name(s) to sort the list of datasets ? I'd start without these complexities in the hdf5 module, as you can do nearly all of this stuff within your dx program. > I very much like the hyperslabbing feature of the ImportHDF5Field module. > The grid size we deal with in our application can be quite large so that > it is then nice to specify a region of interest to import from a series > of very large datasets, or to downsample the data. Yes, its trivially to add. > I want to add this feature to our ImportHDF5 module - so far one can > specify a single (n-1) slice to read from an n-dimensional dataset. > ImportHDF5 has a flag to tell it whether floating point datasets should > be imported in single or double precision. Maybe that's a feature you > want to add to your module ? Hmm - I never found it necessary to read doubles. Richard.
