Richard Guenther wrote: > > On Mon, 26 May 2003, Ireneusz SZCZESNIAK wrote: > > > Hello, > > > > I am sending this letter to seek your advice. We are developing a > > package called dxhdf5 (www-beams.colorado.edu/dxhdf5), used to import > > HDF5 data into OpenDX, and now we are investigating on how to extend > > our modules. > > > > We would appreciate any comments from you. > > Try to make it less dependend on file/dataset structure. Just traverse > the file for groups/datasets and output a string list of available > datasets as one output, selected datasets as another output. Try to > preserve all attributes at the root group and the datasets. > > I tried to do this but got distracted to add some special support for > my personal datasets. But you can get it at > > http://www.tat.physik.uni-tuebingen.de/~rguenth/pooma/hdf5.tar.gz > > Richard. > > -- > Richard Guenther <richard dot guenther at uni-tuebingen dot de> > WWW: http://www.tat.physik.uni-tuebingen.de/~rguenth/
Hi Ireneusz & Richard, I'm glad to see other people developing HDF5 readers for OpenDX. It would be nice to combine all the activities to provide a single import module which then could be made part of the main distribution. Of course, this module should be general enough to fit peoples' needs, and that hightly depends on the HDF5 file layout used by the various applications which generate HDF5 datafiles. For plain unigrid data, I think, there exists a de-facto standard that each dataset should have at least an "origin" and a "delta" attribute which will be used to create the positions and connections of the imported DX field. I tried both the "ImportHDF5Field" and "hdf5" reader modules on some example HDF5 output from our application code (http://www.cactuscode.org). The first module worked just fine, the second produced HDF5 error messages - maybe because your module expects a different layout, Richard ? Like Richard, I find it a bit inconvenient for files with multiple datasets that a user has to specify each dataset to import explicitely by name. One could output the list of dataset names and then let the user pick one (or more) out of this list. Our ImportHDF5 module takes an index of the dataset to read instead, ranging from 0 to the total number of datasets found in the file (this information is output on a separate tab). Our datafiles usually contain a time series of the same variable, with each dataset having a "time" attribute attached to it. If such an attri- bute was found during the browsing then the import module would also sort the datasets by their time values. Feeding the "max" input tab of a Sequencer module with the "total number of datasets" information, and connecting the sequencer output tab back to ImportHDF5, it is thus trivial to play an animation of all available timesteps. To make the dataset selection via indices more general, an import module could receive a (list of) attribute name(s) to sort the list of datasets ? I very much like the hyperslabbing feature of the ImportHDF5Field module. The grid size we deal with in our application can be quite large so that it is then nice to specify a region of interest to import from a series of very large datasets, or to downsample the data. I want to add this feature to our ImportHDF5 module - so far one can specify a single (n-1) slice to read from an n-dimensional dataset. ImportHDF5 has a flag to tell it whether floating point datasets should be imported in single or double precision. Maybe that's a feature you want to add to your module ? Also, our ImportHDF5 module can import data from remote HDF5 files using the Stream Virtual File Driver of the HDF5 library. And we are working on another remote file access driver for HDF5 based on GridFtp. In the future we want to do remote visualization of large-scale hierar- chical datasets using OpenDX. Finally, if people want to try out our ImportHDF5 module, please refer to http://www.cactuscode.org/Documentation/HOWTO/OpenDX-HOWTO This also describes how to check out the OpenDXutils package from our CVS server. The package contains the ImportHDF5 module source code plus makefile as well as some example networks and a sample HDF5 data file. Ciao, Thomas. -- ========================================================================= Thomas Radke Max-Planck-Institute for Gravitational Physics, Albert-Einstein-Institute Am Muehlenberg 1, 14476 Golm, Germany fon +49 331 567-7329 fax +49 331 567-7298 http://www.aei.mpg.de/~tradke "Die USA wollen Irak in 3 Zonen aufteilen: Normal, Super und Diesel." =========================================================================
