Hi All, The file image operations features discussed in the RFC I posted last July have been implemented, and a snap shot is available for down load.
For those who may not remember, the driving use case behind these extensions is to use of the HDF5 library as a packetizing/depacketizing tool. Simply put, in some cases it is cheaper to construct an image of an HDF5 file in a buffer, and then transmit the image to another machine than it is to transfer the same data by writing a file to a commonly accessible file system. If this is something you are interested in doing, please give the current version of the RFC a read, and if appropriate, try out the features in the snapshot. The RFC is available in PDF format via ftp at: ftp://hdfgroup.uiuc.edu/pub/outgoing/mainzer/file_image.21863/load_core_file_driver_from_image_RFC_v17.pdf Note that there have been many changes in the RFC since the previous post. The snapshot is available via ftp at: ftp://hdfgroup.uiuc.edu/pub/outgoing/mainzer/file_image.21863/hdf5-1.9.102-of20120104.tar.gz User level documentation is still in progress, but the RFC should tell you everything you need to know to use the new features. The RFC is 40 pages long, so the following conceptual summary may help you skim through and find the points of interest. The core file driver can already be used to construct buffers containing HDF5 files, and we already have facilities (albeit nasty ones) that allow the user access to these buffers. However, on the receiving end, there is no way to open a buffer containing an HDF5 file without first writing the buffer to the file system. In essence, the file image operations extensions to HDF5 remedy this situation. To do this, we implemented two new sets of sets of property list calls, along with supporting code, and modifications to the core file driver. The first set of property list calls inserts and extracts copies of buffers containing HDF5 file images into and out of file access property lists. Note that since the property list structure is designed to support call by value, this results in two buffer copies -- one when the buffer is copied into a FAPL, and a second when the buffer is copied from the FAPL into the core file driver. The second set of property list calls allows the application to specify callbacks that are used by the property list code and the core file driver when allocating, copying, reallocating and freeing buffers containing file images. From the HDF5 library's perspective, these calls must be functionally identical to the standard malloc, memcpy, realloc, and free C library calls -- although as you will see in the RFC, they can be designed in such a way as to avoid the buffer copies mentioned above. In addition, we implemented a high level library calls to facilitate the more common operations on file images, and a low level call to make it easier to obtain file images. Again, if this sort of functionality is of potential interest to you, please take a look and let us know what you think. Best regards, John Mainzer _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
