Hi, On Sat, Dec 3, 2011 at 9:56 PM, Charles Darwin <[email protected]> wrote: > Hi, > > I'm combining the Split File and File Family drivers in the following way: > - Split File is applied in order to separate the data into two files > (metadata and raw data, i.e. data.meta and data.raw) > > - then additionally, for the raw data storage the File Family driver is > applied > > This results in having one metadata file (data.meta) and multiple raw data > files according to the File Family member size setting (data0.raw, data1.raw > etc.). > > For the most part this combination works as one would expect, but there is > one circumstance in which it doesn't seem to work properly. > > Let's say that the file family member size is set 100 MB when creating the > HDF5 file. If during the first write to this file the amount of data that > ends up being written is more than 100 MB, for example 120 MB, then > everything works as expected - the result is one data.meta file and two raw > files; data0.raw which is 100 MB and data1.raw which is 20 MB, and appending > even more data always results in *.raw files of 100MB (except for the last > raw file which may not be full). > > However, if the first writing to this file is less than 100MB (less than the > file family member size), for example 50 MB, and the file is closed, then > this will create 2 files; data.meta and data0.raw of size 50 MB. This is all > still as expected. If however, we then write more data to that file, then > things work as if the file family member size setting was set to 50 MB > rather than 100 MB, i.e. if we write 120 more MB to this file after closing > it, this will result in data.meta, data0.raw of 50 MB, data1.raw of 50 MB, > data2.raw 50 MB, and data3.raw of 20 MB. > > It's as if the setting for the file family member size is 50 MB rather than > 100 MB, i.e. the file family member size becomes the size of the first raw > file that is written, rather than the one that is specified in the creation > properties. > > The HDF5 version is 1.8.7, operating system is Windows 7 64 bit. > > 1. Is the described file driver combination fully supported?
I would describe this as "Allowed behavior, but not explicitly recommended and certainly not tested by us." That said, I would expect this setup to work correctly. > 2. Is there a way to get around this behavior, so that the raw files end up > respecting the file family member size creation setting, rather than > inheriting the size of the first raw file after the first write session? I'm not sure of a work-around that would fix your problem. I've entered a bug report concerning this behavior since this sounds like a bug in the driver. I'm not sure how high of a priority it will have since this is a more unusual way to use the virtual file drivers, but I'll bring it up at our release planning meeting. Thanks for reporting the issue! Dana > Thank you. > > -- > View this message in context: > http://hdf-forum.184993.n3.nabble.com/Problem-with-Combining-Split-File-and-File-Family-tp3558487p3558487.html > Sent from the hdf-forum mailing list archive at Nabble.com. > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
