If I have a simulation running that manages 3D data, and where some of the data will change after every simulation cycle, and I want to be able to keep track of the changes, so that I could go "back in time" to a previous state, and restart from there in case of a programming bug, how do I best represents this in HDF5? Note that I want to protocol all changes, but in the normal case, I am only interested in efficiently accessing the newest state.
One way would be to define my dataspace unlimited in 4 dimensions, and use the index of the fourth dimension as the point-in-time where some chunk changed, always saving a whole chunk, even if only part of it changed. Then I would need some way, for every 3D chunk, to know all the points-in-time where the data changed, and most importantly, what is the "highest index". This would basically be a sparse "append-only" dataset. Another possibility would be that I divide the 3D space in blocks of a fixed size (that what I would normally use as a chunk size), and instead have for every "logical application chunk" a VLEN that contains a list of composites, where each composite contains the "point-in-time" and the new data. This solves the "when was the data changed?" problem, but am I right to assume that I always have to load the whole VLEN when accessing it, meaning I have to load the "whole history" every time? Also, I only write at the end of the VLEN, but the VLEN size has to be updated, so this usage is not "append-only" like above. Are there any other obvious alternatives, and why should I choose which one? MfG / Regards, Sebastien Diot Grossblittersdorferstrasse 257-259 66119 Saarbruecken Germany Tel.: +49 (681) 8808-0 Fax.: +49 (681) 8808-300 E-Mail.: [email protected] ----------------------------------------------------- EURO DATA GmbH & Co. KG Datenverarbeitungsdienst HR A 6448 Amtsgericht Saarbrücken Komplementär: A. Reiß & Sohn GmbH HR B 4965 Amtsgericht Saarbrücken Geschäftsführer: Dipl.-Kfm. Karl-Heinz Siebenpfeiffer -----------------------------------------------------
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
