Sorry....gave you the wrong reference regarding unlimited dimensions. Here is correct one, https://support.hdfgroup.org/HDF5/doc/RM/RM_H5S.html#Dataspace-CreateSimple
Also, in reading that, using unlimited dimensions *also*requires* chunking too. But, my suggestion #2 replaces use of unlimited dimensions with a max and relies upon ability to set chunk size such that wasted space (due to partially written chunks if any) might be small. Note, HDF5 will only ever allocate space in the file for chunks that are actually written. So, you can have a very large, sparse 2D array (like a banded matrix or something) that really doesn't take up that much space on disk. Mark -- Mark C. Miller, LLNL From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of "Miller, Mark C." <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Monday, October 17, 2016 at 9:47 AM To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Subject: Re: [Hdf-forum] Irregular 2D array write (C++) Hi Steven, Hmm. I am thinking maybe a few approaches. 1. Use "unlimited" dimensions for those dimensions that have variable size, https://support.hdfgroup.org/HDF5/doc/RM/RM_H5S.html#Dataspace-ExtentDims 2. Define row-dim to be max of all your rows and then se row-oriented chunking, https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetChunk. Longer rows use more chunks. Shorter rows use fewer 3. Maybe you could do this with 'virtual datasets' where each row is a separate HDF5 dataset and another, virtual dataset, knits them all together, https://support.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html Hope that helps. Mark -- Mark C. Miller, LLNL From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Steven Walton <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Monday, October 17, 2016 at 9:34 AM To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Subject: [Hdf-forum] Irregular 2D array write (C++) So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes. 9 23 41 54 12 9 46 29 19 60 5 93 18 58 5 29 58 17 9 50 38 58 95 So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept? -Steven
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
