Sorry....gave you the wrong reference regarding unlimited dimensions. Here is 
correct one, 
https://support.hdfgroup.org/HDF5/doc/RM/RM_H5S.html#Dataspace-CreateSimple

Also, in reading that, using unlimited dimensions *also*requires* chunking too.

But, my suggestion #2 replaces use of unlimited dimensions with a max and 
relies upon ability to set chunk size such that wasted space (due to partially 
written chunks if any) might be small. Note, HDF5 will only ever allocate space 
in the file for chunks that are actually written. So, you can have a very 
large, sparse 2D array (like a banded matrix or something) that really doesn't 
take up that much space on disk.

Mark

--
Mark C. Miller, LLNL

From: Hdf-forum 
<hdf-forum-boun...@lists.hdfgroup.org<mailto:hdf-forum-boun...@lists.hdfgroup.org>>
 on behalf of "Miller, Mark C." <mille...@llnl.gov<mailto:mille...@llnl.gov>>
Reply-To: HDF Users Discussion List 
<hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Monday, October 17, 2016 at 9:47 AM
To: HDF Users Discussion List 
<hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] Irregular 2D array write (C++)

Hi Steven,

Hmm. I am thinking maybe a few approaches.


  1.  Use "unlimited" dimensions for those dimensions that have variable size, 
https://support.hdfgroup.org/HDF5/doc/RM/RM_H5S.html#Dataspace-ExtentDims
  2.  Define row-dim to be max of all your rows and then se row-oriented 
chunking, 
https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetChunk. Longer 
rows use more chunks. Shorter rows use fewer
  3.  Maybe you could do this with 'virtual datasets' where each row is a 
separate HDF5 dataset and another, virtual dataset, knits them all together, 
https://support.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html

Hope that helps.

Mark


--
Mark C. Miller, LLNL

From: Hdf-forum 
<hdf-forum-boun...@lists.hdfgroup.org<mailto:hdf-forum-boun...@lists.hdfgroup.org>>
 on behalf of Steven Walton 
<walton.stev...@gmail.com<mailto:walton.stev...@gmail.com>>
Reply-To: HDF Users Discussion List 
<hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Monday, October 17, 2016 at 9:34 AM
To: HDF Users Discussion List 
<hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Irregular 2D array write (C++)

So I'm wondering if there is a good way to write an irregular shaped 2D array 
into hdf5. And example of this would be like storing vtk node connections for 
an unstructured grid. First number noting the cell type and the next numbers 
denoting the nodes.

9 23 41 54 12
9 46 29 19 60
5 93 18 58
5 29 58 17
9 50 38 58 95

So the array has some rows that are length 5 and others that are length 4 (or 
arbitrary). I understand how to do this with C++ vectors and push_back, but 
those don't create contiguous arrays. Is there another way to create this in a 
way that HDF would accept?

-Steven
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Reply via email to