> My original example was about new and allocating to the heap
I remembered incorrectly, sorry. HDF5 works fine with 'new'. You were
trying to to use the heap and have multi dim indexing. The problem was that
you made separate allocations for each row and then were writing the row
pointers instead of the data. To write data with a single call to HD5Write
the data must allocated with a single call to 'new'.
Here's my sample code from before showing how to do what you were
attempting, using 'new' and getting multi dim indexing. You will give
'data' to HDF5 for writing and use 'md_data' for the indexing behavior.
This could be made into a class to encapsulate the behavior for easy reuse.
auto data = new double[rows*cols]; // allocate all data in one block
auto md_data = new double*[rows]; // allocate pointers for each row
for (int r = 0; r != rows; ++r) // set row pointers
md_data[r] = data + r*cols;
md_data[2][5] = 1.0; // row pointer array can be used as a pseudo md array
H5DWrite(... , data);
David
On Tue, May 17, 2016 at 8:31 AM, Steven Walton <[email protected]>
wrote:
> I guess I am just surprised that hdf5 does not accept the standard c++
> "array" creation. New allocates to the heap. It may be pointer, but this is
> pretty standard and they teach it in all c++ classes. New has existed for
> quite some time. I just thought the hdf5 group would follow these standard
> types.
>
> I understand that vector is a 1D array. My original example was about new
> and allocating to the heap, with standard c++.
>
> On Mon, May 16, 2016 at 9:30 PM, David <[email protected]> wrote:
>
>> The limitation you ran into in your original example was that std::vector
>> is only for 1 dimensional arrays and you wanted to use multi dimensional
>> indexing. C++ does not have a built in class for heap allocated multi dim
>> arrays. However you can simply use boost::multi_array. The standards
>> committee wisely has been using boost as a testing ground for new library
>> features, allowing them to mature and prove themselves before being
>> canonized. I will not be surprised if boost::multi_array is adopted into a
>> future standard.
>>
>> David
>>
>>
>> On Mon, May 16, 2016 at 7:01 PM, Steven Walton <[email protected]>
>> wrote:
>>
>>> I actually just created a buffer array and then passed back to a vector.
>>> But what I'm saying is that this is an extremely common way to store data
>>> in C++, if not the default way for most users. Vectors are extremely common
>>> as well. Why are we still held back by using C type arrays?
>>>
>>> On Mon, May 9, 2016 at 12:58 PM, David <[email protected]> wrote:
>>>
>>>> Hi Steve,
>>>>
>>>> boost::multi_array provides a clean interface for multi dimensional
>>>> arrays in C++.
>>>>
>>>> You can also do something like this:
>>>>
>>>> auto data = new double[rows*cols]; // allocate all data in one block
>>>> auto md_data = new double*[rows]; // allocate pointers for each row
>>>> for (int r = 0; r != rows; ++r) // set row pointers
>>>> md_data[r] = data + r*cols;
>>>> md_data[2][5] = 1.0; // row pointer array can be used as a pseudo md array
>>>>
>>>>
>>>> On Fri, May 6, 2016 at 1:29 PM, Steven Walton <[email protected]
>>>> > wrote:
>>>>
>>>>> So I am noticing some interesting behavior and is wondering if there
>>>>> is a way around this.
>>>>> I am able so assign a rank 1 array dynamically and write this to an
>>>>> hdf5 filetype but I do not seem to be able to do with with higher order
>>>>> arrays. I would like to be able to write a PPx array to h5 and retain the
>>>>> data integrity. More specifically I am trying to create a easy to use
>>>>> vector to array library <https://github.com/stevenwalton/H5Easy> that
>>>>> can handle multidimensional data (works with rank 1).
>>>>>
>>>>> Let me give some examples. I will also show the typenames of the
>>>>> arrays.
>>>>>
>>>>> Works:
>>>>> double *a = new double[numPts]; // typename: Pd
>>>>> double a[numPts]; // typename A#pts_d
>>>>> double a[num1][num2]; typename:Anum1_Anum2_d
>>>>>
>>>>> What doesn't work:
>>>>> double **a = new double*[num1];
>>>>> for ( size_t i = 0; i < num1; ++i )
>>>>> a[i] = new double[num2];
>>>>> // typename PPd
>>>>>
>>>>> Testing the saved arrays with h5dump (and loading and reading
>>>>> directly) I find that if I have typename PPx (not necessarily double) I
>>>>> get
>>>>> garbage stored. Here is an example code and output from h5dump showing the
>>>>> behavior.
>>>>> ------------------------------------------------------------
>>>>> compiled with h5c++ -std=c++11
>>>>> ------------------------------------------------------------
>>>>> #include "H5Cpp.h"
>>>>> using namespace H5;
>>>>>
>>>>> #define FILE "multi.h5"
>>>>>
>>>>> int main()
>>>>> {
>>>>> hsize_t dims[2];
>>>>> herr_t status;
>>>>> H5File file(FILE, H5F_ACC_TRUNC);
>>>>> dims[0] = 4;
>>>>> dims[1] = 6;
>>>>>
>>>>> double **data = new double*[dims[0]];
>>>>> for ( size_t i = 0; i < dims[0]; ++i )
>>>>> data[i] = new double[dims[1]];
>>>>>
>>>>> for ( size_t i = 0; i < dims[0]; ++i )
>>>>> for ( size_t j = 0; j < dims[1]; ++j )
>>>>> data[i][j] = i + j;
>>>>>
>>>>> DataSpace dataspace = DataSpace(2,dims);
>>>>> DataSet dataset( file.createDataSet( "test", PredType::IEEE_F64LE,
>>>>> dataspace ) );
>>>>> dataset.write(data, PredType::IEEE_F64LE);
>>>>> dataset.close();
>>>>> dataspace.close();
>>>>> file.close();
>>>>>
>>>>> return 0;
>>>>> }
>>>>> ------------------------------------------------------------
>>>>> h5dump
>>>>> ------------------------------------------------------------
>>>>> HDF5 "multi.h5" {
>>>>> GROUP "/" {
>>>>> DATASET "test" {
>>>>> DATATYPE H5T_IEEE_F64LE
>>>>> DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
>>>>> DATA {
>>>>> (0,0): 1.86018e-316, 1.86018e-316, 1.86018e-316, 1.86019e-316, 0,
>>>>> (0,5): 3.21143e-322,
>>>>> (1,0): 0, 1, 2, 3, 4, 5,
>>>>> (2,0): 0, 3.21143e-322, 1, 2, 3, 4,
>>>>> (3,0): 5, 6, 0, 3.21143e-322, 2, 3
>>>>> }
>>>>> }
>>>>> }
>>>>> }
>>>>> ------------------------------------------------------------------
>>>>> As can be seen the (0,0) set is absolute garbage (except the last
>>>>> character which is the first number of the actual array), (0,5) is out of
>>>>> bounds, and has garbage data. (1,0) has always contained real data
>>>>> (though
>>>>> it should be located at (0,0)). So this seems like some addressing
>>>>> problem.
>>>>>
>>>>> Is this a bug in the h5 libraries that allows me to read and write Pd
>>>>> data as well as Ax0_...Axn_t data but not P...Pt data? Or is this for some
>>>>> reason intentional? As using new is a fairly standard way to assign
>>>>> arrays,
>>>>> making P...Pt type data common, I have a hard time seeing this as
>>>>> intentional. In the mean time is anyone aware of a workaround to this? The
>>>>> data I am taking in will be dynamically allocated so I do not see a way to
>>>>> get Ax_... type data.
>>>>>
>>>>> Thank you,
>>>>> Steven
>>>>>
>>>>> _______________________________________________
>>>>> Hdf-forum is for HDF software users discussion.
>>>>> [email protected]
>>>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>>> Twitter: https://twitter.com/hdf5
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Hdf-forum is for HDF software users discussion.
>>>> [email protected]
>>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>> Twitter: https://twitter.com/hdf5
>>>>
>>>
>>>
>>> _______________________________________________
>>> Hdf-forum is for HDF software users discussion.
>>> [email protected]
>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>> Twitter: https://twitter.com/hdf5
>>>
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>> Twitter: https://twitter.com/hdf5
>>
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
