Hi Werner, Thank you for your suggestion, I will keep it later for the Offsetof and H5Tpack, I can use short data type, but with some trick, by putting the short data type in the last position, and indeed it works. But I should look into your suggestion to make it better.
My confusion is about the H5Tarray create, I couldn't find it in the library, and right now I still do not clear of how to use this to create data type for array of characters. Would you give me another related links? Thanks again. regards, Elisa --- On Wed, 9/1/10, Werner Benger <[email protected]> wrote: From: Werner Benger <[email protected]> Subject: Re: [Hdf-forum] using compound data types To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]> Date: Wednesday, September 1, 2010, 7:35 AM Hi Elisa, so it seems it's indeed an alignment problem, such that the compiler actually introduces an artificial 2 bytes between the phase angle and the EMT float. You should be able to verify this if sizeof(struct) > sizeof(int)+sizeof(float)+sizeof(shot)+sizeof(float). My guess is then that the sizeof(struct1) is two bytes larger. My guess is (haven't needed it myself so far) that you will have to specify the exact relative addresses of the struct members, so or EMT you add a number of 2 (or compute the difference of the address between class member EMT and the struct, e.g. via offsetof http://en.wikipedia.org/wiki/Offsetof ), and then call H5Tpack() to optimize the compound data type to store the data more compact in the file as it is in memory: http://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-Pack For the string, you'd need to create a datatype which is an array of characters http://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-ArrayCreate and use this one as the compound element data type. And yes, the length of that char-array would go into the offset of the next member. It would make sense to make it a multiple of 4 to have it 32-bit aligned as well. Werner On Wed, 01 Sep 2010 13:06:21 +0200, elisa sibarani <[email protected]> wrote: Hi Werner, I try again your suggestion and it is working, I use the sizeof for the compound data type sizes --> sizeof (struct1) But, the problem still arise, the EMT has changed to another value, but if I change the charges into int data type, there is no problem then. Since I use short data type to decrease the size of the file, is there any way so that I still can use the short data type even if it has different offset compare to int and float? Oh and for the string, where can I specify the fixed length size? So, the offset for the other data type will sum up this fixed length size? Thanks again Regards, Elisa MS --- On Wed, 9/1/10, Werner Benger <[email protected]> wrote: From: Werner Benger <[email protected]> Subject: Re: [Hdf-forum] using compound data types To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]> Date: Wednesday, September 1, 2010, 5:25 AM Hi Elisa, the problem with strings is that they are of variable length, and this would make the entire dataset using strings to be a variable-length datatype, which requires a bit of different handling via HDF5. It's possible, but if you know something maximal length of your strings - which would be the case for a time of constant format hh:mm - then it would be better to define it as a fixed-length array of e.g. 6 characters. For the EMT in your code, it looks correct to me, might be another issue. Maybe you could replace the number 14 by some sizeof() expression as well. Possibly there is an issue with 32-bit alignments, since fractional and charges would both be aligned to 32-bits, but the following short would make it a 16-bit offset. Try using an int for the phase angle to see if it changes anything. Werner On Wed, 01 Sep 2010 11:09:37 +0200, elisa sibarani <[email protected]> wrote: Hi Werner, I already try your suggestion using sizeof to fill the offset parameter, but I still found problems. Suppose I create compound data type as below: ------------------------------------------------------------------------------------------------------------------------------------- typeId1 = H5T.create(H5T.CreateClass.COMPOUND,14); H5T.insert(typeId1, "fractional", 0, H5T.H5Type.NATIVE_INT); H5T.insert(typeId1, "charges", sizeof(int), H5T.H5Type.NATIVE_FLOAT); H5T.insert(typeId1, "phase angle", sizeof(float)+sizeof(int), H5T.H5Type.NATIVE_SHORT); H5T.insert(typeId1, "EMT", sizeof(short)+sizeof(float)+sizeof(int), H5T.H5Type.NATIVE_FLOAT); ------------------------------------------------------------------------------------------------------------------------------------- for the first row of the data, it is successfully inserted the right data for fractional, charges, and phase angle. But for EMT, it was converted into different values, and after that for the next row until the end, all of the value was changed from the sources value in dset_data. What will be the problem? And yes, I agree with you about the string in C++, I read issues regarding this matter: HDF5 STRING is not a string but in fact a char * //since it is we need to translate the return into a pointer address Therefore, is there any different ways to implement string in compound data types or it is the only way to use string in HDF5 file? Do you have any codes to implement the string in HDF5? Thank you so much for your help. Regards, Elisa MS --- On Wed, 9/1/10, Werner Benger <[email protected]> wrote: From: Werner Benger <[email protected]> Subject: Re: [Hdf-forum] using compound data types To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]> Date: Wednesday, September 1, 2010, 4:32 AM Hm, it looks suspicious to me that you reserve just one byte of space for the "minutes" entry as in: H5T.insert(typeId1, "minutes", 0, H5T.H5Type.C_S1); H5T.insert(typeId1, "fractional", 1, H5T.H5Type.NATIVE_INT); ^^ H5T.insert(typeId1, "charges", 5, H5T.H5Type.NATIVE_FLOAT); Also, are you sure that "int" is always 4 bytes on any platform you run your code? Using sizeof(int) might be safer as in: H5T.insert(typeId1, "charges", 1+sizeof(int) , H5T.H5Type.NATIVE_FLOAT); However, my main concern would be about reserving one byte for the string. I'm not sure how C-strings are handled in HDF5, but it would seem that this leaves you just space for the 0-byte. C++ strings you can't save directly anyway, as they have an internal data structure with pointers to "somewhere", and you want to store the content of these pointers (that are hidden in the class definition) to the file, not the pointers themselves (which you would get by saving the address of a string object). Werner On Wed, 01 Sep 2010 10:20:10 +0200, elisa sibarani <[email protected]> wrote: I'm adding information, after I try to run my program again, the string data was converted into another symbol, and for other data (int, float, short) as well. I do not know what factors caused it, since I already used the available data type in HDF5DotNet. But, if I try for each of the data, first I try to insert just string, it is still changed into another symbol, but if I only insert integer data, the actual value can be inserted. I guess the problem comes from H5Array<struct1>(dset_data), when it wrap the array, it changes the value as well? H5D.write(dataSetId,typeId1,new H5Array<struct1>(dset_data)); Hope that I can see any reply regarding this matter. Thank you. regards, Elisa --- On Tue, 8/31/10, elisa sibarani <[email protected]> wrote: From: elisa sibarani <[email protected]> Subject: [Hdf-forum] using compound data types To: [email protected] Date: Tuesday, August 31, 2010, 11:48 PM Hi All, I use compound data types for my dataset, and here I create the struct first, and then load the data, after that create compound data types, create dataset using that compound data types, and finally write into the dataset. I don't know what was wrong with my code, because the result's looking was not exactly what I want. The reason because: 1. The string data that should contains hour:minute (for example 1:20, 17:50) was change into symbol in the HDF5 file 2. The dataset do not containt the same structures as the dset_data (array as the source data), it is messy in the HDF5 file. Until now, I'm trying to fix this, but still can't find any clue. Thanks regards, Elisa Here is the copy of my code: public struct struct1 { public string a; public int b; public float c; public short d; public int e; } public void btnInsert() { struct1[] dset_data = new struct1[numline]; foreach (string s in pd_data) { if (temp1 == 1) { dset_data[temp].a = s; } if (temp1 == 2) { dset_data[temp].b = int.Parse(s); } if (temp1 == 3) { if (s == null || s.Length == 0) { dset_data[temp].c = float.Parse("0"); } else { dset_data[temp].c = float.Parse(s); } } if (temp1 == 4) { dset_data[temp].d = short.Parse(s); } if (temp1 == 5) { if (s == null || s.Length == 0) { dset_data[temp].e = int.Parse("0"); } else { dset_data[temp].e = int.Parse(s); } } temp1 = temp1 + 1; } //create data space ulong[] dims = new ulong[1]; dims[0] = ulong.Parse(numline.ToString()); spaceId = H5S.create_simple(1, dims); //create compound data type typeId1 = H5T.create(H5T.CreateClass.COMPOUND,15); H5T.insert(typeId1, "minutes", 0, H5T.H5Type.C_S1); H5T.insert(typeId1, "fractional", 1, H5T.H5Type.NATIVE_INT); H5T.insert(typeId1, "charges", 5, H5T.H5Type.NATIVE_FLOAT); H5T.insert(typeId1, "phase angle", 9, H5T.H5Type.NATIVE_SHORT); H5T.insert(typeId1, "EMT", 11, H5T.H5Type.NATIVE_INT); dataSetId = H5D.create(fileId, "/" + circuitid + "/" + groupname[0] + "/" + groupname[1] + "/" + groupname[2] + "/" + groupname[3], typeId1, spaceId); // Write the data to the data set. H5D.write(dataSetId,typeId1,new H5Array<struct1>(dset_data)); -----Inline Attachment Follows----- _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org -- ___________________________________________________________________________ Dr. Werner Benger Visualization Research Laboratory for Creative Arts and Technology (LCAT) Center for Computation & Technology at Louisiana State University (CCT/LSU) 211 Johnston Hall, Baton Rouge, Louisiana 70803 Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 -- ___________________________________________________________________________ Dr. Werner Benger Visualization Research Laboratory for Creative Arts and Technology (LCAT) Center for Computation & Technology at Louisiana State University (CCT/LSU) 211 Johnston Hall, Baton Rouge, Louisiana 70803 Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 -- ___________________________________________________________________________ Dr. Werner Benger Visualization Research Laboratory for Creative Arts and Technology (LCAT) Center for Computation & Technology at Louisiana State University (CCT/LSU) 211 Johnston Hall, Baton Rouge, Louisiana 70803 Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
