|
hm, this is Java? There I actually have no idea how it works. Someone from the HDF5 Java fraction would know it works (or is supposed to work).
Werner
On Wed, 01 Sep 2010 16:29:06 +0200, elisa sibarani < [email protected]> wrote: Hi,
I saw the code written by Scott in:
http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/2009-September/001765.html
and I create almost the same way, but the problem still there, which is the minutes column contains symbol values. Is it possible because I make the fixed data type for string? And it is actually change the whole dataset again, that makes all data inserted into dataset is the wrong value.
Would mind to take a look again to my code?
Thank you so much for your help.
Regards,
Elisa MS
Here is my code:
------------------------------------------------------------------------------------------------------------------------------------
public unsafe struct Chararray
{
private
char* recordedText;
public char* RecordedText
{
get
{
return recordedText;
}
set
{
recordedText =
value;
}
}
public override string ToString()
{
string s = "";
IntPtr ipp = (IntPtr)this.recordedText;
s = System.Runtime.InteropServices.Marshal.PtrToStringAnsi(ipp);
return s;
}
}
public struct struct1
{
public Chararray[] a;
public float b;
public int c;
public float d;
public short e;
}
------------------------------------------------------------------------------------------------------------------------------------
struct1[] dset_data = new struct1[numline];
unsafe
{
IntPtr ipp =
System.Runtime.InteropServices.Marshal.StringToHGlobalAnsi(s);
dset_data[temp].a = new Chararray[1];
dset_data[temp].a[0].RecordedText = (char*)ipp;
}
------------------------------------------------------------------------------------------------------------------------------------
typestring = H5T.copy(H5T.H5Type.C_S1);
H5T.setSize(typestring,6);
typeId1 = H5T.create(H5T.CreateClass.COMPOUND, (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(struct1)));
H5T.insert(typeId1, "minutes", 0,typestring);
H5T.insert(typeId1, "EMT", H5T.getSize(typestring), H5T.H5Type.NATIVE_FLOAT);
H5T.insert(typeId1, "Fractional time", sizeof(float) + H5T.getSize(typestring), H5T.H5Type.NATIVE_INT);
H5T.insert(typeId1, "Charges", sizeof(int) + sizeof(float) + H5T.getSize(typestring),
H5T.H5Type.NATIVE_FLOAT);
H5T.insert(typeId1, "Phase angle", sizeof(float) + sizeof(int) + sizeof(float) + H5T.getSize(typestring), H5T.H5Type.NATIVE_SHORT);
------------------------------------------------------------------------------------------------------------------------------------
dataSetId = H5D.create(fileId, "/" + circuitid + "/" + groupname[0] + "/" + groupname[1] + "/" + groupname[2] + "/" + groupname[3], typeId1, spaceId);
// Write the integer data to the data set.
H5D.write(dataSetId,typeId1 ,new H5Array<struct1>(dset_data));
------------------------------------------------------------------------------------------------------------------------------------
--- On Wed, 9/1/10, Werner Benger <[email protected]> wrote:
From: Werner Benger
<[email protected]>
Subject: Re: [Hdf-forum] using compound data types
To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]>
Date: Wednesday, September 1, 2010, 9:57 AM
Hm, H5Tarray_create() is both there in the old HDF5 1.6 and 1.8
http://www.hdfgroup.org/HDF5/doc1.6/RM_H5T.html#Datatype-ArrayCreate http://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-ArrayCreate
So it should be there in your version of HDF5?
I thought as well about reordering the short data type to the end, but it would only work if the HDF5 compound is sizeof(struct) but not the sum of the components. It would then contain some undefined data in the file that you will never see (if I guess right how HDF5 is doing it).
Werner
On Wed, 01 Sep 2010 14:45:16 +0200, elisa sibarani < [email protected]> wrote: Hi Werner,
Thank you for your suggestion, I will keep it later for the Offsetof and H5Tpack, I can use short data type, but with some trick, by putting the short data type in the last position, and indeed it works. But I should look into your suggestion to make it better.
My confusion is about the H5Tarray create, I couldn't find it in the library, and right now I still do not clear of how to use this to create data type for array of characters. Would you give me another related links?
Thanks again.
regards,
Elisa
--- On Wed, 9/1/10, Werner Benger <[email protected]> wrote:
From: Werner Benger <[email protected]>
Subject: Re: [Hdf-forum] using compound data types
To: "HDF Users
Discussion List" <[email protected]>, "elisa sibarani" <[email protected]>
Date: Wednesday, September 1, 2010, 7:35 AM
Hi Elisa,
so it seems it's indeed an alignment problem, such that the compiler actually introduces an artificial 2 bytes between the phase angle and the EMT float. You should be able to verify this if sizeof(struct) > sizeof(int)+sizeof(float)+sizeof(shot)+sizeof(float). My guess is then that the sizeof(struct1) is two bytes larger.
My guess is (haven't needed it myself so far) that you will have to specify the exact relative addresses of the struct members, so or EMT you add a number of 2 (or compute the difference of the address between class member EMT and the struct, e.g. via offsetof
http://en.wikipedia.org/wiki/Offsetof
), and then call H5Tpack() to optimize the compound data type to store the data more compact in the file as it is in
memory:
http://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-Pack
For the string, you'd need to create a datatype which is an array of characters
http://www.hdfgroup.org/HDF5/doc/RM/RM_H5T.html#Datatype-ArrayCreate
and use this one as the compound element data type. And yes, the length of that char-array would go into the offset of the next member. It would make sense to make it a multiple of 4 to have it 32-bit aligned as well.
Werner
On Wed, 01 Sep 2010 13:06:21 +0200, elisa sibarani < [email protected]> wrote: Hi Werner,
I try again your suggestion and it is
working, I use the sizeof for the compound data type sizes --> sizeof (struct1)
But, the problem still arise, the EMT has changed to another value, but if I change the charges into int data type, there is no problem then. Since I use short data type to decrease the size of the file, is there any way so that I still can use the short data type even if it has different offset compare to int and float?
Oh and for the string, where can I specify the fixed length size? So, the offset for the other data type will sum up this fixed length size?
Thanks again
Regards,
Elisa MS
--- On Wed, 9/1/10, Werner Benger <[email protected]> wrote:
From: Werner
Benger <[email protected]>
Subject: Re: [Hdf-forum] using compound data types
To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]>
Date: Wednesday, September 1, 2010, 5:25 AM
Hi Elisa,
the problem with strings is that they are of variable length, and this would make the entire dataset using strings to be a variable-length datatype, which requires a bit of different handling via HDF5. It's possible, but if you know something maximal length of your strings - which would be the case for a time of constant format hh:mm - then it would be better to define it as a fixed-length array of e.g. 6 characters.
For the EMT in your code, it looks correct to me, might be another issue. Maybe you could replace the number 14 by some sizeof() _expression_ as well. Possibly there is an issue with 32-bit alignments, since fractional and charges would both be aligned to 32-bits, but the following short would make it a 16-bit offset. Try using an int for the phase angle to see if it changes anything.
Werner
On
Wed, 01 Sep 2010 11:09:37 +0200, elisa sibarani < [email protected]> wrote: Hi Werner,
I already try your suggestion using sizeof to fill the offset parameter, but I still found problems. Suppose I create compound data type as below:
-------------------------------------------------------------------------------------------------------------------------------------
typeId1 = H5T.create(H5T.CreateClass.COMPOUND,14);
H5T.insert(typeId1, "fractional", 0, H5T.H5Type.NATIVE_INT);
H5T.insert(typeId1, "charges", sizeof(int),
H5T.H5Type.NATIVE_FLOAT);
H5T.insert(typeId1, "phase angle", sizeof(float)+sizeof(int), H5T.H5Type.NATIVE_SHORT);
H5T.insert(typeId1, "EMT", sizeof(short)+sizeof(float)+sizeof(int), H5T.H5Type.NATIVE_FLOAT);
-------------------------------------------------------------------------------------------------------------------------------------
for the first row of the data, it is successfully inserted the right data for fractional, charges, and phase angle. But for EMT, it was converted into different values, and after that for the next row until the end, all of the value was changed from the sources value in dset_data. What will be the problem?
And yes, I agree with you about the string in C++, I read issues regarding this matter:
HDF5 STRING is
not
a
string but in fact a char *
//since it is we need to translate the return into a pointer address
Therefore, is there any different ways to implement string in compound data types or it is the only
way to use string in HDF5 file? Do you have any codes to implement the string in HDF5?
Thank you so much for your help.
Regards,
Elisa MS
--- On Wed, 9/1/10, Werner Benger <[email protected]> wrote:
From: Werner Benger <[email protected]>
Subject: Re: [Hdf-forum] using compound data types
To: "HDF Users Discussion List" <[email protected]>, "elisa sibarani" <[email protected]>
Date: Wednesday, September 1, 2010, 4:32 AM
Hm, it looks suspicious to me that you reserve just one byte of space for the "minutes" entry as in:
H5T.insert(typeId1, "minutes", 0, H5T.H5Type.C_S1);
H5T.insert(typeId1, "fractional", 1, H5T.H5Type.NATIVE_INT);
^^
H5T.insert(typeId1, "charges", 5, H5T.H5Type.NATIVE_FLOAT);
Also, are you sure that "int" is always 4 bytes on any platform you run your code? Using sizeof(int) might be safer as in:
H5T.insert(typeId1, "charges", 1+sizeof(int) , H5T.H5Type.NATIVE_FLOAT);
However, my main concern would be about reserving one byte for
the string. I'm not sure how C-strings are handled in HDF5, but it would seem that this leaves you just space for the 0-byte. C++ strings you can't save directly anyway, as they have an internal data structure with pointers to "somewhere", and you want to store the content of these pointers (that are hidden in the class definition) to the file, not the pointers themselves (which you would get by saving the address of a string object).
Werner
On Wed, 01 Sep 2010 10:20:10 +0200, elisa sibarani < [email protected]> wrote: I'm adding information, after I try to run my program again, the string data was converted into another symbol, and for other data (int,
float, short) as well. I do not know what factors caused it, since I already used the available data type in HDF5DotNet.
But, if I try for each of the data, first I try to insert just string, it is still changed into another symbol, but if I only insert integer data, the actual value can be inserted.
I guess the problem comes from H5Array<struct1>(dset_data), when it wrap the array, it changes the value as well?
H5D.write(dataSetId,typeId1,new H5Array<struct1>(dset_data));
Hope that I can see any reply regarding this matter.
Thank you.
regards,
Elisa
--- On Tue, 8/31/10, elisa sibarani <[email protected]> wrote:
From: elisa sibarani <[email protected]>
Subject: [Hdf-forum] using compound data types
To: [email protected]
Date: Tuesday, August
31, 2010, 11:48 PM
Hi All,
I use compound data types for my dataset, and here I create the struct first, and then load the data, after that create compound data types, create dataset using that compound data types, and finally write into the dataset. I don't know what was wrong with my code, because the result's looking was not exactly what I want. The reason because:
1. The string data that should contains hour:minute (for example 1:20, 17:50) was change into symbol in the HDF5 file
2. The dataset do not containt the same structures as the dset_data (array as the source
data), it is messy in the HDF5 file.
Until now, I'm trying to fix this, but still can't find any clue.
Thanks
regards,
Elisa
Here is the copy of my code:
public struct struct1
{
public string a;
public int b;
public float c;
public short d;
public int e;
}
public void btnInsert()
{
struct1[] dset_data = new struct1[numline];
foreach (string s in pd_data)
{
if (temp1 == 1)
{
dset_data[temp].a = s;
}
if (temp1 == 2)
{
dset_data[temp].b =
int.Parse(s);
}
if (temp1 == 3)
{
if (s == null || s.Length == 0)
{
dset_data[temp].c = float.Parse("0");
}
else
{
dset_data[temp].c = float.Parse(s);
}
}
if (temp1 == 4)
{
dset_data[temp].d = short.Parse(s);
}
if (temp1 == 5)
{
if (s == null || s.Length
== 0)
{
dset_data[temp].e = int.Parse("0");
}
else
{
dset_data[temp].e = int.Parse(s);
}
}
temp1 = temp1 + 1;
}
//create data space
ulong[] dims = new ulong[1];
dims[0] = ulong.Parse(numline.ToString());
spaceId = H5S.create_simple(1, dims);
//create compound data type
typeId1 = H5T.create(H5T.CreateClass.COMPOUND,15);
H5T.insert(typeId1, "minutes", 0, H5T.H5Type.C_S1);
H5T.insert(typeId1, "fractional", 1, H5T.H5Type.NATIVE_INT);
H5T.insert(typeId1, "charges", 5,
H5T.H5Type.NATIVE_FLOAT);
H5T.insert(typeId1, "phase angle", 9, H5T.H5Type.NATIVE_SHORT);
H5T.insert(typeId1, "EMT", 11, H5T.H5Type.NATIVE_INT);
dataSetId = H5D.create(fileId, "/" + circuitid + "/" + groupname[0] + "/" + groupname[1] + "/" + groupname[2] + "/" + groupname[3], typeId1, spaceId);
// Write the data to the data set.
H5D.write(dataSetId,typeId1,new H5Array<struct1>(dset_data));
|
-----Inline Attachment Follows-----
|
-- ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
|
-- ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
|
-- ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
|
-- ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
|
-- ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 |
