Hi
So I tried, but not sure it worked. I cant figure out how to connect
the dataset to a file, so I can view it in hdf-viewer. Here is my
method for writing a an array of Strings to a dataset.
private static void writeUn() {
int file_id = -1;
int dcpl_id = -1;
int dataspace_id = -1;
int dataset_id = -1;
int memtype_id = -1;
int filetype_id = -1;
int plist = -1;
long[] dims = { DIM_X };
long[] chunk_dims = { CHUNK_X };
long[] maxdims = { HDF5Constants.H5S_UNLIMITED };
byte[][] dset_data = new byte[DIM_X][SDIM];
StringBuffer[] str_data = new StringBuffer[DIM_X];
// Initialize the dataset.
for (int indx = 0; indx < DIM_X; indx++)
str_data[indx] = new StringBuffer(String.valueOf("iteration "
+ (indx + 1)));
// Create a new file using default properties.
try {
file_id = H5.H5Fcreate(FILENAME_A,
HDF5Constants.H5F_ACC_TRUNC,
HDF5Constants.H5P_DEFAULT, HDF5Constants.H5P_DEFAULT);
} catch (Exception e) {
e.printStackTrace();
}
try {
filetype_id = H5.H5Tcopy(HDF5Constants.H5T_C_S1);
H5.H5Tset_size(filetype_id, SDIM);
plist = H5.H5Pcreate(HDF5Constants.H5P_DATASET_CREATE);
H5.H5Pset_layout(plist, HDF5Constants.H5D_CHUNKED);
H5.H5Pset_chunk(plist, 1, new long[] { 1024 });
H5.H5Pset_deflate(plist, 5);
dataset_id = H5.H5Screate_simple(1, new long[] { 25000000
}, new
long[] { HDF5Constants.H5S_UNLIMITED });
} catch (Exception e) {
e.printStackTrace();
}
// Write the data to the dataset.
try {
for (int indx = 0; indx < DIM_X; indx++) {
for (int jndx = 0; jndx < SDIM; jndx++) {
if (jndx < str_data[indx].length())
dset_data[indx][jndx] = (byte) str_data[indx]
.charAt(jndx);
else
dset_data[indx][jndx] = 0;
}
}
if ((dataset_id >= 0) && (memtype_id >= 0))
H5.H5Dwrite(dataset_id, HDF5Constants.H5T_C_S1,
HDF5Constants.H5S_ALL, HDF5Constants.H5S_ALL,
HDF5Constants.H5P_DEFAULT, dset_data);
} catch (Exception e) {
e.printStackTrace();
}
}
So my question, is this correct way of doing it and how do I connect
the dataset to the file. I guess it's time of creating the dataset.
After this is the way forward, like this
1. H5.H5Dextend(dataset_id, extdims);
2. dataspace_id = H5.H5Dget_space(dataset_id);
3. H5.H5Sselect_all(dataspace_id);
// Subtract a hyperslab reflecting the original dimensions from
// the
// selection. The selection now contains only the newly extended
// portions of the dataset.
count[0] = dims[0];
count[1] = dims[1];
H5.H5Sselect_hyperslab(dataspace_id,
HDF5Constants.H5S_SELECT_NOTB, start,
null,
count, null);
// Write the data to the selected portion of the dataset.
if (dataset_id >= 0)
H5.H5Dwrite(dataset_id, HDF5Constants.H5T_NATIVE_INT,
HDF5Constants.H5S_ALL, dataspace_id,
HDF5Constants.H5P_DEFAULT, extend_dset_data);
I also see that H5.H5Dextend is depricated, from version 1.8, is
there another method to user?
cheers, Håkon
On 19 March 2010 16:05, Peter Cao <[email protected]
<mailto:[email protected]>> wrote:
Hi Hakon,
I assume you are using 1D array of strings. Here are some hints
for you:
1) You may just use string dataype. You can use variable length
string if your strings have different size,
or you can use fixed length string if your strings are about
the same length, e.g.
tid = H5.H5Tcopy(HDF5Constants.H5T_C_S1);
for fixed length of 128
H5.H5Tset_size(128);
for variable length
H5.H5Tset_size(tid, HDF5Constants.H5T_VARIABLE
2) Set dataset creation property for chunking and compression
plist = H5.H5Pcreate(HDF5Constants.H5P_DATASET_CREATE);
H5.H5Pset_layout(plist, HDF5Constants.H5D_CHUNKED);
H5.H5Pset_chunk(plist, 1, new long[] {1024}); // set
the chunk size to be about 2MB for best performance
H5.H5Pset_deflate(plist, 5);
3) Set the dimension size, e.g.
sid = H5.H5Screate_simple(1, new long[]{25000000}, new
long[] {HDF5Constants.H5S_UNLIMITED});
Thanks
--pc
Håkon Sagehaug wrote:
Hi Peter
My problem is actually before I can create the dataset the
first time, I can't figure out the correct data type to use. I
guess I should use a byte type, since the strins are converted
to bytes
Håkon
On 19 March 2010 15:29, Peter Cao <[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>> wrote:
Håkon,
There was a typo in my previous email. You do NOT need to
read the
first chunk in order
to write the second chunk. You can just select whatever
chunks you
want to write.
Sorry for the misleading.
Thanks
--pc
Håkon Sagehaug wrote:
Hi Peter
I'm trying to do it with the read chunk by chunk, but
having
trouble creating the data set, in the example [1] it's done
like this
H5.H5Dcreate(file_id, DATASETNAME,
HDF5Constants.H5T_STD_I32LE,
dataspace_id, dcpl_id);
the type is for int, but I cant seem to find the
correct one
for string, in example[2] with string arrays t looks
like this,
H5.H5Dcreate(file_id, DATASETNAME, filetype_id,
dataspace_id,
HDF5Constants.H5P_DEFAULT);
If I create the dataset like this when I want to
dynamiccaly
add I can only get the first byte in each of the
string. Any
tips on what type I should use?
Håkon
[1]
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/java/examples/datasets/H5Ex_D_UnlimitedAdd.java
[2]http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/java/examples/datatypes/H5Ex_T_String.java
-- Håkon Sagehaug, Scientific Programmer
Parallab, Uni BCCS/Uni Research
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
<mailto:[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>>,
phone +47 55584125
------------------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
------------------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected] <mailto:[email protected]>
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected] <mailto:[email protected]>
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
------------------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
