Hi Kim,
A compound structure of {key,String,int,int} in HDF5 is a good way to go.
Yes, you can write data line by line. Make sure to use chunked layout for
better performance. Check the following page for more information on chunks:
http://www.hdfgroup.org/HDF5/doc/Advanced/Chunking/index.html
HDF5 does not support direct query by keys. Fast access to data in file
by keys
will not be easy. You could store a structure of keys, e.g. (key, index)
in file or
a hierarchical mapping of keys to index. A good solution will depend on
your applications such as the pattern of your keys.
Thanks
--pc
On 7/18/2011 3:20 PM, kim nguyen wrote:
Hi Peter,
Thanks for your answer.
My issue :
-In a java program i use a hashtable, and this hashtable is to huge for memory.
-First i need to write data in this HDF5 structure.
-Secondly i need to read object {String , int, int} by key {string}.
I think that i can use HDF5 to serialize this structure in a table
{key,String,int,int} :
First a can read a file and for each line add a line in the HDF5 table.
After i need a fast access to this HDF5 table by key.
Thanks a lot for your help.
kim
2011/7/18 Peter Cao<[email protected]>:
Kim,
It would be very helpful if you can explain the issue in more details.
My guess is that you are trying to read compound data with structure
of {string, int} from HDF5 file into Java hashtable. Similar for writing.
Currently, there is no direct way to read/write Java hashtable from/to HDF5
files. There are a couple of ways that you can do.
A) Implement your own Java and JNI write functions that can read/write
hashtable, or
B) Use H5CompoundDS to read data from file and create the hashtable
once you have the data in memory, similar for write.
Option A) will have better performance but it will need more work. Option B)
does not need much work but it requires more memory and extra data
conversion.
Thanks
--pc
On 7/18/2011 1:21 PM, kim nguyen wrote:
I am totally newbie with HDF.
I want to use HDF5 to serialize a hashTable. This hashTable is to
large to be managed in memory and i want use HDF for read performance.
The hash table have a String for key and a object (1 string and 2
int) for value.
Have you a java sample code for create this structure, write data and read
data.
Best regards
kim
France
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
