Hi,
I believe I have project very similar to yours. Maybe we could
cooperate and share some code (I use Apache 2 license).
MapDB provides concurrent HashMap and TreeMap backed by disk storage. It
is database engine for permanent storage, but I think it has
performance comparable to 'caching' solutions.
This project has been around since 2001 (known as JDBM before). MapDB
(aka JDBM4) is complete rewrite to support fine grained concurrency.
It is still under development, but already very usable. Checkout:
http://www.mapdb.org
I believe you need some good serialization library.
Here is general library to serialize most 'java.lang' and 'java.util'
classes (Long, ArrayList, Date...). It is very space efficient
https://github.com/jankotek/MapDB/blob/master/src/main/java/org/mapdb/SerializerBase.java
Here is general POJO serialization. It stores class format at single
place in database:
https://github.com/jankotek/MapDB/blob/master/src/main/java/org/mapdb/SerializerPojo.java
Regards,
Jan