Hi,

I am considering using LMDB to store a dynamic graph structure, and it will
be very helpful to support "append to value" having the same semantics as
"append" in Kyoto Cabinet.
http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1BasicDB.html#a23e776e5bd1e3c5caa0f62edffb87a54

For instance, if I were to store an adjacency list representation of a
graph in LMDB, I would use vertex ID as the key, and the value is a vector
of 64-bit IDs. To insert an edge into the graph will simply require
appending 8 bytes to the end of the vector. Re-reading and re-writing the
entire value will be somewhat excessive (and will actually change the
asymptotic runtime of the insertion).

This is also one of those situations where a batch append might be useful.
For instance, if I have an edge list on file which is too large to retain
in memory. To convert it to an adjacency list in LMDB will require
batch-append. Batch-writes are insufficient since that will require me to
cache at least, large parts of the adjacency list in memory, and the
original edge list could be ordered arbitrarily.

The actual graph representation I am thinking of is somewhat more
intelligent than a straight adjacency list, but similar concepts hold.

I have tried looking through the lmdb source to try to implement append
myself, but the put logic is somewhat complicated...

Yucheng

Reply via email to