On Wednesday, July 10, 2013 8:42:42 AM UTC-7, Marco Bonardo wrote:
On 06/07/2013 11:26, Philip Chee wrote:
LMDB is an ultra-fast, ultra-compact key-value data store developed by
Symas for the OpenLDAP Project. It uses memory-mapped files, so it has
the read performance of a pure in-memory database while still offering
the persistence of standard disk-based databases, and is only limited to
the size of the virtual address space, (it is not limited to the size of
physical RAM).
It looks like LMDB has speed and size advantages on memory constrained
systems like Android and B2G.
Ehr, I wrongly replied instead of following-up, let me paste again:
As for many other dbms around, comparisons are pretty much hard, just
relying on microbenchmarking doesn't help much. What's best, LMDB,
levelDB, kyotoCabinet, Sqlite4? It's hard to tell just by looking at
these graphs, you'd need measurements done directly on your most
compelling use-cases. Just getting those measures is a project by itself.
First you need to distinguish SQLite (3 or 4) from the others, which are simply
key/value stores. Do you really need complex multi-column tables and complex
SQL queries for managing your data? I don't see why; you would probably do much
better with just a K/V store.
As for which is better - it's no contest. KyotoCabinet leaks pages like a sieve.
http://www.anchor.com.au/blog/2013/05/second-strike-with-lightning/
Test LMDB in your own applications, you'll probably find that it massively
outperforms whatever you were using before, as many others have.
https://github.com/tspurway/pymdb-lightning
Then there's the cost of conversion and added code size, for which
Sqlite4 may be a winner, considered we'd need less changes and we'd
likely have more code reuse from Sqlite3.
Then there's memshrink, I couldn't find the maximum and average used
memory in the reported benchmarks, we are currently using a max of 2MB
per DB connection, would it be comparable?
I finally wonder how they solved the problem of mmap corruption, also
Sqlite can use mmap IO, and it may be up to ten times faster, but has no
protection for memory corruption due to stray pointers or buffer
overflows in the application.
LMDB is protected from all of these problems. It uses a read-only memory map,
so stray pointer writes will kill the app with SEGV, nothing corrupts the data
on disk.
So, it's definitely an option to consider when we decide to start
working on a new generic storage, but we need far more details to be
able to properly evaluate.
Fwiw, Sqlite4 will support swappable engines, so you can use lmdb as the
Sqlite4 engine, and a backend is already under development from what I
read on the lmdb page.
Yes, but SQLite4 is very very far away from ready for public consumption. The
storage engine interface is pretty simple, plugging LMDB in is trivial. But
until the frontend work is done (query optimizers and all that) it's not going
to do anyone any good.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform