Alberto Mardegan wrote: > Hi all! > I'm trying to understand the impact that a LDMB database will have on > the flash wear of an embedded device. > > In order to minimize disk writes, I'm keeping a DB open all the time, > and I'd like to have the changes written into the physical storage only > when mdb_env_sync() is called (near the end of my process lifetime). > I'm opening the DB with the MDB_NOSYNC | MDB_WRITEMAP | MDB_NOMETASYNC > flags, and if I strace the executable I see that lmdb behaves as > expected: no writes happen until I call mdb_env_sync(). > > However, the writes end up to the physical storage anyway, because the > kernel reserves the right of flushing changes of an mmap'ed file to disk > at any time (as per the documentation of MAP_SHARED). I don't exactly > know if this happens because some other process is calling sync(), or > which criteria the kernel follows, but as a matter of fact a write > happens every few seconds. > > Looking at the code, it looks like it's impossible to instruct LMDB to > use MAP_PRIVATE without modifying the code. But before exploring that > solution, I'd like to be sure I understand the real behaviour of LMDB. > > Do I understand correctly, that if during the lifetime of my process the > same key is assigned different values, with mdb_txn_commit() being > called each time, the same addresses in the shared memory area will be > changed several times (leading to unnecessary re-writes of the same area > of the flash storage)? Or is LMDB only operating in "append" mode, so > that it tries to avoid writing the same area twice?
LMDB never overwrites live data. It always does copy-on-write, except for the two meta pages, of which one is written on every sync. In my own experience I've found that a microSD card used to store a blockchain with LMDB wore out (started returning hard I/O errors) in about 3 years of continual use. If you're using Linux you can control the frequency of periodic flushes by tweaking the writeback parameters in /proc/sys/vm. If you're using some other embedded OS I have no idea. Trying to use MAP_PRIVATE would be stupid, the OS would still flush it as frequently as before, but it would flush to your swap device instead of a user file. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
