I need to better understand the best way to configure the various
environment flags related to sync and map.

When we converted from BDB to LMDB, we instinctively kept #MDB_NOSYNC true.
This proved to make the database subject to corruption if it was killed or
stopped too harshly. It also had the effect on Windows at least to make
Windows gradually use up all available RAM for the memory mapped file and
to bring the server to its knees.

After carefully reading the documentation about leaving "the system with no
hint for when to write transactions to disk" when MDB_NOSYNC was turned on,
we completely went without setting any options. The results were that we
can't corrupt the database, but the speed degradation makes this approach
unusable, as we are very heavily write focused (at least in this part of
our process, where we perform analysis and load the database with 100's
millions of records).

In looking at the documentation I'm tempted to try MDB_WRITEMAP with
MDB_MAPASYNC but do I then need to also issue some manual mdb_env_sync and
if so at what frequency and what should trigger this?

What are best practices combinations of those flags here. We absolutely
can't afford database corruption, but we can deal with one (or maybe more
with some re-design) transactions that are lost.

Please guide us.

Thanks
Alain

Reply via email to