hubou...@microsoft.com wrote: > Hi everyone, > > We’ve been stress‑testing an LMDB database by continually inserting and > deleting records ranging from 4 KB to 5 MB. To track utilization, we rely on > two independent metrics: > Application index – we track the sum of the payload bytes of keys currently > present. > mdb_stat output – pages in use /overlay etc.... > The two figures agree closely. > > Despite this, we repeatedly hit MDB_MAP_FULL when both metrics say the map is > only ≈ 50 % occupied (on both metrics). > Our tentative conclusion is heavy page fragmentation: LMDB can’t find a > sufficiently large contiguous run of free pages even though half of the map > is nominally free. > Questions > - Is it normal to lose that much effective capacity with an insert/delete > workload like ours?
Yes. LMDB is not meant for write intensive workloads, it's for read-heavy workloads. > - How far can we realistically fill the map before hitting fragmentation > limits? > - Are there recommended ways to mitigate or “defragment”? The only way is to make a new copy of the environment, e.g. using mdb_env_copy(). > > Any advice would be greatly appreciated! > > Thanks! > Hugues > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
