Full_Name: Leonid Yuriev Version: 2.4.40 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (31.130.36.33)
Solution for: ITS#7841 and "OpenLDAP + LMDB Back-End - request 300719-14-EXO" When using LMDB as a backend under the heavy load with add/modify/delete transactions, a huge number of disk writes is generated. In generally this patchset give a bonus of 10-100 times write-performance at the cost of consistency on disk in a one second. 1. Adds a configurable LIFO-policy for reclaiming of FreeDB records. Thus, only a small subset of pages will be updated and re-written on disk repetitive. This allow storage subsystem to effective combine such disk writes. As a result write-performance grow up to 100 times in case of write-back cache or "writemap" mode. 2. Checkpoints with consistency and a second exactness. It is possible and very useful the following settings, for example: envflags writemap nosync lifo checkpoint 0 1 3. Related bugfixes and minor extensions. -- The attached files is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Peter-Service LLC, Moscow, Russia. Peter-Service LLC has not assigned rights and/or interest in this work to any party. I, Leonid Yuriev am authorized by Peter-Service LLC, my employer, to release this work under the following terms. Peter-Service LLC hereby places the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice. commit 841059330fd44769e93eb4b937c3ce42654fad6f Author: Leo Yuriev <[email protected]> Date: 2014-09-20 07:16:15 +0400 BUGFIX - lmdb: lock meta-pages in writemap-mode to avoid unexpected write, before the data pages would be synchronized. Without locking the meta-pages may be writen by OS before other data, in this case database would be inconsistent. commit 6240c3350e8bd86337c7e41722cf6a38881f15e7 Author: Leo Yuriev <[email protected]> Date: 2014-09-12 01:32:13 +0400 BUGFIX - lmdb: reordering of instructions which update the txn in a meta-page. Without "volatile" or memory-barrier compiler may reorder instructions for update the "mm_txnid" field in meta-page in "writemap" mode. From the reader's point of view this cause a short time interval when the transaction is corrupted. commit accef62de7fe5660f870f4c5da319a2a8098b2fb Author: Leo Yuriev <[email protected]> Date: 14-0-09-21 02:29:50 +0400 BUGFIX - lmdb: 'volatile' to important fields, which may be updated by readers asynchronously. Without 'volatile' compiler may eliminate a mdb_find_oldest() calls. commit bb83e03cf1b8bceee64550229c3becbdd5400680 Author: Leo Yuriev <[email protected]> Date: 2014-09-19 20:18:17 +0400 FEATURE - lmdb-backend: support config for 'lifo' and 'coalesce' envflags. commit 0c168d0e63ed78d13df3fc8a42f3667335678639 Author: Leo Yuriev <[email protected]> Date: 202014-09-20 10:13:28 +0400 FEATURE - lmdb: MDB_LIFORECLAIM & MDB_COALESCE modes. Reclaim FreeDB in LIFO order - this is a main feature. Also aim to coalesce small FreeDFB records. commit 8ddd63161aeb2689822d1a8d27385d62e4e341ae Author: Leo Yuriev <[email protected]> Date: 2014-09-19 22:47:19 +0400 BUGFIX - lmdb: properly sync meta-pages in mdb_sync_env(). Meta-pages may be updated during data-syncing in mdb_sync_env(), in this case database would be inconsistent. Check-and-retry if lead txn-id changed during flushing data in mdb_sync_env(). commit 908677f989588d06b9f00620576dea3c5c8675d7 Author: Leo Yuriev <[email protected]> Date: 2014-09-04 16:10:05 +0400 FEATURE - lmdb-backend: support for "checkpoint kbytes" config-option. commit 147f41a8110f28456bc32123bde86d47183f9c0a Author: Leo Yuriev <[email protected]> Date: 2014-09-04 16:01:15 +0400 FEATURE - lmdb: implementation of "checkpoint kbytes". A0A Force flush when volume of the changes reached a configurable threshold. commit fb82a0b688f4c31313d0790415feda8aaa18651c Author: Leo Yuriev <[email protected]> Date: 2014-09-04 15:18:16 +0400 CHANGE - lmdb-backend: checkpoint-interval in seconds instead of minutes. commit fc409d89e0d9dde20f612e34c2a463c8a81ea000 Author: Leo Yuriev <[email protected]> Date: 2014-09-20 06:51:04 +0400 EXTENSION - lmdb: more usefull info from mdb_stat tool. commit ccc7da690ffbff440643295b945fdf7886f48c97 Author: Leo Yuriev <[email protected]> Date: 2014-09-05 00:19:16 +0400 TRIVIA - lmdb: clean testdb-dir while "make test".
