all I have a patch that can speed up parallel ldap add/delete a lot; in my testing about 2.5X. The idea is described here: the mdb backend does not really support parallelism in rw transactions, so add/delete are serialized by a mutex inside the mdb backend as small transactions. In my patch, all add/delete are put in a queue and executed in a dedicated worker thread. It may seem pointless at first, but by doing them all in the same thread, now I can merge concurrent ops from different clients into larger transactions, and reduce the number of expensive txn_commit calls (fsyncs).
my test results are ( your mileage may vary) 10 thread parallel add: 1, unpatched slapd + default config, ~ 2800 op/s (performance stable over time) 2, unpatched slapd + dbnosync ~9000 op/s (lots of fluctuation due to background dirty page flush) 3, unpatched slapd + dpnosync + checkpoint every minute ~8000 op/s, with reduced fluctuation 4, patched slapd ~7300 op/s (fluctuation nearly gone) 2 is not a recommended config, because crashing or power lost will cause massive data lost. 3 is better, but still up to 1 minutes of data can be lost at crash. With this patch, I can achieve >90% of the performance of 3, with data durability as good as 1; or 2.5X performance of 1 with same data durability guaranty. The patch still need some tidy up; is this email list the place to send patches? -- Derek Zhou Shannon Systems http://www.shannon-sys.com
