Hi All,
We were debugging the CPU usage in a ctl_conversationsdb rebuild yesterday, and
noticed an interesting thing. 70% of the CPU utilisation for this one process
was inside the kernel! Mostly with dirty pages.
ctl_conversationsdb -R is particularly heavy on the twoskip database - it's
rewriting a lot of random keys. This leads to writes all over the place, as it
stitches records into the skiplists.
Of course the "real answer"[tm] is zeroskip, which doesn't do random writes -
but until then, we suspect that the cost is largely due to the face that we use
mmap to read, and fwrite to write! We know that might be less efficient already
from Linus' comments about 10 years ago! And I guess here's the proof.
An option would be to switch to using mmap to write as well. We could easily
modify lib/mappedfile to memcpy to do the writes.
Does anybody see any strong reason not to?
Bron.
--
Bron Gondwana, CEO, FastMail Pty Ltd
br...@fastmailteam.com
