I used a mutex in implementing mod_spamhaus, mod_sorbscheck and mod_torcheck to maintain consistency across threads. I have not seen problems since.
I found the Apache mutex code to be somewhat opaque, so I wrote my own. I did find that "interesting" but after several revisions I had something I could return to later on and still understand, and (possibly more importantly) I felt it would be fairly clear to an unsophisticated reader. http://www.miim.com/software/linux/index.shtml
