We did a lot of testing of RW locks in another multi-threaded project and we use successfully "PTHREAD_RWLOCK_PREFER_WRITER_NP" which works perfectly fine if you avoid recursive read locks. It is probably good practice to avoid any recursive mutex in general.
If one stay's with the default RW mutex without setting attributes we have seen that you starve writers for long periods if there is a contention on the read mutex (n-reader >> n-writer). I agree this is a very 'sensitive' topic but could well be an origin for threading scalability problems - depends how and where you use the RW locks in CEPH. Cheers Andreas. ________________________________________ From: Gregory Farnum [[email protected]] Sent: 09 September 2014 21:56 To: Sage Weil Cc: Andreas Joachim Peters; [email protected] Subject: Re: Question to RWLock & reverse DNS ip=>hostname On Tue, Sep 9, 2014 at 10:50 AM, Sage Weil <[email protected]> wrote: > On Tue, 9 Sep 2014, Andreas Joachim Peters wrote: >> Hi, >> by chance I had a look to the RWLock class. To my best knowledge the way >> you create RW locks it defaults to writer-starvation e.g. all readers >> will always jump a head of a pending writer. I cannot imagine that you >> never have the opposite requirement in the CEPH multithreaded code but I >> didn't review where it is used. In case you are aware, you can just >> ignore this comment, otherwise one could add the option to create a >> writer-prefering RWLock at construction time using e.g. >> pthread_rwlockattr_setkind_np(&attr, PTHREAD_RWLOCK_PREFER_WRITER_NP) > > Hmm, I was not aware. Perhaps we should make the reader vs writer > preference explicit during construction? Yeah, this is interesting as I thought I'd learned somewhere that the default behavior was to prefer writers. Poking around the internet makes me wonder if we actually want to make any changes — best I can tell PTHREAD_RWLOCK_PREFER_WRITER_NP does not actually behave the way one wants (as part of the spec!) because pthreads requires recursive locking, so we'd have to use PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP. But we'd probably want to profile the performance changes (and, uh, run tests to make sure we don't deadlock somehow) before putting any such code upstream. >> Another question: I couldn't find a base implementation to translate >> IPV4/6 ip's to host names? Is there same configuration table/method >> allowing to get back from an OSD address retrieved via >> OSDMap::get_addr() the original host name avoiding the use of DNS >> lookups? > > I don't think we are doing reverse DNS lookups anywhere in the ceph > code ... In particular I think we've looked at doing this before and been unable to reliably map to the *proper* hostname in cases where servers have multiple NICs. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
