On 03/02/15 22:30, Devon H. O'Dell wrote: > 2014-10-30 18:05 GMT-07:00 J.T. Conklin <j...@acorntoolworks.com > <mailto:j...@acorntoolworks.com>>: > > Daniel Pocock <dan...@pocock.pro <mailto:dan...@pocock.pro>> writes: > > > - 3.7 adds a new dependency, Concurrency Kit > > Given the problems you describe with Concurrency Kit, is it simply > premature for ganglia to depend on it? From what I can tell, its > use is isolated to lib/hash.[ch], and at first glance looks like it > an alternate reader/writer lock implementation could be selected at > runtime. Are the performance benefits of CK rwlocks really worth the > added complexity, especially as ganglia continues to depend on APR, > which also provides a rwlock implementation? > > > Originally, I added CK in for the spinlocks because it seemed they would > be held for very little time. However, it seems like a number of people > are running into scenarios where gmetad usage spikes. I don't know what > APR's rwlock has over pthreads, but it might make sense to switch back > to a lock type that goes to sleep, given that there's significant > contention over these locks when iterating the hash table. (Actually, I > just looked at APR's rwlock implementation and it seems to be just a > thin wrapper around pthreads on any OS I care about. So I'd be fine with > that replacement.) > > After giving it a fair amount of thought over the last few months, I > think our use case would not suffer greatly. Most of our performance > improvements came with the ability to multiplex requests to rrdcached > over TCP instead over a unix domain socket. > > That all said, there was a reason that I introduced a dependency to CK. > The current hash table implementation avoids callers needing to think > about thread safety by doing a huge amount of allocations and copies, > which really hurts performance (especially on iterations, which have a > huge impact with our number of metrics). gmetad is a great use case for > Concurrency Kit in general, and I had hoped that putting it in would > make it easier to do more performance work on it in the future. (Sadly, > I just haven't had the time.) > > I'll catch up with Vlad about testing our metrics with APR rwlocks and > see if that's going to work for us. (I think it will.) However, I would > also like to solve whatever CK packaging problems are outstanding, as I > would like to use it in future improvements. >
I made an effort to get ck into Debian and Ubuntu The problems I encountered are likely to be much the same for Fedora and EPEL so if we solve it in Debian then the rest will be easier too. The links to full build logs are here: https://packages.qa.debian.org/c/ck.html Notice that 0.3.5 made it into testing and 0.4.4 is stuck in unstable because of the build failures. The CK developers were willing to help and I indicated that I could sponsor them for a guest account on the build machines where CK fails so they can get to the bottom of it. In any case, if gmetad can run without ck then we should probably make that a 3.8.x release series, e.g. Ganglia 3.8.0 tag. The 3.7.x stuff can then become a branch for now and maybe CK support can be merged again in future when distributions are carrying it. ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
