2014-10-30 18:05 GMT-07:00 J.T. Conklin <j...@acorntoolworks.com>:
> Daniel Pocock <dan...@pocock.pro> writes:
>
> > - 3.7 adds a new dependency, Concurrency Kit
>
> Given the problems you describe with Concurrency Kit, is it simply
> premature for ganglia to depend on it? From what I can tell, its
> use is isolated to lib/hash.[ch], and at first glance looks like it
> an alternate reader/writer lock implementation could be selected at
> runtime. Are the performance benefits of CK rwlocks really worth the
> added complexity, especially as ganglia continues to depend on APR,
> which also provides a rwlock implementation?
>
Originally, I added CK in for the spinlocks because it seemed they would be
held for very little time. However, it seems like a number of people are
running into scenarios where gmetad usage spikes. I don't know what APR's
rwlock has over pthreads, but it might make sense to switch back to a lock
type that goes to sleep, given that there's significant contention over
these locks when iterating the hash table. (Actually, I just looked at
APR's rwlock implementation and it seems to be just a thin wrapper around
pthreads on any OS I care about. So I'd be fine with that replacement.)
After giving it a fair amount of thought over the last few months, I think
our use case would not suffer greatly. Most of our performance improvements
came with the ability to multiplex requests to rrdcached over TCP instead
over a unix domain socket.
That all said, there was a reason that I introduced a dependency to CK. The
current hash table implementation avoids callers needing to think about
thread safety by doing a huge amount of allocations and copies, which
really hurts performance (especially on iterations, which have a huge
impact with our number of metrics). gmetad is a great use case for
Concurrency Kit in general, and I had hoped that putting it in would make
it easier to do more performance work on it in the future. (Sadly, I just
haven't had the time.)
I'll catch up with Vlad about testing our metrics with APR rwlocks and see
if that's going to work for us. (I think it will.) However, I would also
like to solve whatever CK packaging problems are outstanding, as I would
like to use it in future improvements.
--dho
--jtc
>
> --
> J.T. Conklin
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Ganglia-developers mailing list
> Ganglia-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ganglia-developers
>
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
