>I have verified the lock contention problem by viewing mutex statistics.
>The
>offending nsv bucket got 100 times more locks than any other, and under
>heavy load was busy up to 48% of the time. Not good.
>
>I also verified that the hashtable in question was the primary source of
>the
>locks. It's a single nsv. It can be (and frequently is) used multiple times
>per page.
I did a little load/mutex testing and these are my results:
Setting: load test was using ab on local host and was using existing app
that has approx 70 main nsv arrays. For test I preloaded one of the arrays
with 100,000 extra distince values at server startup in addition to other
values that would be referenced on the page that ab would call. The targeted
page would reference the *heavy* array 13 times during test in addition to a
few other refs to other nsv in bucket. The page that was loaded had ~100 nsv
locks required for full load.
Am using aolserver 3.3.1 on linux 2.4.3 kernel. I have 11 buckets setup and
statistic/mutex tracking set on in config file. The tested page does 1 db
query.
Page Load Results for 1000 pages at concurrency 5
nsd7.6 returned 85 page/sec at 613 kb/sec
nsd8.x returned 63 page/sec at 452 kb/sec
Mutex results:
7.6 nsv array that contained the 100,000 entries grabed 17,100 locks during
test
and was busy 106 times. As a side note another nsv that had under 200
entries grabbed ~37,000 locks during test and was busy 192 times.
8.x test while slower in performance had less mutex contention with the
targeted
nsv with 100,000 entries only waiting 10 times for the 17,000 locks. The
other
entry that did wait 192 times was also reduced to 105 times
I have previously load tested this setup with over 1,000,000 objects loaded
(made the process over 400 Meg :) and did not notice any significant
performance decrease compared to non heavy cache. I am not convinced that
the number of entries in the hash is what is adversly effecting mutex
contention. I'm also not
convinced that load is effected too much by waiting but Im interested in
what may be causing these issues. Can you post more info ...and possibly
offending code samples?
Best regards,
Carl Garland
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
