Dustin wrote:
On Mar 5, 7:11 pm, Toru Maesaka <[email protected]> wrote:
As for the global stats_lock, I think its safe to assume that most of
the operations are write related so read-write locking is definitely
not the answer here. I think Facebook was dead-on with their solution
on scaling this area up (use scoreboarding per thread basis and
aggregate the results when needed).
Trond's already done that per-thread work and pushed it, so it's
ready in that branch.
I moved the "command counters" to a thread local structure and aggregate
the result when you execute the stats command. Today I pushed a patch
moving the bytes_read / bytes_written to the thread local structure as well.
He's currently working on adding more locks to
a single hash table (a la java's ConcurrentHashMap).
The read/write stuff sounds pretty interesting, though.
I haven't had the time to look at the result of the benchmark of the
partitioned hash table prototype, but I'll let you know what I find out :-)
To begin with I'm going to only experiment with using rw-locking for
cache_lock. I'll test the performance with libmemcached's memslap and
let you all know how it turned out... hopefully in not so distant
future.
Great. Trond still doing some work to reduce contention, that might
be an alternative approach. I think he was pretty far along with that
work, though. The results would be useful for general mutex strategy
advice.
make test works for all tests except stats detailed (because I haven't
implemented those yet). The lru test fails because right now each
partition have its own LRU... This started out as a quick prototype (I
hacked it together while I was watching TV with my girlfriend
yesterday), so if the results looks promising I'll clean it up and push
it to github so that all of you may look at it :)
Now that we don't support non-threaded memcached servers anymore I would
love to move the locks out of the threads.c file, and use more fine
grained locking.
Cheers,
Trond
