Hello, I am working for a big web site. We have around 9000 hits/s on our MySQL replication trees and 500 000 unique visitors each day, just to give a clue about the load we are experiencing. We run on MySQL, Apache2, Gentoo, PHP 4 + PECL Memcache module. We've been using a single 12G memcached instance for speeding up things (we've reached the point where we can't solely rely on our DB). Using a single instance is not what memcached is meant for, so we decided to scale things up a bit, so we added 12 more instances, 2G each (32 bit servers, 4 instances per server, 3 servers). Then we switched from the "standard" (naive) method of hash distribution to the "consistent" method.
What happened was that the load on our web nodes (we have 3 of them) went up about 3 times the usual. I'm guessing it's the new hash distribution method that's doing this. Am I missing something or using this method is always so CPU intensive? Do we have another choice or we should invest in more web nodes, to distribute the new load if we decide to stick to the consistent hashing algorithm?
