how does one unsubscribe from this list? On Sat, Nov 21, 2009 at 1:43 PM, Chris F <[email protected]> wrote:
> I have a small memcached server pool of 3 instances(m1.small) and an > autoscaling cloud of linux instances(4-20 c1.xlarge AWS EC2) that > store sessions to them using the memcache PECL extension wrapped by > Symfony 1.2 framework. > > In the past week we've had an increase in traffic which has resulted > in a recurring latency issue caused by memcache. > > We see the network throughput of our web servers slowly creep up over > an hour then drop suddenly. > Please see the links below to the zabbix graph pics. > > As the network throughput rises on both web and memcache servers the > latency rises. > The web traffic does not correlate directly with the dropoffs in > network throughput that can be seen on the web servers. Analyzing the > traffic using ntop showed 98% of the traffix was from memcache. > > It seems that the pecl memcache extension is not properly hashing the > keys evenly across the 3 memcache servers. In fact I had added mem3 > to alleviate the load with no significant gain. > Also the amount of memcache traffic seems highly suspect. > > Can someone assist in diagnosing this issue? > See below for additional info and links. > > Thanks > > Chris > > ELB Latency statistics: http://imgur.com/ZM4mh.png > Memcache server network utilization: http://imgur.com/Sg8fJ.png > PHP server network utilization: http://imgur.com/zxzWU.png > Memcache server CPU load: http://imgur.com/tW8La.png > PHP server CPU load: http://imgur.com/S5wio.png > WWW requests per minute: http://imgur.com/242eV.png > > pecl memcache settings: > Active persistent connections 1 > Version 2.2.5 > Revision $Revision: 1.111 $ > > Directive Local Value Master Value > memcache.allow_failover 1 1 > memcache.chunk_size 32768 32768 > memcache.default_port 11211 11211 > memcache.default_timeout_ms 1000 1000 > memcache.hash_function crc32 crc32 > memcache.hash_strategy consistent consistent > memcache.max_failover_attempts 20 20 > > memcache server setting: > memcached -d -p 11211 -u nobody -m 1024 -c 2048 -P /var/run/memcached/ > memcached.pid > v 1.4.0 > -- "Never accept what you hear at face value -- ask questions, do your homework and make sure your interests are protected"
