Chris,
I cannot tell what your problem may be from the information provided.
However, I might suggest instead trying PECL memcached (memcached ending
with a 'd') as it uses libmemcached and could possibly perform better
than PECL memcache.
regards,
Patrick
Chris F wrote:
I have a small memcached server pool of 3 instances(m1.small) and an
autoscaling cloud of linux instances(4-20 c1.xlarge AWS EC2) that
store sessions to them using the memcache PECL extension wrapped by
Symfony 1.2 framework.
In the past week we've had an increase in traffic which has resulted
in a recurring latency issue caused by memcache.
We see the network throughput of our web servers slowly creep up over
an hour then drop suddenly.
Please see the links below to the zabbix graph pics.
As the network throughput rises on both web and memcache servers the
latency rises.
The web traffic does not correlate directly with the dropoffs in
network throughput that can be seen on the web servers. Analyzing the
traffic using ntop showed 98% of the traffix was from memcache.
It seems that the pecl memcache extension is not properly hashing the
keys evenly across the 3 memcache servers. In fact I had added mem3
to alleviate the load with no significant gain.
Also the amount of memcache traffic seems highly suspect.
Can someone assist in diagnosing this issue?
See below for additional info and links.
Thanks
Chris
ELB Latency statistics: http://imgur.com/ZM4mh.png
Memcache server network utilization: http://imgur.com/Sg8fJ.png
PHP server network utilization: http://imgur.com/zxzWU.png
Memcache server CPU load: http://imgur.com/tW8La.png
PHP server CPU load: http://imgur.com/S5wio.png
WWW requests per minute: http://imgur.com/242eV.png
pecl memcache settings:
Active persistent connections 1
Version 2.2.5
Revision $Revision: 1.111 $
Directive Local Value Master Value
memcache.allow_failover 1 1
memcache.chunk_size 32768 32768
memcache.default_port 11211 11211
memcache.default_timeout_ms 1000 1000
memcache.hash_function crc32 crc32
memcache.hash_strategy consistent consistent
memcache.max_failover_attempts 20 20
memcache server setting:
memcached -d -p 11211 -u nobody -m 1024 -c 2048 -P /var/run/memcached/
memcached.pid
v 1.4.0
