dormando wrote:
I run a memcached server on my server, after certain amount of
connections the next connections to this server are very slow (php is
waiting like 5s to connect) and the response time looks to be very
slow as well. The memory usage is very small, about 1.5GB from 8GB of
available ram and the load average and cpu usage is also very small,
so server is in no ways overloaded (load average shows never more than
0.5). As far as I checked everything there are no OS limits doing any
problem here, I checked amount of available sockets, connections and
file descriptors
I start the memcached this way:
/usr/local/bin/memcached -d -u nobody -m 7168 -t 2 -P /var/run/
memcached.pid -c 10000 -v >> /var/log/memcached.log 2>&1
the server is dual xeon 5130 2GHZ, 8GB ram, scsi disk, running redhat
enterprise linux 5.4 64 bit
the memcached is the newest version
I login to my memcached server using telnet and I issue "stats"
command, I noticed the poor performance starts when the value of
curr_connections grows to around 3800. There are numerous servers
connecting to this memcache server, each one of them using about 200
webserver threads
any idea what might be causing the performance problem?
What version of memcached are you using?
If you do a quick strace of the process, is it calling epoll_wait (and
similar) or select()/poll()?
Can you verify that yor host is not using any swap, and is not actively
swapping? (watch vmstat 1 for a minute or two, the si/so columns).
Are you using large multigets at all?
Finally, can you pastebin the output of "stats" and "stats settings"
somewhere? (assuming you're on 1.4)
And finally: what is the connect time when you telnet to the port to run
the stats command... if that's faster than the 5 sec your php clients
are waiting, I would start looking at the client boxes first..
Cheers,
Trond
