> I run a memcached server on my server, after certain amount of > connections the next connections to this server are very slow (php is > waiting like 5s to connect) and the response time looks to be very > slow as well. The memory usage is very small, about 1.5GB from 8GB of > available ram and the load average and cpu usage is also very small, > so server is in no ways overloaded (load average shows never more than > 0.5). As far as I checked everything there are no OS limits doing any > problem here, I checked amount of available sockets, connections and > file descriptors > > I start the memcached this way: > > /usr/local/bin/memcached -d -u nobody -m 7168 -t 2 -P /var/run/ > memcached.pid -c 10000 -v >> /var/log/memcached.log 2>&1 > > the server is dual xeon 5130 2GHZ, 8GB ram, scsi disk, running redhat > enterprise linux 5.4 64 bit > > the memcached is the newest version > I login to my memcached server using telnet and I issue "stats" > command, I noticed the poor performance starts when the value of > curr_connections grows to around 3800. There are numerous servers > connecting to this memcache server, each one of them using about 200 > webserver threads > > any idea what might be causing the performance problem?
What version of memcached are you using? If you do a quick strace of the process, is it calling epoll_wait (and similar) or select()/poll()? Can you verify that yor host is not using any swap, and is not actively swapping? (watch vmstat 1 for a minute or two, the si/so columns). Are you using large multigets at all? Finally, can you pastebin the output of "stats" and "stats settings" somewhere? (assuming you're on 1.4) Thanks, -Dormando
