ok guys, so I am still having this problem we are using php client with persistent connections (this is pooled connections in other words) using $memcache_obj->pconnect, which means that each php thread has it's own connection to memcached there are 20 http servers, each with minimum 20 php threads, so this is total of 4000 clients (and that is minimum)
however from about 3700 clients we are seeing performance decrease, the answer from memcached for a select can be up to 1 second, this is much, am I right? the memcached server from my calculations is getting about 5000 requests per second, is this a lot? this is high performance machine, there is absolutely no swapping *and never was*!!! the load average is 0.17!! Maybe I just need to install more memcached servers? but this one seems ot be doing nothing anyway below are the stats and stats settings, the problem is visible right now stats settings STAT maxbytes 3221225472 STAT maxconns 10000 STAT tcpport 11211 STAT udpport 11211 STAT inter NULL STAT verbosity 1 STAT oldest 0 STAT evictions on STAT domain_socket NULL STAT umask 700 STAT growth_factor 1.25 STAT chunk_size 48 STAT num_threads 3 STAT stat_key_prefix : STAT detail_enabled no STAT reqs_per_event 20 STAT cas_enabled yes STAT tcp_backlog 1024 STAT binding_protocol auto-negotiate END stats STAT pid 2709 STAT uptime 1727263 STAT time 1259772057 STAT version 1.4.1 STAT pointer_size 64 STAT rusage_user 29672.975022 STAT rusage_system 51701.090239 STAT curr_connections 3188 STAT total_connections 20452452 STAT connection_structures 3623 STAT cmd_get 5977832958 STAT cmd_set 665620469 STAT cmd_flush 0 STAT get_hits 5355401281 STAT get_misses 622431677 STAT delete_misses 0 STAT delete_hits 0 STAT incr_misses 0 STAT incr_hits 24302362 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT bytes_read 403401321019 STAT bytes_written 971168343526 STAT limit_maxbytes 7516192768 STAT accepting_conns 1 STAT listen_disabled_num 8554 STAT threads 3 STAT conn_yields 0 STAT bytes 198980367 STAT curr_items 1077436 STAT total_items 665620605 STAT evictions 0 END
