> Dormando,Yes, have to admit - we cache too aggressively (just do not want to > use different less polite word :)). > > Going to do two test experiments: enable compression and auto reallocation. > Before doing this: > 1) why auto reallocation is not enabled by default, what issues/disadvantage > to expect?
Because it pulls memory from other places and evicts those items regardless of if they were still valid or expired. There's no way for it to reassign slab pages of "just expired memory". Some people would prefer to just let evictions fall from the tail (least used) rather than do this, so we didn't change the defaults after introducing the feature. > 2) why memcached does not have compression on server side if CPU is idle, > because of ideology to keep it simple and fast? (just asking) I said already: in typical use case there are many more clients, and a very high rate of usage. If you flipped where the compression happens the server would run out of CPU very quickly, and be much more latent. We could support it in the server but it'd be a very low priority feature. > On Tuesday, May 6, 2014 6:40:07 PM UTC-7, Dormando wrote: > > Hi Dormando, > > Full Slabs and Items stats are below. The problem is that other slabs > are full too, so rebalancing is not trivial. I will try to > create a wrapper > > that will do some analysis and do slab rebalancing based on stats > (the idea to move try to shrink slabs with low eviction but need to > think more). > > But i see there is "Slabs Automove" in protocol.txt. Do you recommend > it? > > If it fits your needs. Otherwise, write an external daemon that controls > the automover based on your own needs. > > > >You either need to add more memory to the total system or rebalance > them. > > we run many-many memcached servers with 30Gb+ memory each box. And > the problem occurs on some boxes periodically. So I am thinking > how to convert > > manual restart to automatic action. > > I'm not sure why restarting will fix it, if above you say "rebalancing > is > not trivial". If restarting would fix it, rebalancing would also fix it. > > From the stats below, you do have a fair amount of memory spread out > among the higher order slab classes. Compression, or otherwise > re-evaluating how you store those values may make a big difference. > > There's also a huge amount of stuff being evicted without ever being > fetched again. Are you caching too aggressively, or is memory just way > too > small and they never get a chance to be fetched after being set? > > I'm just eyeballing it but evicted_time seems pretty short (a matter of > hours). That's the last access time of the last object to be evicted... > and it's like that across most of your slab classes. > > So, shuffle and compress and whatnot, but I think you're out of ram > dude. > > > server > > stats > > STAT pid 15480 > > STAT uptime 2476264 > > STAT time 1399422427 > > STAT version 1.4.15 > > STAT libevent 1.4.13-stable > > STAT pointer_size 64 > > STAT rusage_user 639012.117392 > > STAT rusage_system 2076810.323840 > > STAT curr_connections 5237 > > STAT total_connections 122995977 > > STAT connection_structures 23402 > > STAT reserved_fds 40 > > STAT cmd_get 91928675147 > > STAT cmd_set 4358475896 > > STAT cmd_flush 1 > > STAT cmd_touch 0 > > STAT get_hits 85005900667 > > STAT get_misses 6922774480 > > STAT delete_misses 4238049567 > > STAT delete_hits 885535057 > > STAT incr_misses 0 > > STAT incr_hits 0 > > STAT decr_misses 0 > > STAT decr_hits 0 > > STAT cas_misses 1074 > > STAT cas_hits 4784930 > > STAT cas_badval 14966 > > STAT touch_hits 0 > > STAT touch_misses 0 > > STAT auth_cmds 0 > > STAT auth_errors 0 > > STAT bytes_read 32317259718167 > > STAT bytes_written 221039272582722 > > STAT limit_maxbytes 25769803776 > > STAT accepting_conns 1 > > STAT listen_disabled_num 0 > > STAT threads 8 > > STAT conn_yields 0 > > STAT hash_power_level 25 > > STAT hash_bytes 268435456 > > STAT hash_is_expanding 0 > > STAT slab_reassign_running 0 > > STAT slabs_moved 0 > > STAT bytes 23567307974 > > STAT curr_items 32559669 > > STAT total_items 61290586 > > STAT expired_unfetched 6664504 > > STAT evicted_unfetched 1244432758 > > STAT evictions 2522683859 > > STAT reclaimed 7626148 > > END > > > > > > > > stats slabs > > STAT 1:chunk_size 96 > > STAT 1:chunks_per_page 10922 > > STAT 1:total_pages 1 > > STAT 1:total_chunks 10922 > > STAT 1:used_chunks 0 > > STAT 1:free_chunks 10922 > > STAT 1:free_chunks_end 0 > > STAT 1:mem_requested 0 > > STAT 1:get_hits 9905 > > STAT 1:cmd_set 10362 > > STAT 1:delete_hits 9582 > > STAT 1:incr_hits 0 > > STAT 1:decr_hits 0 > > STAT 1:cas_hits 0 > > STAT 1:cas_badval 0 > > STAT 1:touch_hits 0 > > STAT 2:chunk_size 120 > > STAT 2:chunks_per_page 8738 > > STAT 2:total_pages 1 > > STAT 2:total_chunks 8738 > > STAT 2:used_chunks 13 > > STAT 2:free_chunks 8725 > > STAT 2:free_chunks_end 0 > > STAT 2:mem_requested 1350 > > STAT 2:get_hits 1309125 > > STAT 2:cmd_set 2963710 > > STAT 2:delete_hits 199018 > > STAT 2:incr_hits 0 > > STAT 2:decr_hits 0 > > STAT 2:cas_hits 770681 > > STAT 2:cas_badval 3697 > > STAT 2:touch_hits 0 > > STAT 3:chunk_size 152 > > STAT 3:chunks_per_page 6898 > > STAT 3:total_pages 5 > > STAT 3:total_chunks 34490 > > STAT 3:used_chunks 34240 > > STAT 3:free_chunks 250 > > STAT 3:free_chunks_end 0 > > STAT 3:mem_requested 4800003 > > STAT 3:get_hits 2088979 > > STAT 3:cmd_set 4355223 > > STAT 3:delete_hits 3392 > > STAT 3:incr_hits 0 > > STAT 3:decr_hits 0 > > STAT 3:cas_hits 0 > > STAT 3:cas_badval 0 > > STAT 3:touch_hits 0 > > STAT 4:chunk_size 192 > > STAT 4:chunks_per_page 5461 > > STAT 4:total_pages 11 > > STAT 4:total_chunks 60071 > > STAT 4:used_chunks 60070 > > STAT 4:free_chunks 1 > > STAT 4:free_chunks_end 0 > > STAT 4:mem_requested 10821971 > > STAT 4:get_hits 65413752 > > STAT 4:cmd_set 22935889 > > STAT 4:delete_hits 6028 > > STAT 4:incr_hits 0 > > STAT 4:decr_hits 0 > > STAT 4:cas_hits 0 > > STAT 4:cas_badval 0 > > STAT 4:touch_hits 0 > > STAT 5:chunk_size 240 > > STAT 5:chunks_per_page 4369 > > STAT 5:total_pages 756 > > STAT 5:total_chunks 3302964 > > STAT 5:used_chunks 3302964 > > STAT 5:free_chunks 0 > > STAT 5:free_chunks_end 0 > > STAT 5:mem_requested 766866823 > > STAT 5:get_hits 2762768607 > > STAT 5:cmd_set 445418784 > > STAT 5:delete_hits 15806705 > > STAT 5:incr_hits 0 > > STAT 5:decr_hits 0 > > STAT 5:cas_hits 0 > > STAT 5:cas_badval 0 > > STAT 5:touch_hits 0 > > STAT 6:chunk_size 304 > > STAT 6:chunks_per_page 3449 > > STAT 6:total_pages 2304 > > STAT 6:total_chunks 7946496 > > STAT 6:used_chunks 7946496 > > STAT 6:free_chunks 0 > > STAT 6:free_chunks_end 0 > > STAT 6:mem_requested 2031962836 > > STAT 6:get_hits 8200882378 > > STAT 6:cmd_set 766512658 > > STAT 6:delete_hits 2583058 > > STAT 6:incr_hits 0 > > STAT 6:decr_hits 0 > > STAT 6:cas_hits 0 > > STAT 6:cas_badval 0 > > STAT 6:touch_hits 0 > > STAT 7:chunk_size 384 > > STAT 7:chunks_per_page 2730 > > STAT 7:total_pages 2889 > > STAT 7:total_chunks 7886970 > > STAT 7:used_chunks 7886970 > > STAT 7:free_chunks 0 > > STAT 7:free_chunks_end 0 > > STAT 7:mem_requested 2690869375 > > STAT 7:get_hits 7499037701 > > STAT 7:cmd_set 608169539 > > STAT 7:delete_hits 13689385 > > STAT 7:incr_hits 0 > > STAT 7:decr_hits 0 > > STAT 7:cas_hits 0 > > STAT 7:cas_badval 0 > > STAT 7:touch_hits 0 > > STAT 8:chunk_size 480 > > STAT 8:chunks_per_page 2184 > > STAT 8:total_pages 1212 > > STAT 8:total_chunks 2647008 > > STAT 8:used_chunks 2647007 > > STAT 8:free_chunks 1 > > STAT 8:free_chunks_end 0 > > STAT 8:mem_requested 1207287810 > > STAT 8:get_hits 3840682046 > > STAT 8:cmd_set 269966906 > > STAT 8:delete_hits 58383193 > > STAT 8:incr_hits 0 > > STAT 8:decr_hits 0 > > STAT 8:cas_hits 0 > > STAT 8:cas_badval 0 > > STAT 8:touch_hits 0 > > STAT 9:chunk_size 600 > > STAT 9:chunks_per_page 1747 > > STAT 9:total_pages 1186 > > STAT 9:total_chunks 2071942 > > STAT 9:used_chunks 2071940 > > STAT 9:free_chunks 2 > > STAT 9:free_chunks_end 0 > > STAT 9:mem_requested 1147470917 > > STAT 9:get_hits 13132583673 > > STAT 9:cmd_set 243148804 > > STAT 9:delete_hits 48047043 > > STAT 9:incr_hits 0 > > STAT 9:decr_hits 0 > > STAT 9:cas_hits 4014206 > > STAT 9:cas_badval 11269 > > STAT 9:touch_hits 0 > > STAT 10:chunk_size 752 > > STAT 10:chunks_per_page 1394 > > STAT 10:total_pages 594 > > STAT 10:total_chunks 828036 > > STAT 10:used_chunks 828034 > > STAT 10:free_chunks 2 > > STAT 10:free_chunks_end 0 > > STAT 10:mem_requested 564012816 > > STAT 10:get_hits 1933642946 > > STAT 10:cmd_set 112475431 > > STAT 10:delete_hits 49946608 > > STAT 10:incr_hits 0 > > STAT 10:decr_hits 0 > > STAT 10:cas_hits 29 > > STAT 10:cas_badval 0 > > STAT 10:touch_hits 0 > > STAT 11:chunk_size 944 > > STAT 11:chunks_per_page 1110 > > STAT 11:total_pages 1077 > > STAT 11:total_chunks 1195470 > > STAT 11:used_chunks 1195467 > > STAT 11:free_chunks 3 > > STAT 11:free_chunks_end 0 > > STAT 11:mem_requested 1018678728 > > STAT 11:get_hits 10188475487 > > STAT 11:cmd_set 224064010 > > STAT 11:delete_hits 108223108 > > STAT 11:incr_hits 0 > > STAT 11:decr_hits 0 > > STAT 11:cas_hits 0 > > STAT 11:cas_badval 0 > > STAT 11:touch_hits 0 > > STAT 12:chunk_size 1184 > > STAT 12:chunks_per_page 885 > > STAT 12:total_pages 1490 > > STAT 12:total_chunks 1318650 > > STAT 12:used_chunks 1318649 > > STAT 12:free_chunks 1 > > STAT 12:free_chunks_end 0 > > STAT 12:mem_requested 1330968030 > > STAT 12:get_hits 1072553156 > > STAT 12:cmd_set 99248484 > > STAT 12:delete_hits 46013332 > > STAT 12:incr_hits 0 > > STAT 12:decr_hits 0 > > STAT 12:cas_hits 0 > > STAT 12:cas_badval 0 > > STAT 12:touch_hits 0 > > STAT 13:chunk_size 1480 > > STAT 13:chunks_per_page 708 > > STAT 13:total_pages 387 > > STAT 13:total_chunks 273996 > > STAT 13:used_chunks 273996 > > STAT 13:free_chunks 0 > > STAT 13:free_chunks_end 0 > > STAT 13:mem_requested 354860947 > > STAT 13:get_hits 4377461297 > > STAT 13:cmd_set 461802286 > > STAT 13:delete_hits 227426652 > > STAT 13:incr_hits 0 > > STAT 13:decr_hits 0 > > STAT 13:cas_hits 0 > > STAT 13:cas_badval 0 > > STAT 13:touch_hits 0 > > STAT 14:chunk_size 1856 > > STAT 14:chunks_per_page 564 > > STAT 14:total_pages 125 > > STAT 14:total_chunks 70500 > > STAT 14:used_chunks 70496 > > STAT 14:free_chunks 4 > > STAT 14:free_chunks_end 0 > > STAT 14:mem_requested 115854604 > > STAT 14:get_hits 2022854542 > > STAT 14:cmd_set 31647636 > > STAT 14:delete_hits 8951400 > > STAT 14:incr_hits 0 > > STAT 14:decr_hits 0 > > STAT 14:cas_hits 0 > > STAT 14:cas_badval 0 > > STAT 14:touch_hits 0 > > STAT 15:chunk_size 2320 > > STAT 15:chunks_per_page 451 > > STAT 15:total_pages 10206 > > STAT 15:total_chunks 4602906 > > STAT 15:used_chunks 4602906 > > STAT 15:free_chunks 0 > > STAT 15:free_chunks_end 0 > > STAT 15:mem_requested 10366038157 > > STAT 15:get_hits 6073432864 > > STAT 15:cmd_set 605229702 > > STAT 15:delete_hits 221933803 > > STAT 15:incr_hits 0 > > STAT 15:decr_hits 0 > > STAT 15:cas_hits 0 > > STAT 15:cas_badval 0 > > STAT 15:touch_hits 0 > > STAT 16:chunk_size 2904 > > STAT 16:chunks_per_page 361 > > STAT 16:total_pages 670 > > STAT 16:total_chunks 241870 > > STAT 16:used_chunks 241870 > > STAT 16:free_chunks 0 > > STAT 16:free_chunks_end 0 > > STAT 16:mem_requested 584915184 > > STAT 16:get_hits 1792921599 > > STAT 16:cmd_set 37813060 > > STAT 16:delete_hits 10276685 > > STAT 16:incr_hits 0 > > STAT 16:decr_hits 0 > > STAT 16:cas_hits 0 > > STAT 16:cas_badval 0 > > STAT 16:touch_hits 0 > > STAT 17:chunk_size 3632 > > STAT 17:chunks_per_page 288 > > STAT 17:total_pages 56 > > STAT 17:total_chunks 16128 > > STAT 17:used_chunks 16127 > > STAT 17:free_chunks 1 > > STAT 17:free_chunks_end 0 > > STAT 17:mem_requested 48404622 > > STAT 17:get_hits 11635110663 > > STAT 17:cmd_set 332089193 > > STAT 17:delete_hits 69781587 > > STAT 17:incr_hits 0 > > STAT 17:decr_hits 0 > > STAT 17:cas_hits 0 > > STAT 17:cas_badval 0 > > STAT 17:touch_hits 0 > > STAT 18:chunk_size 4544 > > STAT 18:chunks_per_page 230 > > STAT 18:total_pages 59 > > STAT 18:total_chunks 13570 > > STAT 18:used_chunks 13377 > > STAT 18:free_chunks 193 > > STAT 18:free_chunks_end 0 > > STAT 18:mem_requested 54587001 > > STAT 18:get_hits 6541745634 > > STAT 18:cmd_set 6239313 > > STAT 18:delete_hits 1058115 > > STAT 18:incr_hits 0 > > STAT 18:decr_hits 0 > > STAT 18:cas_hits 0 > > STAT 18:cas_badval 0 > > STAT 18:touch_hits 0 > > STAT 19:chunk_size 5680 > > STAT 19:chunks_per_page 184 > > STAT 19:total_pages 59 > > STAT 19:total_chunks 10856 > > STAT 19:used_chunks 10853 > > STAT 19:free_chunks 3 > > STAT 19:free_chunks_end 0 > > STAT 19:mem_requested 54711323 > > STAT 19:get_hits 2549599632 > > STAT 19:cmd_set 4404829 > > STAT 19:delete_hits 674692 > > STAT 19:incr_hits 0 > > STAT 19:decr_hits 0 > > STAT 19:cas_hits 0 > > STAT 19:cas_badval 0 > > STAT 19:touch_hits 0 > > STAT 20:chunk_size 7104 > > STAT 20:chunks_per_page 147 > > STAT 20:total_pages 47 > > STAT 20:total_chunks 6909 > > STAT 20:used_chunks 6908 > > STAT 20:free_chunks 1 > > STAT 20:free_chunks_end 0 > > STAT 20:mem_requested 43685252 > > STAT 20:get_hits 536509528 > > STAT 20:cmd_set 3652653 > > STAT 20:delete_hits 408748 > > STAT 20:incr_hits 0 > > STAT 20:decr_hits 0 > > STAT 20:cas_hits 0 > > STAT 20:cas_badval 0 > > STAT 20:touch_hits 0 > > STAT 21:chunk_size 8880 > > STAT 21:chunks_per_page 118 > > STAT 21:total_pages 49 > > STAT 21:total_chunks 5782 > > STAT 21:used_chunks 5679 > > STAT 21:free_chunks 103 > > STAT 21:free_chunks_end 0 > > STAT 21:mem_requested 44988474 > > STAT 21:get_hits 30970925 > > STAT 21:cmd_set 3488884 > > STAT 21:delete_hits 312035 > > STAT 21:incr_hits 0 > > STAT 21:decr_hits 0 > > STAT 21:cas_hits 0 > > STAT 21:cas_badval 0 > > STAT 21:touch_hits 0 > > STAT 22:chunk_size 11104 > > STAT 22:chunks_per_page 94 > > STAT 22:total_pages 53 > > STAT 22:total_chunks 4982 > > STAT 22:used_chunks 4979 > > STAT 22:free_chunks 3 > > STAT 22:free_chunks_end 0 > > STAT 22:mem_requested 49385581 > > STAT 22:get_hits 28664684 > > STAT 22:cmd_set 3408366 > > STAT 22:delete_hits 267636 > > STAT 22:incr_hits 0 > > STAT 22:decr_hits 0 > > STAT 22:cas_hits 0 > > STAT 22:cas_badval 0 > > STAT 22:touch_hits 0 > > STAT 23:chunk_size 13880 > > STAT 23:chunks_per_page 75 > > STAT 23:total_pages 53 > > STAT 23:total_chunks 3975 > > STAT 23:used_chunks 3973 > > STAT 23:free_chunks 2 > > STAT 23:free_chunks_end 0 > > STAT 23:mem_requested 49578512 > > STAT 23:get_hits 195769483 > > STAT 23:cmd_set 3571084 > > STAT 23:delete_hits 329032 > > STAT 23:incr_hits 0 > > STAT 23:decr_hits 0 > > STAT 23:cas_hits 0 > > STAT 23:cas_badval 0 > > STAT 23:touch_hits 0 > > STAT 24:chunk_size 17352 > > STAT 24:chunks_per_page 60 > > STAT 24:total_pages 55 > > STAT 24:total_chunks 3300 > > STAT 24:used_chunks 3299 > > STAT 24:free_chunks 1 > > STAT 24:free_chunks_end 0 > > STAT 24:mem_requested 51647575 > > STAT 24:get_hits 23675199 > > STAT 24:cmd_set 3446711 > > STAT 24:delete_hits 193657 > > STAT 24:incr_hits 0 > > STAT 24:decr_hits 0 > > STAT 24:cas_hits 0 > > STAT 24:cas_badval 0 > > STAT 24:touch_hits 0 > > STAT 25:chunk_size 21696 > > STAT 25:chunks_per_page 48 > > STAT 25:total_pages 56 > > STAT 25:total_chunks 2688 > > STAT 25:used_chunks 2688 > > STAT 25:free_chunks 0 > > STAT 25:free_chunks_end 0 > > STAT 25:mem_requested 52302706 > > STAT 25:get_hits 52421013 > > STAT 25:cmd_set 3537309 > > STAT 25:delete_hits 174082 > > STAT 25:incr_hits 0 > > STAT 25:decr_hits 0 > > STAT 25:cas_hits 0 > > STAT 25:cas_badval 0 > > STAT 25:touch_hits 0 > > STAT 26:chunk_size 27120 > > STAT 26:chunks_per_page 38 > > STAT 26:total_pages 62 > > STAT 26:total_chunks 2356 > > STAT 26:used_chunks 2350 > > STAT 26:free_chunks 6 > > STAT 26:free_chunks_end 0 > > STAT 26:mem_requested 56782144 > > STAT 26:get_hits 18822879 > > STAT 26:cmd_set 3698018 > > STAT 26:delete_hits 264739 > > STAT 26:incr_hits 0 > > STAT 26:decr_hits 0 > > STAT 26:cas_hits 0 > > STAT 26:cas_badval 0 > > STAT 26:touch_hits 0 > > STAT 27:chunk_size 33904 > > STAT 27:chunks_per_page 30 > > STAT 27:total_pages 56 > > STAT 27:total_chunks 1680 > > STAT 27:used_chunks 1679 > > STAT 27:free_chunks 1 > > STAT 27:free_chunks_end 0 > > STAT 27:mem_requested 50521865 > > STAT 27:get_hits 23321697 > > STAT 27:cmd_set 3355508 > > STAT 27:delete_hits 85179 > > STAT 27:incr_hits 0 > > STAT 27:decr_hits 0 > > STAT 27:cas_hits 0 > > STAT 27:cas_badval 0 > > STAT 27:touch_hits 0 > > STAT 28:chunk_size 42384 > > STAT 28:chunks_per_page 24 > > STAT 28:total_pages 55 > > STAT 28:total_chunks 1320 > > STAT 28:used_chunks 1243 > > STAT 28:free_chunks 77 > > STAT 28:free_chunks_end 0 > > STAT 28:mem_requested 47053818 > > STAT 28:get_hits 21281440 > > STAT 28:cmd_set 3164451 > > STAT 28:delete_hits 78640 > > STAT 28:incr_hits 0 > > STAT 28:decr_hits 0 > > STAT 28:cas_hits 0 > > STAT 28:cas_badval 0 > > STAT 28:touch_hits 0 > > STAT 29:chunk_size 52984 > > STAT 29:chunks_per_page 19 > > STAT 29:total_pages 66 > > STAT 29:total_chunks 1254 > > STAT 29:used_chunks 1252 > > STAT 29:free_chunks 2 > > STAT 29:free_chunks_end 0 > > STAT 29:mem_requested 58819531 > > STAT 29:get_hits 42795540 > > STAT 29:cmd_set 2912143 > > STAT 29:delete_hits 65604 > > STAT 29:incr_hits 0 > > STAT 29:decr_hits 0 > > STAT 29:cas_hits 0 > > STAT 29:cas_badval 0 > > STAT 29:touch_hits 0 > > STAT 30:chunk_size 66232 > > STAT 30:chunks_per_page 15 > > STAT 30:total_pages 61 > > STAT 30:total_chunks 915 > > STAT 30:used_chunks 912 > > STAT 30:free_chunks 3 > > STAT 30:free_chunks_end 0 > > STAT 30:mem_requested 53983942 > > STAT 30:get_hits 26730121 > > STAT 30:cmd_set 2677426 > > STAT 30:delete_hits 64344 > > STAT 30:incr_hits 0 > > STAT 30:decr_hits 0 > > STAT 30:cas_hits 0 > > STAT 30:cas_badval 0 > > STAT 30:touch_hits 0 > > STAT 31:chunk_size 82792 > > STAT 31:chunks_per_page 12 > > STAT 31:total_pages 66 > > STAT 31:total_chunks 792 > > STAT 31:used_chunks 549 > > STAT 31:free_chunks 243 > > STAT 31:free_chunks_end 0 > > STAT 31:mem_requested 40660882 > > STAT 31:get_hits 15272618 > > STAT 31:cmd_set 2498978 > > STAT 31:delete_hits 48565 > > STAT 31:incr_hits 0 > > STAT 31:decr_hits 0 > > STAT 31:cas_hits 0 > > STAT 31:cas_badval 0 > > STAT 31:touch_hits 0 > > STAT 32:chunk_size 103496 > > STAT 32:chunks_per_page 10 > > STAT 32:total_pages 87 > > STAT 32:total_chunks 870 > > STAT 32:used_chunks 541 > > STAT 32:free_chunks 329 > > STAT 32:free_chunks_end 0 > > STAT 32:mem_requested 50263471 > > STAT 32:get_hits 20072357 > > STAT 32:cmd_set 2499953 > > STAT 32:delete_hits 61782 > > STAT 32:incr_hits 0 > > STAT 32:decr_hits 0 > > STAT 32:cas_hits 0 > > STAT 32:cas_badval 0 > > STAT 32:touch_hits 0 > > STAT 33:chunk_size 129376 > > STAT 33:chunks_per_page 8 > > STAT 33:total_pages 53 > > STAT 33:total_chunks 424 > > STAT 33:used_chunks 424 > > STAT 33:free_chunks 0 > > STAT 33:free_chunks_end 0 > > STAT 33:mem_requested 47641769 > > STAT 33:get_hits 59881119 > > STAT 33:cmd_set 2295537 > > STAT 33:delete_hits 22440 > > STAT 33:incr_hits 0 > > STAT 33:decr_hits 0 > > STAT 33:cas_hits 0 > > STAT 33:cas_badval 0 > > STAT 33:touch_hits 0 > > STAT 34:chunk_size 161720 > > STAT 34:chunks_per_page 6 > > STAT 34:total_pages 65 > > STAT 34:total_chunks 390 > > STAT 34:used_chunks 376 > > STAT 34:free_chunks 14 > > STAT 34:free_chunks_end 0 > > STAT 34:mem_requested 54966686 > > STAT 34:get_hits 92653779 > > STAT 34:cmd_set 2297068 > > STAT 34:delete_hits 15615 > > STAT 34:incr_hits 0 > > STAT 34:decr_hits 0 > > STAT 34:cas_hits 0 > > STAT 34:cas_badval 0 > > STAT 34:touch_hits 0 > > STAT 35:chunk_size 202152 > > STAT 35:chunks_per_page 5 > > STAT 35:total_pages 73 > > STAT 35:total_chunks 365 > > STAT 35:used_chunks 364 > > STAT 35:free_chunks 1 > > STAT 35:free_chunks_end 0 > > STAT 35:mem_requested 65557022 > > STAT 35:get_hits 17003409 > > STAT 35:cmd_set 2461151 > > STAT 35:delete_hits 11487 > > STAT 35:incr_hits 0 > > STAT 35:decr_hits 0 > > STAT 35:cas_hits 0 > > STAT 35:cas_badval 0 > > STAT 35:touch_hits 0 > > STAT 36:chunk_size 252696 > > STAT 36:chunks_per_page 4 > > STAT 36:total_pages 69 > > STAT 36:total_chunks 276 > > STAT 36:used_chunks 275 > > STAT 36:free_chunks 1 > > STAT 36:free_chunks_end 0 > > STAT 36:mem_requested 61781922 > > STAT 36:get_hits 9859716 > > STAT 36:cmd_set 2585201 > > STAT 36:delete_hits 7886 > > STAT 36:incr_hits 0 > > STAT 36:decr_hits 0 > > STAT 36:cas_hits 0 > > STAT 36:cas_badval 0 > > STAT 36:touch_hits 0 > > STAT 37:chunk_size 315872 > > STAT 37:chunks_per_page 3 > > STAT 37:total_pages 62 > > STAT 37:total_chunks 186 > > STAT 37:used_chunks 185 > > STAT 37:free_chunks 1 > > STAT 37:free_chunks_end 0 > > STAT 37:mem_requested 52014786 > > STAT 37:get_hits 19023342 > > STAT 37:cmd_set 2800717 > > STAT 37:delete_hits 9511 > > STAT 37:incr_hits 0 > > STAT 37:decr_hits 0 > > STAT 37:cas_hits 0 > > STAT 37:cas_badval 0 > > STAT 37:touch_hits 0 > > STAT 38:chunk_size 394840 > > STAT 38:chunks_per_page 2 > > STAT 38:total_pages 78 > > STAT 38:total_chunks 156 > > STAT 38:used_chunks 146 > > STAT 38:free_chunks 10 > > STAT 38:free_chunks_end 0 > > STAT 38:mem_requested 50928175 > > STAT 38:get_hits 8349949 > > STAT 38:cmd_set 3183214 > > STAT 38:delete_hits 5892 > > STAT 38:incr_hits 0 > > STAT 38:decr_hits 0 > > STAT 38:cas_hits 0 > > STAT 38:cas_badval 0 > > STAT 38:touch_hits 0 > > STAT 39:chunk_size 493552 > > STAT 39:chunks_per_page 2 > > STAT 39:total_pages 73 > > STAT 39:total_chunks 146 > > STAT 39:used_chunks 145 > > STAT 39:free_chunks 1 > > STAT 39:free_chunks_end 0 > > STAT 39:mem_requested 63336441 > > STAT 39:get_hits 12247799 > > STAT 39:cmd_set 3670728 > > STAT 39:delete_hits 8660 > > STAT 39:incr_hits 0 > > STAT 39:decr_hits 0 > > STAT 39:cas_hits 0 > > STAT 39:cas_badval 0 > > STAT 39:touch_hits 0 > > STAT 40:chunk_size 616944 > > STAT 40:chunks_per_page 1 > > STAT 40:total_pages 92 > > STAT 40:total_chunks 92 > > STAT 40:used_chunks 90 > > STAT 40:free_chunks 2 > > STAT 40:free_chunks_end 0 > > STAT 40:mem_requested 48619482 > > STAT 40:get_hits 10464471 > > STAT 40:cmd_set 4291497 > > STAT 40:delete_hits 3919 > > STAT 40:incr_hits 0 > > STAT 40:decr_hits 0 > > STAT 40:cas_hits 0 > > STAT 40:cas_badval 0 > > STAT 40:touch_hits 0 > > STAT 41:chunk_size 771184 > > STAT 41:chunks_per_page 1 > > STAT 41:total_pages 67 > > STAT 41:total_chunks 67 > > STAT 41:used_chunks 64 > > STAT 41:free_chunks 3 > > STAT 41:free_chunks_end 0 > > STAT 41:mem_requested 44676493 > > STAT 41:get_hits 11868280 > > STAT 41:cmd_set 5114144 > > STAT 41:delete_hits 2942 > > STAT 41:incr_hits 0 > > STAT 41:decr_hits 0 > > STAT 41:cas_hits 0 > > STAT 41:cas_badval 0 > > STAT 41:touch_hits 0 > > STAT 42:chunk_size 1048576 > > STAT 42:chunks_per_page 1 > > STAT 42:total_pages 90 > > STAT 42:total_chunks 90 > > STAT 42:used_chunks 87 > > STAT 42:free_chunks 3 > > STAT 42:free_chunks_end 0 > > STAT 42:mem_requested 78399773 > > STAT 42:get_hits 28459379 > > STAT 42:cmd_set 8948109 > > STAT 42:delete_hits 4045 > > STAT 42:incr_hits 0 > > STAT 42:decr_hits 0 > > STAT 42:cas_hits 0 > > STAT 42:cas_badval 0 > > STAT 42:touch_hits 0 > > STAT active_slabs 42 > > STAT total_malloced 25769803776 > > END > > > > > > Items: > > > > stats items > > STAT items:2:number 13 > > STAT items:2:age 142 > > STAT items:2:evicted 115 > > STAT items:2:evicted_nonzero 115 > > STAT items:2:evicted_time 21704 > > STAT items:2:outofmemory 0 > > STAT items:2:tailrepairs 0 > > STAT items:2:reclaimed 1216206 > > STAT items:2:expired_unfetched 1071602 > > STAT items:2:evicted_unfetched 39 > > STAT items:3:number 34239 > > STAT items:3:age 15264 > > STAT items:3:evicted 46 > > STAT items:3:evicted_nonzero 46 > > STAT items:3:evicted_time 13071 > > STAT items:3:outofmemory 0 > > STAT items:3:tailrepairs 0 > > STAT items:3:reclaimed 4246565 > > STAT items:3:expired_unfetched 4243613 > > STAT items:3:evicted_unfetched 10 > > STAT items:4:number 60070 > > STAT items:4:age 15297 > > STAT items:4:evicted 3056364 > > STAT items:4:evicted_nonzero 3056364 > > STAT items:4:evicted_time 15297 > > STAT items:4:outofmemory 0 > > STAT items:4:tailrepairs 0 > > STAT items:4:reclaimed 826482 > > STAT items:4:expired_unfetched 825837 > > STAT items:4:evicted_unfetched 3005491 > > STAT items:5:number 3302964 > > STAT items:5:age 16967 > > STAT items:5:evicted 264916205 > > STAT items:5:evicted_nonzero 259964350 > > STAT items:5:evicted_time 16967 > > STAT items:5:outofmemory 0 > > STAT items:5:tailrepairs 0 > > STAT items:5:reclaimed 94853 > > STAT items:5:expired_unfetched 74504 > > STAT items:5:evicted_unfetched 158459283 > > STAT items:6:number 7946495 > > STAT items:6:age 17389 > > STAT items:6:evicted 530949743 > > STAT items:6:evicted_nonzero 518236092 > > STAT items:6:evicted_time 17389 > > STAT items:6:outofmemory 0 > > STAT items:6:tailrepairs 0 > > STAT items:6:reclaimed 214094 > > STAT items:6:expired_unfetched 103531 > > STAT items:6:evicted_unfetched 226791315 > > STAT items:7:number 7886970 > > STAT items:7:age 19458 > > STAT items:7:evicted 520515654 > > STAT items:7:evicted_nonzero 518625834 > > STAT items:7:evicted_time 19458 > > STAT items:7:outofmemory 0 > > STAT items:7:tailrepairs 0 > > STAT items:7:reclaimed 193574 > > STAT items:7:expired_unfetched 90290 > > STAT items:7:evicted_unfetched 229790958 > > STAT items:8:number 2647000 > > STAT items:8:age 34693 > > STAT items:8:evicted 141229400 > > STAT items:8:evicted_nonzero 141228604 > > STAT items:8:evicted_time 34693 > > STAT items:8:outofmemory 0 > > STAT items:8:tailrepairs 0 > > STAT items:8:reclaimed 54331 > > STAT items:8:expired_unfetched 28142 > > STAT items:8:evicted_unfetched 84251291 > > STAT items:9:number 2071942 > > STAT items:9:age 21735 > > STAT items:9:evicted 134744880 > > STAT items:9:evicted_nonzero 134744372 > > STAT items:9:evicted_time 21735 > > STAT items:9:outofmemory 0 > > STAT items:9:tailrepairs 0 > > STAT items:9:reclaimed 32191 > > STAT items:9:expired_unfetched 11257 > > STAT items:9:evicted_unfetched 93361123 > > STAT items:10:number 828035 > > STAT items:10:age 26547 > > STAT items:10:evicted 36032782 > > STAT items:10:evicted_nonzero 36032525 > > STAT items:10:evicted_time 26547 > > STAT items:10:outofmemory 0 > > STAT items:10:tailrepairs 0 > > STAT items:10:reclaimed 22587 > > STAT items:10:expired_unfetched 7195 > > STAT items:10:evicted_unfetched 23547610 > > STAT items:11:number 1195460 > > STAT items:11:age 18676 > > STAT items:11:evicted 73376576 > > STAT items:11:evicted_nonzero 73376430 > > STAT items:11:evicted_time 18674 > > STAT items:11:outofmemory 0 > > STAT items:11:tailrepairs 0 > > STAT items:11:reclaimed 63137 > > STAT items:11:expired_unfetched 17436 > > STAT items:11:evicted_unfetched 32825253 > > STAT items:12:number 1318646 > > STAT items:12:age 114056 > > STAT items:12:evicted 9903968 > > STAT items:12:evicted_nonzero 9903912 > > STAT items:12:evicted_time 114056 > > STAT items:12:outofmemory 0 > > STAT items:12:tailrepairs 0 > > STAT items:12:reclaimed 405700 > > STAT items:12:expired_unfetched 25828 > > STAT items:12:evicted_unfetched 5526986 > > STAT items:13:number 273996 > > STAT items:13:age 1195 > > STAT items:13:evicted 197762250 > > STAT items:13:evicted_nonzero 197762162 > > STAT items:13:evicted_time 1195 > > STAT items:13:outofmemory 0 > > STAT items:13:tailrepairs 0 > > STAT items:13:reclaimed 11932 > > STAT items:13:expired_unfetched 3789 > > STAT items:13:evicted_unfetched 101326795 > > STAT items:14:number 70449 > > STAT items:14:age 9982 > > STAT items:14:evicted 5039191 > > STAT items:14:evicted_nonzero 5039120 > > STAT items:14:evicted_time 9954 > > STAT items:14:outofmemory 0 > > STAT items:14:tailrepairs 0 > > STAT items:14:reclaimed 4396 > > STAT items:14:expired_unfetched 761 > > STAT items:14:evicted_unfetched 3433298 > > STAT items:15:number 4602905 > > STAT items:15:age 15795 > > STAT items:15:evicted 342545517 > > STAT items:15:evicted_nonzero 342545492 > > STAT items:15:evicted_time 15795 > > STAT items:15:outofmemory 0 > > STAT items:15:tailrepairs 0 > > STAT items:15:reclaimed 138296 > > STAT items:15:expired_unfetched 70412 > > STAT items:15:evicted_unfetched 127596123 > > STAT items:16:number 241791 > > STAT items:16:age 19808 > > STAT items:16:evicted 8454537 > > STAT items:16:evicted_nonzero 8454504 > > STAT items:16:evicted_time 19797 > > STAT items:16:outofmemory 0 > > STAT items:16:tailrepairs 0 > > STAT items:16:reclaimed 13733 > > STAT items:16:expired_unfetched 7505 > > STAT items:16:evicted_unfetched 4798153 > > STAT items:17:number 16128 > > STAT items:17:age 97 > > STAT items:17:evicted 248266258 > > STAT items:17:evicted_nonzero 248266235 > > STAT items:17:evicted_time 97 > > STAT items:17:outofmemory 33098 > > STAT items:17:tailrepairs 0 > > STAT items:17:reclaimed 1188 > > STAT items:17:expired_unfetched 196 > > STAT items:17:evicted_unfetched 144563461 > > STAT items:18:number 13350 > > STAT items:18:age 18096 > > STAT items:18:evicted 1071797 > > STAT items:18:evicted_nonzero 1071789 > > STAT items:18:evicted_time 17994 > > STAT items:18:outofmemory 0 > > STAT items:18:tailrepairs 0 > > STAT items:18:reclaimed 4823 > > STAT items:18:expired_unfetched 3971 > > STAT items:18:evicted_unfetched 885304 > > STAT items:19:number 10856 > > STAT items:19:age 14092 > > STAT items:19:evicted 889367 > > STAT items:19:evicted_nonzero 889360 > > STAT items:19:evicted_time 14092 > > STAT items:19:outofmemory 0 > > STAT items:19:tailrepairs 0 > > STAT items:19:reclaimed 11714 > > STAT items:19:expired_unfetched 10662 > > STAT items:19:evicted_unfetched 725843 > > STAT items:20:number 6885 > > STAT items:20:age 12986 > > STAT items:20:evicted 665775 > > STAT items:20:evicted_nonzero 665756 > > STAT items:20:evicted_time 12964 > > STAT items:20:outofmemory 0 > > STAT items:20:tailrepairs 0 > > STAT items:20:reclaimed 1411 > > STAT items:20:expired_unfetched 223 > > STAT items:20:evicted_unfetched 594803 > > STAT items:21:number 5706 > > STAT items:21:age 13977 > > STAT items:21:evicted 546802 > > STAT items:21:evicted_nonzero 546744 > > STAT items:21:evicted_time 13739 > > STAT items:21:outofmemory 0 > > STAT items:21:tailrepairs 0 > > STAT items:21:reclaimed 2895 > > STAT items:21:expired_unfetched 2540 > > STAT items:21:evicted_unfetched 499599 > > STAT items:22:number 4981 > > STAT items:22:age 14068 > > STAT items:22:evicted 453461 > > STAT items:22:evicted_nonzero 453455 > > STAT items:22:evicted_time 14065 > > STAT items:22:outofmemory 0 > > STAT items:22:tailrepairs 0 > > STAT items:22:reclaimed 6551 > > STAT items:22:expired_unfetched 6333 > > STAT items:22:evicted_unfetched 418385 > > STAT items:23:number 3972 > > STAT items:23:age 12160 > > STAT items:23:evicted 375968 > > STAT items:23:evicted_nonzero 375964 > > STAT items:23:evicted_time 12152 > > STAT items:23:outofmemory 0 > > STAT items:23:tailrepairs 0 > > STAT items:23:reclaimed 5417 > > STAT items:23:expired_unfetched 5314 > > STAT items:23:evicted_unfetched 346875 > > STAT items:24:number 3290 > > STAT items:24:age 11832 > > STAT items:24:evicted 322194 > > STAT items:24:evicted_nonzero 322192 > > STAT items:24:evicted_time 11783 > > STAT items:24:outofmemory 0 > > STAT items:24:tailrepairs 0 > > STAT items:24:reclaimed 2502 > > STAT items:24:expired_unfetched 2406 > > STAT items:24:evicted_unfetched 298088 > > STAT items:25:number 2687 > > STAT items:25:age 10155 > > STAT items:25:evicted 276493 > > STAT items:25:evicted_nonzero 276493 > > STAT items:25:evicted_time 10152 > > STAT items:25:outofmemory 0 > > STAT items:25:tailrepairs 0 > > STAT items:25:reclaimed 5471 > > STAT items:25:expired_unfetched 5342 > > STAT items:25:evicted_unfetched 254356 > > STAT items:26:number 2356 > > STAT items:26:age 11052 > > STAT items:26:evicted 219706 > > STAT items:26:evicted_nonzero 219706 > > STAT items:26:evicted_time 11057 > > STAT items:26:outofmemory 0 > > STAT items:26:tailrepairs 0 > > STAT items:26:reclaimed 6487 > > STAT items:26:expired_unfetched 6456 > > STAT items:26:evicted_unfetched 202516 > > STAT items:27:number 1674 > > STAT items:27:age 8823 > > STAT items:27:evicted 183982 > > STAT items:27:evicted_nonzero 183980 > > STAT items:27:evicted_time 8584 > > STAT items:27:outofmemory 0 > > STAT items:27:tailrepairs 0 > > STAT items:27:reclaimed 4722 > > STAT items:27:expired_unfetched 4674 > > STAT items:27:evicted_unfetched 168879 > > STAT items:28:number 1240 > > STAT items:28:age 6423 > > STAT items:28:evicted 150133 > > STAT items:28:evicted_nonzero 150133 > > STAT items:28:evicted_time 5706 > > STAT items:28:outofmemory 0 > > STAT items:28:tailrepairs 0 > > STAT items:28:reclaimed 5675 > > STAT items:28:expired_unfetched 5649 > > STAT items:28:evicted_unfetched 136667 > > STAT items:29:number 1253 > > STAT items:29:age 8778 > > STAT items:29:evicted 113070 > > STAT items:29:evicted_nonzero 113070 > > STAT items:29:evicted_time 8779 > > STAT items:29:outofmemory 0 > > STAT items:29:tailrepairs 0 > > STAT items:29:reclaimed 5288 > > STAT items:29:expired_unfetched 5276 > > STAT items:29:evicted_unfetched 105238 > > STAT items:30:number 914 > > STAT items:30:age 7757 > > STAT items:30:evicted 93514 > > STAT items:30:evicted_nonzero 93514 > > STAT items:30:evicted_time 7753 > > STAT items:30:outofmemory 0 > > STAT items:30:tailrepairs 0 > > STAT items:30:reclaimed 4483 > > STAT items:30:expired_unfetched 4475 > > STAT items:30:evicted_unfetched 86632 > > STAT items:31:number 547 > > STAT items:31:age 11174 > > STAT items:31:evicted 73771 > > STAT items:31:evicted_nonzero 73763 > > STAT items:31:evicted_time 2077 > > STAT items:31:outofmemory 0 > > STAT items:31:tailrepairs 0 > > STAT items:31:reclaimed 4357 > > STAT items:31:expired_unfetched 4348 > > STAT items:31:evicted_unfetched 69276 > > STAT items:32:number 532 > > STAT items:32:age 6207 > > STAT items:32:evicted 55150 > > STAT items:32:evicted_nonzero 55150 > > STAT items:32:evicted_time 1757 > > STAT items:32:outofmemory 0 > > STAT items:32:tailrepairs 0 > > STAT items:32:reclaimed 12503 > > STAT items:32:expired_unfetched 12500 > > STAT items:32:evicted_unfetched 52393 > > STAT items:33:number 423 > > STAT items:33:age 162 > > STAT items:33:evicted 48555 > > STAT items:33:evicted_nonzero 48555 > > STAT items:33:evicted_time 153 > > STAT items:33:outofmemory 0 > > STAT items:33:tailrepairs 0 > > STAT items:33:reclaimed 467 > > STAT items:33:expired_unfetched 460 > > STAT items:33:evicted_unfetched 45402 > > STAT items:34:number 375 > > STAT items:34:age 3825 > > STAT items:34:evicted 35449 > > STAT items:34:evicted_nonzero 35442 > > STAT items:34:evicted_time 2668 > > STAT items:34:outofmemory 0 > > STAT items:34:tailrepairs 0 > > STAT items:34:reclaimed 266 > > STAT items:34:expired_unfetched 266 > > STAT items:34:evicted_unfetched 34115 > > STAT items:35:number 363 > > STAT items:35:age 4599 > > STAT items:35:evicted 29243 > > STAT items:35:evicted_nonzero 29243 > > STAT items:35:evicted_time 4578 > > STAT items:35:outofmemory 0 > > STAT items:35:tailrepairs 0 > > STAT items:35:reclaimed 210 > > STAT items:35:expired_unfetched 208 > > STAT items:35:evicted_unfetched 27952 > > STAT items:36:number 275 > > STAT items:36:age 7506 > > STAT items:36:evicted 30173 > > STAT items:36:evicted_nonzero 30173 > > STAT items:36:evicted_time 7562 > > STAT items:36:outofmemory 0 > > STAT items:36:tailrepairs 0 > > STAT items:36:reclaimed 198 > > STAT items:36:expired_unfetched 196 > > STAT items:36:evicted_unfetched 29241 > > STAT items:37:number 185 > > STAT items:37:age 10176 > > STAT items:37:evicted 19809 > > STAT items:37:evicted_nonzero 19809 > > STAT items:37:evicted_time 10265 > > STAT items:37:outofmemory 0 > > STAT items:37:tailrepairs 0 > > STAT items:37:reclaimed 77 > > STAT items:37:expired_unfetched 77 > > STAT items:37:evicted_unfetched 19093 > > STAT items:38:number 146 > > STAT items:38:age 5250 > > STAT items:38:evicted 17170 > > STAT items:38:evicted_nonzero 17170 > > STAT items:38:evicted_time 5000 > > STAT items:38:outofmemory 0 > > STAT items:38:tailrepairs 0 > > STAT items:38:reclaimed 110 > > STAT items:38:expired_unfetched 109 > > STAT items:38:evicted_unfetched 16761 > > STAT items:39:number 144 > > STAT items:39:age 7665 > > STAT items:39:evicted 13960 > > STAT items:39:evicted_nonzero 13960 > > STAT items:39:evicted_time 8831 > > STAT items:39:outofmemory 0 > > STAT items:39:tailrepairs 0 > > STAT items:39:reclaimed 215 > > STAT items:39:expired_unfetched 214 > > STAT items:39:evicted_unfetched 13593 > > STAT items:40:number 89 > > STAT items:40:age 1225 > > STAT items:40:evicted 10413 > > STAT items:40:evicted_nonzero 10413 > > STAT items:40:evicted_time 1243 > > STAT items:40:outofmemory 0 > > STAT items:40:tailrepairs 0 > > STAT items:40:reclaimed 3 > > STAT items:40:expired_unfetched 3 > > STAT items:40:evicted_unfetched 10076 > > STAT items:41:number 64 > > STAT items:41:age 1805 > > STAT items:41:evicted 8375 > > STAT items:41:evicted_nonzero 8375 > > STAT items:41:evicted_time 2016 > > STAT items:41:outofmemory 0 > > STAT items:41:tailrepairs 0 > > STAT items:41:reclaimed 0 > > STAT items:41:expired_unfetched 0 > > STAT items:41:evicted_unfetched 8144 > > STAT items:42:number 85 > > STAT items:42:age 13461 > > STAT items:42:evicted 9221 > > STAT items:42:evicted_nonzero 9221 > > STAT items:42:evicted_time 13650 > > STAT items:42:outofmemory 0 > > STAT items:42:tailrepairs 0 > > STAT items:42:reclaimed 1 > > STAT items:42:expired_unfetched 1 > > STAT items:42:evicted_unfetched 9014 > > END > > > > > > > > > > > > > > On Tuesday, May 6, 2014 4:32:23 PM UTC-7, Dormando wrote: > > > > > Hi, > > > Does anybody know good way to handle OOM during set > operation? Server is fully calcified :) (no new pages to allocate) and i > have > > this issue for > > > slab 17 > > > STAT items:17:number 16128 > > > STAT items:17:age 90 > > > STAT items:17:evicted 246790897 > > > STAT items:17:evicted_nonzero 246790874 > > > STAT items:17:evicted_time 90 > > > STAT items:17:outofmemory 33098 > > > STAT items:17:tailrepairs 0 > > > STAT items:17:reclaimed 1183 > > > STAT items:17:expired_unfetched 196 > > > STAT items:17:evicted_unfetched 143699820 > > > > > > running memcached : STAT version 1.4.15 > > > > "stats slabs" ? Is memory unbalanced from other slabs? > > > > > nothing except reboot periodically comes to my mind but this > solution does not make me happy :) > > > > There's the slab rebalance feature. OOM errors only happen when > there is > > truly very few pages free and all of the ones in the tail are > locked, or > > there's a bug. It should always evict. The rebalance feature is > documented > > in doc/protocol.txt. > > > > However your eviction seems to be very highly pressured. The > > evicted_unfetched stat is high compared to the tota number of > evictions. > > So they're not even staying in long enough to get fetched > again. There > > aren't that many OOM errors overall, so perhaps you are just > hitting that > > slab way too hard and occasionally locking everything in the > tail. > > > > You either need to add more memory to the total system or > rebalance them. > > > > > other option - enable compression to allow more items but > need to experiment (why memcached does not provide server side > compression? > > as i see in > > > stats memcached cpu is not used, so would be good to utilize > it.) > > > > Very high rate of access is expected and the ratio of clients > to servers > > might be high, so compression is done in the client instead. It > was also > > designed to let you run it wherever there's free memory (extra > installed > > in webservers/etc) so it wants to avoid excess cpu usage. > > > > It's a trivial switch either way. > > > > Also consider upgrading to .17 or .19. might be some good fixes. > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
