What version are they running now? That stat output looks pretty sparse. Unfortunately when it comes to elasticache the answer is probably to just beg them to upgrade to a newer version. they tend to run really old and newer verisons do a lot better at balancing memory.
This is also suspect: > STAT 62:chunk_size 712 > STAT 62:total_chunks 4416 > STAT 62:free_chunks 631 > STAT 62:free_chunks_end 0 > STAT 63:chunk_size 524288 anything you store that's larger than 712 bytes will use half a megabyte of memory. If you're setting the slab multiplier (-f) to something really aggressive, you should stop that :) On Thu, 12 Nov 2020, Tony Wu wrote: > We are currently using memcached provided by AWS ElastiCache service in > modern mode to store web session keys > among other things. We are observing session keys being evicted before TTL > even though the cluster seems to have > ample free memory, which hints at slab imbalance. I've included a print out > of slab stats below in terms of > sizing and total / free pages. > What would be the best way to smooth out this imbalance? We already enabled > slab reassign / auto-move under > modern mode. > > Thanks. > > STAT 1:chunk_size 96 > STAT 1:total_chunks 131064 > STAT 1:free_chunks 1664 > STAT 1:free_chunks_end 0 > STAT 2:chunk_size 104 > STAT 2:total_chunks 60492 > STAT 2:free_chunks 5327 > STAT 2:free_chunks_end 0 > STAT 3:chunk_size 112 > STAT 3:total_chunks 46810 > STAT 3:free_chunks 79 > STAT 3:free_chunks_end 0 > STAT 4:chunk_size 120 > STAT 4:total_chunks 26214 > STAT 4:free_chunks 48 > STAT 4:free_chunks_end 0 > STAT 5:chunk_size 128 > STAT 5:total_chunks 32768 > STAT 5:free_chunks 6193 > STAT 5:free_chunks_end 0 > STAT 6:chunk_size 136 > STAT 6:total_chunks 23130 > STAT 6:free_chunks 4594 > STAT 6:free_chunks_end 0 > STAT 7:chunk_size 144 > STAT 7:total_chunks 7281 > STAT 7:free_chunks 5894 > STAT 7:free_chunks_end 0 > STAT 8:chunk_size 152 > STAT 8:total_chunks 13796 > STAT 8:free_chunks 12008 > STAT 8:free_chunks_end 0 > STAT 9:chunk_size 160 > STAT 9:total_chunks 6553 > STAT 9:free_chunks 6316 > STAT 9:free_chunks_end 0 > STAT 10:chunk_size 168 > STAT 10:total_chunks 6241 > STAT 10:free_chunks 6186 > STAT 10:free_chunks_end 0 > STAT 11:chunk_size 176 > STAT 11:total_chunks 5957 > STAT 11:free_chunks 5946 > STAT 11:free_chunks_end 0 > STAT 12:chunk_size 184 > STAT 12:total_chunks 5698 > STAT 12:free_chunks 5664 > STAT 12:free_chunks_end 0 > STAT 13:chunk_size 192 > STAT 13:total_chunks 5461 > STAT 13:free_chunks 5380 > STAT 13:free_chunks_end 0 > STAT 14:chunk_size 200 > STAT 14:total_chunks 5242 > STAT 14:free_chunks 5135 > STAT 14:free_chunks_end 0 > STAT 15:chunk_size 208 > STAT 15:total_chunks 5041 > STAT 15:free_chunks 4892 > STAT 15:free_chunks_end 0 > STAT 16:chunk_size 216 > STAT 16:total_chunks 4854 > STAT 16:free_chunks 4770 > STAT 16:free_chunks_end 0 > STAT 17:chunk_size 224 > STAT 17:total_chunks 4681 > STAT 17:free_chunks 4655 > STAT 17:free_chunks_end 0 > STAT 18:chunk_size 232 > STAT 18:total_chunks 4519 > STAT 18:free_chunks 4489 > STAT 18:free_chunks_end 0 > STAT 19:chunk_size 240 > STAT 19:total_chunks 4369 > STAT 19:free_chunks 4345 > STAT 19:free_chunks_end 0 > STAT 20:chunk_size 248 > STAT 20:total_chunks 4228 > STAT 20:free_chunks 4217 > STAT 20:free_chunks_end 0 > STAT 21:chunk_size 256 > STAT 21:total_chunks 4096 > STAT 21:free_chunks 4091 > STAT 21:free_chunks_end 0 > STAT 22:chunk_size 264 > STAT 22:total_chunks 3971 > STAT 22:free_chunks 3966 > STAT 22:free_chunks_end 0 > STAT 23:chunk_size 272 > STAT 23:total_chunks 3855 > STAT 23:free_chunks 3851 > STAT 23:free_chunks_end 0 > STAT 24:chunk_size 280 > STAT 24:total_chunks 3744 > STAT 24:free_chunks 3738 > STAT 24:free_chunks_end 0 > STAT 25:chunk_size 288 > STAT 25:total_chunks 3640 > STAT 25:free_chunks 3638 > STAT 25:free_chunks_end 0 > STAT 26:chunk_size 296 > STAT 26:total_chunks 3542 > STAT 26:free_chunks 3533 > STAT 26:free_chunks_end 0 > STAT 27:chunk_size 304 > STAT 27:total_chunks 3449 > STAT 27:free_chunks 3448 > STAT 27:free_chunks_end 0 > STAT 28:chunk_size 312 > STAT 28:total_chunks 3360 > STAT 28:free_chunks 3350 > STAT 28:free_chunks_end 0 > STAT 29:chunk_size 320 > STAT 29:total_chunks 3276 > STAT 29:free_chunks 3273 > STAT 29:free_chunks_end 0 > STAT 30:chunk_size 328 > STAT 30:total_chunks 3196 > STAT 30:free_chunks 3174 > STAT 30:free_chunks_end 0 > STAT 31:chunk_size 336 > STAT 31:total_chunks 3120 > STAT 31:free_chunks 3118 > STAT 31:free_chunks_end 0 > STAT 32:chunk_size 344 > STAT 32:total_chunks 3048 > STAT 32:free_chunks 3043 > STAT 32:free_chunks_end 0 > STAT 33:chunk_size 352 > STAT 33:total_chunks 2978 > STAT 33:free_chunks 2954 > STAT 33:free_chunks_end 0 > STAT 34:chunk_size 360 > STAT 34:total_chunks 2912 > STAT 34:free_chunks 2903 > STAT 34:free_chunks_end 0 > STAT 35:chunk_size 368 > STAT 35:total_chunks 2849 > STAT 35:free_chunks 2828 > STAT 35:free_chunks_end 0 > STAT 36:chunk_size 376 > STAT 36:total_chunks 2788 > STAT 36:free_chunks 2766 > STAT 36:free_chunks_end 0 > STAT 37:chunk_size 384 > STAT 37:total_chunks 2730 > STAT 37:free_chunks 2684 > STAT 37:free_chunks_end 0 > STAT 38:chunk_size 392 > STAT 38:total_chunks 2674 > STAT 38:free_chunks 2621 > STAT 38:free_chunks_end 0 > STAT 39:chunk_size 400 > STAT 39:total_chunks 2621 > STAT 39:free_chunks 2564 > STAT 39:free_chunks_end 0 > STAT 40:chunk_size 408 > STAT 40:total_chunks 2570 > STAT 40:free_chunks 2527 > STAT 40:free_chunks_end 0 > STAT 41:chunk_size 416 > STAT 41:total_chunks 2520 > STAT 41:free_chunks 2216 > STAT 41:free_chunks_end 0 > STAT 42:chunk_size 424 > STAT 42:total_chunks 2473 > STAT 42:free_chunks 2256 > STAT 42:free_chunks_end 0 > STAT 43:chunk_size 432 > STAT 43:total_chunks 2427 > STAT 43:free_chunks 2299 > STAT 43:free_chunks_end 0 > STAT 44:chunk_size 440 > STAT 44:total_chunks 2383 > STAT 44:free_chunks 1898 > STAT 44:free_chunks_end 0 > STAT 45:chunk_size 448 > STAT 45:total_chunks 2340 > STAT 45:free_chunks 1962 > STAT 45:free_chunks_end 0 > STAT 46:chunk_size 456 > STAT 46:total_chunks 2299 > STAT 46:free_chunks 2192 > STAT 46:free_chunks_end 0 > STAT 47:chunk_size 472 > STAT 47:total_chunks 2221 > STAT 47:free_chunks 2061 > STAT 47:free_chunks_end 0 > STAT 48:chunk_size 488 > STAT 48:total_chunks 2148 > STAT 48:free_chunks 2047 > STAT 48:free_chunks_end 0 > STAT 49:chunk_size 504 > STAT 49:total_chunks 2080 > STAT 49:free_chunks 1955 > STAT 49:free_chunks_end 0 > STAT 50:chunk_size 520 > STAT 50:total_chunks 2016 > STAT 50:free_chunks 1788 > STAT 50:free_chunks_end 0 > STAT 51:chunk_size 536 > STAT 51:total_chunks 1956 > STAT 51:free_chunks 1870 > STAT 51:free_chunks_end 0 > STAT 52:chunk_size 552 > STAT 52:total_chunks 1899 > STAT 52:free_chunks 1829 > STAT 52:free_chunks_end 0 > STAT 53:chunk_size 568 > STAT 53:total_chunks 1846 > STAT 53:free_chunks 1762 > STAT 53:free_chunks_end 0 > STAT 54:chunk_size 584 > STAT 54:total_chunks 1795 > STAT 54:free_chunks 1613 > STAT 54:free_chunks_end 0 > STAT 55:chunk_size 600 > STAT 55:total_chunks 1747 > STAT 55:free_chunks 1446 > STAT 55:free_chunks_end 0 > STAT 56:chunk_size 616 > STAT 56:total_chunks 1702 > STAT 56:free_chunks 1358 > STAT 56:free_chunks_end 0 > STAT 57:chunk_size 632 > STAT 57:total_chunks 1659 > STAT 57:free_chunks 1497 > STAT 57:free_chunks_end 0 > STAT 58:chunk_size 648 > STAT 58:total_chunks 1618 > STAT 58:free_chunks 1398 > STAT 58:free_chunks_end 0 > STAT 59:chunk_size 664 > STAT 59:total_chunks 3158 > STAT 59:free_chunks 1784 > STAT 59:free_chunks_end 0 > STAT 60:chunk_size 680 > STAT 60:total_chunks 1542 > STAT 60:free_chunks 1301 > STAT 60:free_chunks_end 0 > STAT 61:chunk_size 696 > STAT 61:total_chunks 3012 > STAT 61:free_chunks 302 > STAT 61:free_chunks_end 0 > STAT 62:chunk_size 712 > STAT 62:total_chunks 4416 > STAT 62:free_chunks 631 > STAT 62:free_chunks_end 0 > STAT 63:chunk_size 524288 > STAT 63:total_chunks 26108 > STAT 63:free_chunks 0 > STAT 63:free_chunks_end 0 > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/927615f7-187d-4c8f-9e08-f1f341b9a04an%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.22.394.2011121424300.2800104%40dskull.
