Re: Memcached Testing Evictions
Oh I see. Yes that is the case. Thank you very much for helping me out!! On Thu, Apr 29, 2021 at 2:07 AM dormando wrote: > Hey, > > I don't know what you're doing but I can say there's nothing buffering > prints. Maybe you're starting memcached as a daemon? Prints won't work in > that case. > > On Thu, 29 Apr 2021, Qingchen Dang wrote: > > > Thank you for helping! Yes I actually figured it out that I used wrong > parameters to load data which causes the overwrite. > > > > But I do have one more question, how to add print statements to > memcached source code? When I use ‘make test’, printf works as usual. But > in the actual run, it seems the print is re-bufferred to somewhere else > which makes it very hard to debug. Is there a way to print to stdout? > > > > Thank you very much! > > QD > > > > Sent from my iPhone > > > > > On Apr 28, 2021, at 10:33 PM, dormando wrote: > > > > > > How're you loading the data? > > > > > > From the stats it looks like you're probably overwriting the same > values > > > over and over (high total_items but low curr_items and no get_expired) > > > > > >> On Wed, 28 Apr 2021, Qingchen Dang wrote: > > >> > > >> Hi, > > >> I am trying to test my optimization of Memcached eviction, but it > seems even though I got a lot of misses, the evictions stats is always 0. > > >> Here is the stats for the original Memcached without any change. I > wonder what memcached config/benchmark config can make eviction happen? > > >> > > >> STAT pointer_size 64 > > >> > > >> STAT rusage_user 110.668826 > > >> > > >> STAT rusage_system 769.767632 > > >> > > >> STAT max_connections 1024 > > >> > > >> STAT curr_connections 2 > > >> > > >> STAT total_connections 8004 > > >> > > >> STAT rejected_connections 0 > > >> > > >> STAT connection_structures 402 > > >> > > >> STAT response_obj_oom 0 > > >> > > >> STAT response_obj_count 1 > > >> > > >> STAT response_obj_bytes 131072 > > >> > > >> STAT read_buf_count 16 > > >> > > >> STAT read_buf_bytes 262144 > > >> > > >> STAT read_buf_bytes_free 114688 > > >> > > >> STAT read_buf_oom 0 > > >> > > >> STAT reserved_fds 40 > > >> > > >> STAT cmd_get 7272 > > >> > > >> STAT cmd_set 728 > > >> > > >> STAT cmd_flush 0 > > >> > > >> STAT get_hits 626512 > > >> > > >> STAT get_misses 72093488 > > >> > > >> STAT get_expired 0 > > >> > > >> STAT get_flushed 0 > > >> > > >> STAT delete_misses 0 > > >> > > >> STAT delete_hits 0 > > >> > > >> STAT incr_misses 0 > > >> > > >> STAT incr_hits 0 > > >> > > >> STAT decr_misses 0 > > >> > > >> STAT decr_hits 0 > > >> > > >> STAT bytes_read 1809568092 > > >> > > >> STAT bytes_written 459401348 > > >> > > >> STAT limit_maxbytes 2097152 > > >> > > >> STAT accepting_conns 1 > > >> > > >> STAT listen_disabled_num 0 > > >> > > >> STAT time_in_listen_disabled_us 0 > > >> > > >> STAT threads 8 > > >> > > >> STAT conn_yields 0 > > >> > > >> STAT hash_power_level 16 > > >> > > >> STAT hash_bytes 524288 > > >> > > >> STAT hash_is_expanding 0 > > >> > > >> STAT slab_reassign_rescues 0 > > >> > > >> STAT slab_reassign_chunk_rescues 0 > > >> > > >> STAT slab_reassign_evictions_nomem 0 > > >> > > >> STAT slab_reassign_inline_reclaim 0 > > >> > > >> STAT slab_reassign_busy_items 0 > > >> > > >> STAT slab_reassign_busy_deletes 0 > > >> > > >> STAT slab_reassign_running 0 > > >> > > >> STAT slabs_moved 0 > > >> > > >> STAT lru_crawler_running 0 > > >> > > >> STAT lru_crawler_starts 4 > > >> > > >> STAT lru_maintainer_juggles 101778 > > >> > > >> STAT malloc_fails 0 > > >> > > >> STAT bytes 94437 > > >> > > >> STAT curr_items 909 > > >> > > >> STAT total_items 728 > > >> > > >> STAT slab_global_page_pool 0 > > >> > > >> STAT expired_unfetched 0 > > >> > > >> STAT evicted_unfetched 0 > > >> > > >> STAT evicted_active 0 > > >> > > >> STAT evictions 0 > > >> > > >> STAT reclaimed 0 > > >> > > >> STAT crawler_reclaimed 0 > > >> > > >> STAT crawler_items_checked 2732 > > >> > > >> STAT lrutail_reflocked 12 > > >> > > >> STAT moves_to_cold 116840 > > >> > > >> STAT moves_to_warm 3185 > > >> > > >> STAT moves_within_lru 538 > > >> > > >> STAT direct_reclaims 0 > > >> > > >> STAT lru_bumps_dropped 0 > > >> > > >> > > >> Thanks! > > >> > > >> QD > > >> > > >> -- > > >> > > >> --- > > >> You received this message because you are subscribed to the Google > Groups "memcached" group. > > >> To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > >> To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/95e1a261-7ad5-49c2-a917-0635b7e3cbbfn%40googlegroups.com > . > > >> > > >> > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to a topic in the > Google Groups "memcached" group. > > > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/memcached/cIh-fFlEEh8/unsubscribe. > > > To unsubscribe from this group and all its topics, send an email to >
Re: Memcached Testing Evictions
Hey, I don't know what you're doing but I can say there's nothing buffering prints. Maybe you're starting memcached as a daemon? Prints won't work in that case. On Thu, 29 Apr 2021, Qingchen Dang wrote: > Thank you for helping! Yes I actually figured it out that I used wrong > parameters to load data which causes the overwrite. > > But I do have one more question, how to add print statements to memcached > source code? When I use ‘make test’, printf works as usual. But in the actual > run, it seems the print is re-bufferred to somewhere else which makes it very > hard to debug. Is there a way to print to stdout? > > Thank you very much! > QD > > Sent from my iPhone > > > On Apr 28, 2021, at 10:33 PM, dormando wrote: > > > > How're you loading the data? > > > > From the stats it looks like you're probably overwriting the same values > > over and over (high total_items but low curr_items and no get_expired) > > > >> On Wed, 28 Apr 2021, Qingchen Dang wrote: > >> > >> Hi, > >> I am trying to test my optimization of Memcached eviction, but it seems > >> even though I got a lot of misses, the evictions stats is always 0. > >> Here is the stats for the original Memcached without any change. I wonder > >> what memcached config/benchmark config can make eviction happen? > >> > >> STAT pointer_size 64 > >> > >> STAT rusage_user 110.668826 > >> > >> STAT rusage_system 769.767632 > >> > >> STAT max_connections 1024 > >> > >> STAT curr_connections 2 > >> > >> STAT total_connections 8004 > >> > >> STAT rejected_connections 0 > >> > >> STAT connection_structures 402 > >> > >> STAT response_obj_oom 0 > >> > >> STAT response_obj_count 1 > >> > >> STAT response_obj_bytes 131072 > >> > >> STAT read_buf_count 16 > >> > >> STAT read_buf_bytes 262144 > >> > >> STAT read_buf_bytes_free 114688 > >> > >> STAT read_buf_oom 0 > >> > >> STAT reserved_fds 40 > >> > >> STAT cmd_get 7272 > >> > >> STAT cmd_set 728 > >> > >> STAT cmd_flush 0 > >> > >> STAT get_hits 626512 > >> > >> STAT get_misses 72093488 > >> > >> STAT get_expired 0 > >> > >> STAT get_flushed 0 > >> > >> STAT delete_misses 0 > >> > >> STAT delete_hits 0 > >> > >> STAT incr_misses 0 > >> > >> STAT incr_hits 0 > >> > >> STAT decr_misses 0 > >> > >> STAT decr_hits 0 > >> > >> STAT bytes_read 1809568092 > >> > >> STAT bytes_written 459401348 > >> > >> STAT limit_maxbytes 2097152 > >> > >> STAT accepting_conns 1 > >> > >> STAT listen_disabled_num 0 > >> > >> STAT time_in_listen_disabled_us 0 > >> > >> STAT threads 8 > >> > >> STAT conn_yields 0 > >> > >> STAT hash_power_level 16 > >> > >> STAT hash_bytes 524288 > >> > >> STAT hash_is_expanding 0 > >> > >> STAT slab_reassign_rescues 0 > >> > >> STAT slab_reassign_chunk_rescues 0 > >> > >> STAT slab_reassign_evictions_nomem 0 > >> > >> STAT slab_reassign_inline_reclaim 0 > >> > >> STAT slab_reassign_busy_items 0 > >> > >> STAT slab_reassign_busy_deletes 0 > >> > >> STAT slab_reassign_running 0 > >> > >> STAT slabs_moved 0 > >> > >> STAT lru_crawler_running 0 > >> > >> STAT lru_crawler_starts 4 > >> > >> STAT lru_maintainer_juggles 101778 > >> > >> STAT malloc_fails 0 > >> > >> STAT bytes 94437 > >> > >> STAT curr_items 909 > >> > >> STAT total_items 728 > >> > >> STAT slab_global_page_pool 0 > >> > >> STAT expired_unfetched 0 > >> > >> STAT evicted_unfetched 0 > >> > >> STAT evicted_active 0 > >> > >> STAT evictions 0 > >> > >> STAT reclaimed 0 > >> > >> STAT crawler_reclaimed 0 > >> > >> STAT crawler_items_checked 2732 > >> > >> STAT lrutail_reflocked 12 > >> > >> STAT moves_to_cold 116840 > >> > >> STAT moves_to_warm 3185 > >> > >> STAT moves_within_lru 538 > >> > >> STAT direct_reclaims 0 > >> > >> STAT lru_bumps_dropped 0 > >> > >> > >> Thanks! > >> > >> QD > >> > >> -- > >> > >> --- > >> You received this message because you are subscribed to the Google Groups > >> "memcached" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to memcached+unsubscr...@googlegroups.com. > >> To view this discussion on the web visit > >> https://groups.google.com/d/msgid/memcached/95e1a261-7ad5-49c2-a917-0635b7e3cbbfn%40googlegroups.com. > >> > >> > > > > -- > > > > --- > > You received this message because you are subscribed to a topic in the > > Google Groups "memcached" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/memcached/cIh-fFlEEh8/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/memcached/e274ce6a-63ab-2896-d7c0-befa57222a%40rydia.net. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit >
Re: Memcached Testing Evictions
Thank you for helping! Yes I actually figured it out that I used wrong parameters to load data which causes the overwrite. But I do have one more question, how to add print statements to memcached source code? When I use ‘make test’, printf works as usual. But in the actual run, it seems the print is re-bufferred to somewhere else which makes it very hard to debug. Is there a way to print to stdout? Thank you very much! QD Sent from my iPhone > On Apr 28, 2021, at 10:33 PM, dormando wrote: > > How're you loading the data? > > From the stats it looks like you're probably overwriting the same values > over and over (high total_items but low curr_items and no get_expired) > >> On Wed, 28 Apr 2021, Qingchen Dang wrote: >> >> Hi, >> I am trying to test my optimization of Memcached eviction, but it seems even >> though I got a lot of misses, the evictions stats is always 0. >> Here is the stats for the original Memcached without any change. I wonder >> what memcached config/benchmark config can make eviction happen? >> >> STAT pointer_size 64 >> >> STAT rusage_user 110.668826 >> >> STAT rusage_system 769.767632 >> >> STAT max_connections 1024 >> >> STAT curr_connections 2 >> >> STAT total_connections 8004 >> >> STAT rejected_connections 0 >> >> STAT connection_structures 402 >> >> STAT response_obj_oom 0 >> >> STAT response_obj_count 1 >> >> STAT response_obj_bytes 131072 >> >> STAT read_buf_count 16 >> >> STAT read_buf_bytes 262144 >> >> STAT read_buf_bytes_free 114688 >> >> STAT read_buf_oom 0 >> >> STAT reserved_fds 40 >> >> STAT cmd_get 7272 >> >> STAT cmd_set 728 >> >> STAT cmd_flush 0 >> >> STAT get_hits 626512 >> >> STAT get_misses 72093488 >> >> STAT get_expired 0 >> >> STAT get_flushed 0 >> >> STAT delete_misses 0 >> >> STAT delete_hits 0 >> >> STAT incr_misses 0 >> >> STAT incr_hits 0 >> >> STAT decr_misses 0 >> >> STAT decr_hits 0 >> >> STAT bytes_read 1809568092 >> >> STAT bytes_written 459401348 >> >> STAT limit_maxbytes 2097152 >> >> STAT accepting_conns 1 >> >> STAT listen_disabled_num 0 >> >> STAT time_in_listen_disabled_us 0 >> >> STAT threads 8 >> >> STAT conn_yields 0 >> >> STAT hash_power_level 16 >> >> STAT hash_bytes 524288 >> >> STAT hash_is_expanding 0 >> >> STAT slab_reassign_rescues 0 >> >> STAT slab_reassign_chunk_rescues 0 >> >> STAT slab_reassign_evictions_nomem 0 >> >> STAT slab_reassign_inline_reclaim 0 >> >> STAT slab_reassign_busy_items 0 >> >> STAT slab_reassign_busy_deletes 0 >> >> STAT slab_reassign_running 0 >> >> STAT slabs_moved 0 >> >> STAT lru_crawler_running 0 >> >> STAT lru_crawler_starts 4 >> >> STAT lru_maintainer_juggles 101778 >> >> STAT malloc_fails 0 >> >> STAT bytes 94437 >> >> STAT curr_items 909 >> >> STAT total_items 728 >> >> STAT slab_global_page_pool 0 >> >> STAT expired_unfetched 0 >> >> STAT evicted_unfetched 0 >> >> STAT evicted_active 0 >> >> STAT evictions 0 >> >> STAT reclaimed 0 >> >> STAT crawler_reclaimed 0 >> >> STAT crawler_items_checked 2732 >> >> STAT lrutail_reflocked 12 >> >> STAT moves_to_cold 116840 >> >> STAT moves_to_warm 3185 >> >> STAT moves_within_lru 538 >> >> STAT direct_reclaims 0 >> >> STAT lru_bumps_dropped 0 >> >> >> Thanks! >> >> QD >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "memcached" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to memcached+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/memcached/95e1a261-7ad5-49c2-a917-0635b7e3cbbfn%40googlegroups.com. >> >> > > -- > > --- > You received this message because you are subscribed to a topic in the Google > Groups "memcached" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/memcached/cIh-fFlEEh8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/e274ce6a-63ab-2896-d7c0-befa57222a%40rydia.net. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/55A7989D-20BC-4830-8719-83D75CFA9194%40princeton.edu.
Re: Memcached Testing Evictions
How're you loading the data? >From the stats it looks like you're probably overwriting the same values over and over (high total_items but low curr_items and no get_expired) On Wed, 28 Apr 2021, Qingchen Dang wrote: > Hi, > I am trying to test my optimization of Memcached eviction, but it seems even > though I got a lot of misses, the evictions stats is always 0. > Here is the stats for the original Memcached without any change. I wonder > what memcached config/benchmark config can make eviction happen? > > STAT pointer_size 64 > > STAT rusage_user 110.668826 > > STAT rusage_system 769.767632 > > STAT max_connections 1024 > > STAT curr_connections 2 > > STAT total_connections 8004 > > STAT rejected_connections 0 > > STAT connection_structures 402 > > STAT response_obj_oom 0 > > STAT response_obj_count 1 > > STAT response_obj_bytes 131072 > > STAT read_buf_count 16 > > STAT read_buf_bytes 262144 > > STAT read_buf_bytes_free 114688 > > STAT read_buf_oom 0 > > STAT reserved_fds 40 > > STAT cmd_get 7272 > > STAT cmd_set 728 > > STAT cmd_flush 0 > > STAT get_hits 626512 > > STAT get_misses 72093488 > > STAT get_expired 0 > > STAT get_flushed 0 > > STAT delete_misses 0 > > STAT delete_hits 0 > > STAT incr_misses 0 > > STAT incr_hits 0 > > STAT decr_misses 0 > > STAT decr_hits 0 > > STAT bytes_read 1809568092 > > STAT bytes_written 459401348 > > STAT limit_maxbytes 2097152 > > STAT accepting_conns 1 > > STAT listen_disabled_num 0 > > STAT time_in_listen_disabled_us 0 > > STAT threads 8 > > STAT conn_yields 0 > > STAT hash_power_level 16 > > STAT hash_bytes 524288 > > STAT hash_is_expanding 0 > > STAT slab_reassign_rescues 0 > > STAT slab_reassign_chunk_rescues 0 > > STAT slab_reassign_evictions_nomem 0 > > STAT slab_reassign_inline_reclaim 0 > > STAT slab_reassign_busy_items 0 > > STAT slab_reassign_busy_deletes 0 > > STAT slab_reassign_running 0 > > STAT slabs_moved 0 > > STAT lru_crawler_running 0 > > STAT lru_crawler_starts 4 > > STAT lru_maintainer_juggles 101778 > > STAT malloc_fails 0 > > STAT bytes 94437 > > STAT curr_items 909 > > STAT total_items 728 > > STAT slab_global_page_pool 0 > > STAT expired_unfetched 0 > > STAT evicted_unfetched 0 > > STAT evicted_active 0 > > STAT evictions 0 > > STAT reclaimed 0 > > STAT crawler_reclaimed 0 > > STAT crawler_items_checked 2732 > > STAT lrutail_reflocked 12 > > STAT moves_to_cold 116840 > > STAT moves_to_warm 3185 > > STAT moves_within_lru 538 > > STAT direct_reclaims 0 > > STAT lru_bumps_dropped 0 > > > Thanks! > > QD > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/95e1a261-7ad5-49c2-a917-0635b7e3cbbfn%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/e274ce6a-63ab-2896-d7c0-befa57222a%40rydia.net.
Memcached Testing Evictions
Hi, I am trying to test my optimization of Memcached eviction, but it seems even though I got a lot of misses, the evictions stats is always 0. Here is the stats for the original Memcached without any change. I wonder what memcached config/benchmark config can make eviction happen? STAT pointer_size 64 STAT rusage_user 110.668826 STAT rusage_system 769.767632 STAT max_connections 1024 STAT curr_connections 2 STAT total_connections 8004 STAT rejected_connections 0 STAT connection_structures 402 STAT response_obj_oom 0 STAT response_obj_count 1 STAT response_obj_bytes 131072 STAT read_buf_count 16 STAT read_buf_bytes 262144 STAT read_buf_bytes_free 114688 STAT read_buf_oom 0 STAT reserved_fds 40 STAT cmd_get 7272 STAT cmd_set 728 STAT cmd_flush 0 STAT get_hits 626512 STAT get_misses 72093488 STAT get_expired 0 STAT get_flushed 0 STAT delete_misses 0 STAT delete_hits 0 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT bytes_read 1809568092 STAT bytes_written 459401348 STAT limit_maxbytes 2097152 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT time_in_listen_disabled_us 0 STAT threads 8 STAT conn_yields 0 STAT hash_power_level 16 STAT hash_bytes 524288 STAT hash_is_expanding 0 STAT slab_reassign_rescues 0 STAT slab_reassign_chunk_rescues 0 STAT slab_reassign_evictions_nomem 0 STAT slab_reassign_inline_reclaim 0 STAT slab_reassign_busy_items 0 STAT slab_reassign_busy_deletes 0 STAT slab_reassign_running 0 STAT slabs_moved 0 STAT lru_crawler_running 0 STAT lru_crawler_starts 4 STAT lru_maintainer_juggles 101778 STAT malloc_fails 0 STAT bytes 94437 STAT curr_items 909 STAT total_items 728 STAT slab_global_page_pool 0 STAT expired_unfetched 0 STAT evicted_unfetched 0 STAT evicted_active 0 STAT evictions 0 STAT reclaimed 0 STAT crawler_reclaimed 0 STAT crawler_items_checked 2732 STAT lrutail_reflocked 12 STAT moves_to_cold 116840 STAT moves_to_warm 3185 STAT moves_within_lru 538 STAT direct_reclaims 0 STAT lru_bumps_dropped 0 Thanks! QD -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/95e1a261-7ad5-49c2-a917-0635b7e3cbbfn%40googlegroups.com.
