haodiao commented on issue #3208: URL: https://github.com/apache/kvrocks/issues/3208#issuecomment-3427427315
master server Info ``` > connecting...... > 192.168.88.88@6379 connected! > info replication # Replication role:master connected_slaves:3 slave0:ip=10.8.8.13,port=6666,offset=11935,lag=10 slave1:ip=10.8.8.14,port=6666,offset=11935,lag=10 slave2:ip=10.8.8.12,port=6666,offset=11918,lag=27 master_repl_offset:11945 > info # Server version:unstable kvrocks_version:unstable redis_version:4.0.0 git_sha1:7d52e8d kvrocks_git_sha1:7d52e8d redis_mode:standalone kvrocks_mode:standalone os:Linux 6.12.43+deb13-amd64 x86_64 gcc_version:12.2.0 rocksdb_version:10.4.2 arch_bits:64 process_id:1 tcp_port:6666 server_time_usec:1761061641074554 uptime_in_seconds:724816 uptime_in_days:8 executable:/usr/bin/kvrocks config_file:/kvrocks/kvrocks.conf # Clients maxclients:10000 connected_clients:12 monitor_clients:0 blocked_clients:1 # Memory used_memory_rss:178163712 used_memory_rss_human:169.91M used_memory_lua:4013056 used_memory_lua_human:3.83M used_memory_startup:57872384 mem_allocator:jemalloc # Persistence loading:0 bgsave_in_progress:0 last_bgsave_time:1760336825 last_bgsave_status:ok last_bgsave_time_sec:-1 # Stats total_connections_received:71751 total_commands_processed:1759790 instantaneous_ops_per_sec:3 total_net_input_bytes:134026737 total_net_output_bytes:151442902 instantaneous_input_kbps:0.000000 instantaneous_output_kbps:0.000000 sync_full:5 sync_partial_ok:31 sync_partial_err:5 keyspace_hits:8552 keyspace_misses:1737078 pubsub_channels:0 pubsub_patterns:0 # Replication role:master connected_slaves:3 slave0:ip=10.8.8.13,port=6666,offset=11935,lag=10 slave1:ip=10.8.8.14,port=6666,offset=11935,lag=10 slave2:ip=10.8.8.12,port=6666,offset=11918,lag=27 master_repl_offset:11945 # CPU used_cpu_sys:39439.000000 used_cpu_user:12841.000000 # CommandStats cmdstat__db_name:calls=36,usec=538,usec_per_call=14.944444444444445 cmdstat__fetch_file:calls=5,usec=765,usec_per_call=153 cmdstat__fetch_meta:calls=5,usec=767,usec_per_call=153.4 cmdstat_auth:calls=318,usec=962,usec_per_call=3.0251572327044025 cmdstat_brpop:calls=1563151,usec=93382114,usec_per_call=59.739663026796514 cmdstat_client:calls=16,usec=1426,usec_per_call=89.125 cmdstat_config:calls=17,usec=530,usec_per_call=31.176470588235293 cmdstat_del:calls=305,usec=60255,usec_per_call=197.55737704918033 cmdstat_eval:calls=806,usec=688152,usec_per_call=853.786600496278 cmdstat_exists:calls=39,usec=2834,usec_per_call=72.66666666666667 cmdstat_expire:calls=54,usec=8135,usec_per_call=150.64814814814815 cmdstat_get:calls=3411,usec=163907,usec_per_call=48.05247727939021 cmdstat_hdel:calls=276,usec=69198,usec_per_call=250.7173913043478 cmdstat_hello:calls=312,usec=7669,usec_per_call=24.580128205128204 cmdstat_hget:calls=955,usec=79272,usec_per_call=83.00732984293194 cmdstat_hgetall:calls=16,usec=5534,usec_per_call=345.875 cmdstat_hlen:calls=99,usec=5401,usec_per_call=54.55555555555556 cmdstat_hmget:calls=1111,usec=413590,usec_per_call=372.26822682268227 cmdstat_hmset:calls=49,usec=15378,usec_per_call=313.83673469387753 cmdstat_hscan:calls=99,usec=126540,usec_per_call=1278.1818181818182 cmdstat_hset:calls=629,usec=199567,usec_per_call=317.2766295707472 cmdstat_incrby:calls=5,usec=1007,usec_per_call=201.4 cmdstat_info:calls=33,usec=55389,usec_per_call=1678.4545454545455 cmdstat_ping:calls=11002,usec=22987,usec_per_call=2.089347391383385 cmdstat_psync:calls=36,usec=712142,usec_per_call=19781.722222222223 cmdstat_quit:calls=4,usec=183,usec_per_call=45.75 cmdstat_replconf:calls=36,usec=4,usec_per_call=0.1111111111111111 cmdstat_rpush:calls=430,usec=128067,usec_per_call=297.8302325581395 cmdstat_sadd:calls=17,usec=3892,usec_per_call=228.94117647058823 cmdstat_scan:calls=21,usec=19264,usec_per_call=917.3333333333334 cmdstat_scard:calls=1,usec=1107,usec_per_call=1107 cmdstat_set:calls=1015,usec=200427,usec_per_call=197.46502463054188 cmdstat_spop:calls=1226,usec=82175,usec_per_call=67.02691680261012 cmdstat_sscan:calls=1,usec=198,usec_per_call=198 cmdstat_ttl:calls=127,usec=7103,usec_per_call=55.92913385826772 cmdstat_type:calls=131,usec=8042,usec_per_call=61.38931297709924 cmdstat_zadd:calls=25,usec=7889,usec_per_call=315.56 cmdstat_zcard:calls=4,usec=138,usec_per_call=34.5 cmdstat_zrangebyscore:calls=173940,usec=9521410,usec_per_call=54.73962285845694 cmdstat_zrem:calls=23,usec=7524,usec_per_call=327.1304347826087 cmdstat_zrevrange:calls=4,usec=3003,usec_per_call=750.75 # Cluster cluster_enabled:0 # Keyspace last_dbsize_scan_timestamp:0 db0:keys=0,expires=0,avg_ttl=0,expired=0 sequence:11945 used_db_size:2061183 max_db_size:0 used_percent:0% disk_capacity:495252013056 used_disk_size:8486780928 used_disk_percent:1.713628759554455% # RocksDB block_cache_usage:2279136 block_cache_pinned_usage[default]:1184 estimate_keys[default]:185 index_and_filter_cache_usage[default]:2566 level0_file_limit_slowdown[default]:0 level0_file_limit_stop[default]:0 pending_compaction_bytes_slowdown[default]:0 pending_compaction_bytes_stop[default]:0 level0_file_limit_stop_with_ongoing_compaction[default]:0 level0_file_limit_slowdown_with_ongoing_compaction[default]:0 memtable_count_limit_slowdown[default]:0 memtable_count_limit_stop[default]:0 num_files_at_level[default]:[0,0,0,0,0,0,1] estimate_pending_compaction_bytes[default]:0 estimate_keys[metadata]:1781 index_and_filter_cache_usage[metadata]:2556 level0_file_limit_slowdown[metadata]:0 level0_file_limit_stop[metadata]:0 pending_compaction_bytes_slowdown[metadata]:0 pending_compaction_bytes_stop[metadata]:0 level0_file_limit_stop_with_ongoing_compaction[metadata]:0 level0_file_limit_slowdown_with_ongoing_compaction[metadata]:0 memtable_count_limit_slowdown[metadata]:0 memtable_count_limit_stop[metadata]:0 num_files_at_level[metadata]:[0,0,0,0,0,0,1] estimate_pending_compaction_bytes[metadata]:0 estimate_keys[zset_score]:2 index_and_filter_cache_usage[zset_score]:0 level0_file_limit_slowdown[zset_score]:0 level0_file_limit_stop[zset_score]:0 pending_compaction_bytes_slowdown[zset_score]:0 pending_compaction_bytes_stop[zset_score]:0 level0_file_limit_stop_with_ongoing_compaction[zset_score]:0 level0_file_limit_slowdown_with_ongoing_compaction[zset_score]:0 memtable_count_limit_slowdown[zset_score]:0 memtable_count_limit_stop[zset_score]:0 num_files_at_level[zset_score]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[zset_score]:0 estimate_keys[pubsub]:0 index_and_filter_cache_usage[pubsub]:0 level0_file_limit_slowdown[pubsub]:0 level0_file_limit_stop[pubsub]:0 pending_compaction_bytes_slowdown[pubsub]:0 pending_compaction_bytes_stop[pubsub]:0 level0_file_limit_stop_with_ongoing_compaction[pubsub]:0 level0_file_limit_slowdown_with_ongoing_compaction[pubsub]:0 memtable_count_limit_slowdown[pubsub]:0 memtable_count_limit_stop[pubsub]:0 num_files_at_level[pubsub]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[pubsub]:0 estimate_keys[propagate]:0 index_and_filter_cache_usage[propagate]:0 level0_file_limit_slowdown[propagate]:0 level0_file_limit_stop[propagate]:0 pending_compaction_bytes_slowdown[propagate]:0 pending_compaction_bytes_stop[propagate]:0 level0_file_limit_stop_with_ongoing_compaction[propagate]:0 level0_file_limit_slowdown_with_ongoing_compaction[propagate]:0 memtable_count_limit_slowdown[propagate]:0 memtable_count_limit_stop[propagate]:0 num_files_at_level[propagate]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[propagate]:0 estimate_keys[stream]:0 index_and_filter_cache_usage[stream]:0 level0_file_limit_slowdown[stream]:0 level0_file_limit_stop[stream]:0 pending_compaction_bytes_slowdown[stream]:0 pending_compaction_bytes_stop[stream]:0 level0_file_limit_stop_with_ongoing_compaction[stream]:0 level0_file_limit_slowdown_with_ongoing_compaction[stream]:0 memtable_count_limit_slowdown[stream]:0 memtable_count_limit_stop[stream]:0 num_files_at_level[stream]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[stream]:0 estimate_keys[search]:0 index_and_filter_cache_usage[search]:0 level0_file_limit_slowdown[search]:0 level0_file_limit_stop[search]:0 pending_compaction_bytes_slowdown[search]:0 pending_compaction_bytes_stop[search]:0 level0_file_limit_stop_with_ongoing_compaction[search]:0 level0_file_limit_slowdown_with_ongoing_compaction[search]:0 memtable_count_limit_slowdown[search]:0 memtable_count_limit_stop[search]:0 num_files_at_level[search]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[search]:0 estimate_keys[index]:0 index_and_filter_cache_usage[index]:0 level0_file_limit_slowdown[index]:0 level0_file_limit_stop[index]:0 pending_compaction_bytes_slowdown[index]:0 pending_compaction_bytes_stop[index]:0 level0_file_limit_stop_with_ongoing_compaction[index]:0 level0_file_limit_slowdown_with_ongoing_compaction[index]:0 memtable_count_limit_slowdown[index]:0 memtable_count_limit_stop[index]:0 num_files_at_level[index]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[index]:0 block_cache_data_hit:2175 block_cache_data_miss:60 block_cache_filter_hit:1140831 block_cache_filter_miss:13 block_cache_hit:1144834 block_cache_index_hit:1828 block_cache_index_miss:14 block_cache_miss:87 all_mem_tables:24657920 cur_mem_tables:24657920 snapshots:0 num_immutable_tables:0 num_running_flushes:0 memtable_flush_pending:0 compaction_pending:0 num_running_compactions:0 num_live_versions:8 num_super_version:12 num_background_errors:0 flush_count:1 compaction_count:2 put_per_sec:0 get_per_sec:3 seek_per_sec:0 next_per_sec:0 prev_per_sec:0 is_bgsaving:no is_compacting:no ``` slave server info: ``` > connecting...... > 10.8.8.12 connected! > info # Server version:unstable kvrocks_version:unstable redis_version:4.0.0 git_sha1:2215279 kvrocks_git_sha1:2215279 redis_mode:standalone kvrocks_mode:standalone os:Linux 5.10.0-33-amd64 x86_64 clang_version:14.0.6 rocksdb_version:10.4.2 arch_bits:64 process_id:1 tcp_port:6666 server_time_usec:1761062014846945 uptime_in_seconds:132523 uptime_in_days:1 executable:/usr/bin/kvrocks config_file:/var/lib/kvrocks/kvrocks.conf # Clients maxclients:10000 connected_clients:12 monitor_clients:0 blocked_clients:0 # Memory used_memory_rss:32169984 used_memory_rss_human:30.68M used_memory_lua:1922048 used_memory_lua_human:1.83M used_memory_startup:34017280 mem_allocator:jemalloc # Persistence loading:0 bgsave_in_progress:0 last_bgsave_time:1760929491 last_bgsave_status:ok last_bgsave_time_sec:-1 # Stats total_connections_received:4472 total_commands_processed:8393 instantaneous_ops_per_sec:0 total_net_input_bytes:316309 total_net_output_bytes:8571820 instantaneous_input_kbps:0.000000 instantaneous_output_kbps:0.000000 sync_full:0 sync_partial_ok:0 sync_partial_err:0 keyspace_hits:2442 keyspace_misses:1098 pubsub_channels:0 pubsub_patterns:0 # Replication role:slave master_host:10.8.8.8 master_port:6379 master_link_status:up master_sync_unrecoverable_error:no master_sync_in_progress:0 master_last_io_seconds_ago:1 slave_repl_offset:11918 slave_priority:100 connected_slaves:0 master_repl_offset:11918 # CPU used_cpu_sys:72.000000 used_cpu_user:79.000000 # CommandStats cmdstat_client:calls=5,usec=2,usec_per_call=0.4 cmdstat_config:calls=4,usec=1603,usec_per_call=400.75 cmdstat_eval:calls=305,usec=59330,usec_per_call=194.52459016393442 cmdstat_hello:calls=54,usec=473,usec_per_call=8.75925925925926 cmdstat_hget:calls=2793,usec=139311,usec_per_call=49.87862513426423 cmdstat_hgetall:calls=442,usec=41395,usec_per_call=93.65384615384616 cmdstat_hmget:calls=305,usec=36906,usec_per_call=121.00327868852459 cmdstat_info:calls=7,usec=12616,usec_per_call=1802.2857142857142 cmdstat_ping:calls=4474,usec=1264,usec_per_call=0.2825212337952615 cmdstat_scan:calls=4,usec=1389,usec_per_call=347.25 # Cluster cluster_enabled:0 # Keyspace last_dbsize_scan_timestamp:0 db0:keys=0,expires=0,avg_ttl=0,expired=0 sequence:11918 used_db_size:2090993 max_db_size:0 used_percent:0% disk_capacity:15502950400 used_disk_size:5432086528 used_disk_percent:35.03904990884832% # RocksDB block_cache_usage:1115504 block_cache_pinned_usage[default]:1184 estimate_keys[default]:133 index_and_filter_cache_usage[default]:2566 level0_file_limit_slowdown[default]:0 level0_file_limit_stop[default]:0 pending_compaction_bytes_slowdown[default]:0 pending_compaction_bytes_stop[default]:0 level0_file_limit_stop_with_ongoing_compaction[default]:0 level0_file_limit_slowdown_with_ongoing_compaction[default]:0 memtable_count_limit_slowdown[default]:0 memtable_count_limit_stop[default]:0 num_files_at_level[default]:[0,0,0,0,0,0,1] estimate_pending_compaction_bytes[default]:0 estimate_keys[metadata]:673 index_and_filter_cache_usage[metadata]:2556 level0_file_limit_slowdown[metadata]:0 level0_file_limit_stop[metadata]:0 pending_compaction_bytes_slowdown[metadata]:0 pending_compaction_bytes_stop[metadata]:0 level0_file_limit_stop_with_ongoing_compaction[metadata]:0 level0_file_limit_slowdown_with_ongoing_compaction[metadata]:0 memtable_count_limit_slowdown[metadata]:0 memtable_count_limit_stop[metadata]:0 num_files_at_level[metadata]:[0,0,0,0,0,0,1] estimate_pending_compaction_bytes[metadata]:0 estimate_keys[zset_score]:2 index_and_filter_cache_usage[zset_score]:0 level0_file_limit_slowdown[zset_score]:0 level0_file_limit_stop[zset_score]:0 pending_compaction_bytes_slowdown[zset_score]:0 pending_compaction_bytes_stop[zset_score]:0 level0_file_limit_stop_with_ongoing_compaction[zset_score]:0 level0_file_limit_slowdown_with_ongoing_compaction[zset_score]:0 memtable_count_limit_slowdown[zset_score]:0 memtable_count_limit_stop[zset_score]:0 num_files_at_level[zset_score]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[zset_score]:0 estimate_keys[pubsub]:0 index_and_filter_cache_usage[pubsub]:0 level0_file_limit_slowdown[pubsub]:0 level0_file_limit_stop[pubsub]:0 pending_compaction_bytes_slowdown[pubsub]:0 pending_compaction_bytes_stop[pubsub]:0 level0_file_limit_stop_with_ongoing_compaction[pubsub]:0 level0_file_limit_slowdown_with_ongoing_compaction[pubsub]:0 memtable_count_limit_slowdown[pubsub]:0 memtable_count_limit_stop[pubsub]:0 num_files_at_level[pubsub]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[pubsub]:0 estimate_keys[propagate]:0 index_and_filter_cache_usage[propagate]:0 level0_file_limit_slowdown[propagate]:0 level0_file_limit_stop[propagate]:0 pending_compaction_bytes_slowdown[propagate]:0 pending_compaction_bytes_stop[propagate]:0 level0_file_limit_stop_with_ongoing_compaction[propagate]:0 level0_file_limit_slowdown_with_ongoing_compaction[propagate]:0 memtable_count_limit_slowdown[propagate]:0 memtable_count_limit_stop[propagate]:0 num_files_at_level[propagate]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[propagate]:0 estimate_keys[stream]:0 index_and_filter_cache_usage[stream]:0 level0_file_limit_slowdown[stream]:0 level0_file_limit_stop[stream]:0 pending_compaction_bytes_slowdown[stream]:0 pending_compaction_bytes_stop[stream]:0 level0_file_limit_stop_with_ongoing_compaction[stream]:0 level0_file_limit_slowdown_with_ongoing_compaction[stream]:0 memtable_count_limit_slowdown[stream]:0 memtable_count_limit_stop[stream]:0 num_files_at_level[stream]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[stream]:0 estimate_keys[search]:0 index_and_filter_cache_usage[search]:0 level0_file_limit_slowdown[search]:0 level0_file_limit_stop[search]:0 pending_compaction_bytes_slowdown[search]:0 pending_compaction_bytes_stop[search]:0 level0_file_limit_stop_with_ongoing_compaction[search]:0 level0_file_limit_slowdown_with_ongoing_compaction[search]:0 memtable_count_limit_slowdown[search]:0 memtable_count_limit_stop[search]:0 num_files_at_level[search]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[search]:0 estimate_keys[index]:0 index_and_filter_cache_usage[index]:0 level0_file_limit_slowdown[index]:0 level0_file_limit_stop[index]:0 pending_compaction_bytes_slowdown[index]:0 pending_compaction_bytes_stop[index]:0 level0_file_limit_stop_with_ongoing_compaction[index]:0 level0_file_limit_slowdown_with_ongoing_compaction[index]:0 memtable_count_limit_slowdown[index]:0 memtable_count_limit_stop[index]:0 num_files_at_level[index]:[0,0,0,0,0,0,0] estimate_pending_compaction_bytes[index]:0 block_cache_data_hit:3566 block_cache_data_miss:231 block_cache_filter_hit:4826 block_cache_filter_miss:11 block_cache_hit:11958 block_cache_index_hit:3566 block_cache_index_miss:12 block_cache_miss:254 all_mem_tables:13844480 cur_mem_tables:13844480 snapshots:0 num_immutable_tables:0 num_running_flushes:0 memtable_flush_pending:0 compaction_pending:0 num_running_compactions:0 num_live_versions:8 num_super_version:10 num_background_errors:0 flush_count:0 compaction_count:2 put_per_sec:0 get_per_sec:0 seek_per_sec:0 next_per_sec:0 prev_per_sec:0 is_bgsaving:no is_compacting:no ``` It seems to be ok, but it just stopped syncing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
