[email protected] (Ferenc Wágner) writes: > For a start I attached the dump output from another node.
I meant to...
146 dlm_controld 4.0.5 started 146 our_nodeid 167773708 146 found /dev/misc/dlm-control minor 58 146 found /dev/misc/dlm-monitor minor 57 146 found /dev/misc/dlm_plock minor 56 146 /sys/kernel/config/dlm/cluster/comms: opendir failed: 2 146 /sys/kernel/config/dlm/cluster/spaces: opendir failed: 2 146 set recover_callbacks 1 146 cmap totem.cluster_name = 'vhbl' 146 set cluster_name vhbl 146 /dev/misc/dlm-monitor fd 10 146 cluster quorum 1 seq 3648 nodes 5 146 cluster node 167773705 added seq 3648 146 set_configfs_node 167773705 10.0.6.9 local 0 146 cluster node 167773707 added seq 3648 146 set_configfs_node 167773707 10.0.6.11 local 0 146 cluster node 167773708 added seq 3648 146 set_configfs_node 167773708 10.0.6.12 local 1 146 cluster node 167773709 added seq 3648 146 set_configfs_node 167773709 10.0.6.13 local 0 146 cluster node 167773710 added seq 3648 146 set_configfs_node 167773710 10.0.6.14 local 0 146 cpg_join dlm:controld ... 146 setup_cpg_daemon 12 146 dlm:controld conf 5 1 0 memb 167773705 167773707 167773708 167773709 167773710 join 167773708 left 146 daemon joined 167773705 146 daemon joined 167773707 146 daemon joined 167773708 146 daemon joined 167773709 146 daemon joined 167773710 146 dlm:controld ring 167773705:3648 5 memb 167773705 167773707 167773708 167773709 167773710 146 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 146 daemon node 167773705 prot max 0.0.0.0 run 0.0.0.0 146 daemon node 167773705 save max 3.1.1.0 run 3.1.1.1 146 run protocol from nodeid 167773705 146 daemon run 3.1.1 max 3.1.1 kernel run 1.1.1 max 1.1.1 146 plocks 13 146 receive_fence_clear from 167773705 for 167773708 result 0 flags 6 146 fence_in_progress_unknown 0 recv 146 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.1 146 daemon node 167773707 prot max 0.0.0.0 run 0.0.0.0 146 daemon node 167773707 save max 3.1.1.0 run 3.1.1.1 146 receive_protocol 167773708 max 3.1.1.0 run 0.0.0.0 146 daemon node 167773708 prot max 0.0.0.0 run 0.0.0.0 146 daemon node 167773708 save max 3.1.1.0 run 0.0.0.0 146 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.0 146 daemon node 167773708 prot max 3.1.1.0 run 0.0.0.0 146 daemon node 167773708 save max 3.1.1.0 run 3.1.1.0 146 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 146 daemon node 167773709 prot max 0.0.0.0 run 0.0.0.0 146 daemon node 167773709 save max 3.1.1.0 run 3.1.1.1 146 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.1 146 daemon node 167773710 prot max 0.0.0.0 run 0.0.0.0 146 daemon node 167773710 save max 3.1.1.0 run 3.1.1.1 147 uevent: add@/kernel/dlm/clvmd 147 kernel: add@ clvmd 147 uevent: online@/kernel/dlm/clvmd 147 kernel: online@ clvmd 147 clvmd cpg_join dlm:ls:clvmd ... 147 dlm:ls:clvmd conf 5 1 0 memb 167773705 167773707 167773708 167773709 167773710 join 167773708 left 147 clvmd add_change cg 1 joined nodeid 167773708 147 clvmd add_change cg 1 we joined 147 clvmd add_change cg 1 counts member 5 joined 1 remove 0 failed 0 147 clvmd check_ringid cluster 3648 cpg 0:0 147 dlm:ls:clvmd ring 167773705:3648 5 memb 167773705 167773707 167773708 167773709 167773710 147 clvmd check_ringid done cluster 3648 cpg 167773705:3648 147 clvmd check_fencing disabled 147 clvmd send_start 167773708:1 counts 0 5 1 0 0 147 clvmd wait_messages cg 1 need 5 of 5 147 clvmd receive_start 167773708:1 len 92 147 clvmd match_change 167773708:1 matches cg 1 147 clvmd wait_messages cg 1 need 4 of 5 147 clvmd receive_start 167773709:12 len 92 147 clvmd match_change 167773709:12 matches cg 1 147 clvmd wait_messages cg 1 need 3 of 5 147 clvmd receive_start 167773710:14 len 92 147 clvmd match_change 167773710:14 matches cg 1 147 clvmd wait_messages cg 1 need 2 of 5 147 clvmd receive_start 167773705:4 len 92 147 clvmd match_change 167773705:4 matches cg 1 147 clvmd wait_messages cg 1 need 1 of 5 147 clvmd receive_start 167773707:8 len 92 147 clvmd match_change 167773707:8 matches cg 1 147 clvmd wait_messages cg 1 got all 5 147 clvmd start_kernel cg 1 member_count 5 147 write "1090842362" to "/sys/kernel/dlm/clvmd/id" 147 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773705" 147 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773707" 147 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773708" 147 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773709" 147 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773710" 147 write "1" to "/sys/kernel/dlm/clvmd/control" 147 write "0" to "/sys/kernel/dlm/clvmd/event_done" 147 clvmd prepare_plocks 147 clvmd set_plock_data_node from 0 to 167773705 147 clvmd save_plocks start 147 clvmd receive_plocks_done 167773705:4 flags 2 plocks_data 0 need 1 save 1 147 clvmd match_change 167773705:4 matches cg 1 147 clvmd process_saved_plocks begin 147 clvmd process_saved_plocks 0 done 147 clvmd receive_plocks_done 167773705:4 plocks_data_count 0 147 uevent: add@/devices/virtual/misc/dlm_clvmd 371 dlm:controld ring 167773705:3652 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 371 fence work wait for cluster ringid 371 dlm:ls:clvmd ring 167773705:3652 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 371 fence work wait for cluster ringid 371 cluster quorum 1 seq 3652 nodes 6 371 cluster node 167773706 added seq 3652 371 set_configfs_node 167773706 10.0.6.10 local 0 373 dlm:controld conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773706 left 373 daemon joined 167773706 373 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.1 373 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 373 receive_fence_clear from 167773705 for 167773706 result 0 flags 6 373 receive_protocol 167773706 max 3.1.1.0 run 0.0.0.0 373 daemon node 167773706 prot max 0.0.0.0 run 0.0.0.0 373 daemon node 167773706 save max 3.1.1.0 run 0.0.0.0 373 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.0 373 daemon node 167773706 prot max 3.1.1.0 run 0.0.0.0 373 daemon node 167773706 save max 3.1.1.0 run 3.1.1.0 373 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.1 373 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.1 373 daemon node 167773708 prot max 3.1.1.0 run 3.1.1.0 373 daemon node 167773708 save max 3.1.1.0 run 3.1.1.1 373 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 374 dlm:ls:clvmd conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773706 left 374 clvmd add_change cg 2 joined nodeid 167773706 374 clvmd add_change cg 2 counts member 6 joined 1 remove 0 failed 0 374 clvmd stop_kernel cg 2 374 write "0" to "/sys/kernel/dlm/clvmd/control" 374 clvmd check_ringid done cluster 3652 cpg 167773705:3652 374 clvmd check_fencing disabled 374 clvmd send_start 167773708:2 counts 1 6 1 0 0 374 clvmd receive_start 167773710:15 len 96 374 clvmd match_change 167773710:15 matches cg 2 374 clvmd wait_messages cg 2 need 5 of 6 374 clvmd receive_start 167773705:5 len 96 374 clvmd match_change 167773705:5 matches cg 2 374 clvmd wait_messages cg 2 need 4 of 6 374 clvmd receive_start 167773706:1 len 96 374 clvmd match_change 167773706:1 matches cg 2 374 clvmd wait_messages cg 2 need 3 of 6 374 clvmd receive_start 167773707:9 len 96 374 clvmd match_change 167773707:9 matches cg 2 374 clvmd wait_messages cg 2 need 2 of 6 374 clvmd receive_start 167773708:2 len 96 374 clvmd match_change 167773708:2 matches cg 2 374 clvmd wait_messages cg 2 need 1 of 6 374 clvmd receive_start 167773709:13 len 96 374 clvmd match_change 167773709:13 matches cg 2 374 clvmd wait_messages cg 2 got all 6 374 clvmd start_kernel cg 2 member_count 6 374 dir_member 167773710 374 dir_member 167773709 374 dir_member 167773708 374 dir_member 167773707 374 dir_member 167773705 374 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773706" 374 write "1" to "/sys/kernel/dlm/clvmd/control" 374 clvmd prepare_plocks 374 clvmd set_plock_data_node from 167773705 to 167773705 374 clvmd receive_plocks_done 167773705:5 flags 2 plocks_data 0 need 0 save 0 2914 dlm:ls:clvmd conf 5 0 1 memb 167773706 167773707 167773708 167773709 167773710 join left 167773705 2914 clvmd add_change cg 3 remove nodeid 167773705 reason leave 2914 clvmd add_change cg 3 counts member 5 joined 0 remove 1 failed 0 2914 clvmd stop_kernel cg 3 2914 write "0" to "/sys/kernel/dlm/clvmd/control" 2914 clvmd purged 0 plocks for 167773705 2914 clvmd check_ringid done cluster 3652 cpg 167773705:3652 2914 clvmd check_fencing disabled 2914 clvmd send_start 167773708:3 counts 2 5 0 1 0 2914 clvmd receive_start 167773710:16 len 92 2914 clvmd match_change 167773710:16 matches cg 3 2914 clvmd wait_messages cg 3 need 4 of 5 2914 clvmd receive_start 167773706:2 len 92 2914 clvmd match_change 167773706:2 matches cg 3 2914 clvmd wait_messages cg 3 need 3 of 5 2914 clvmd receive_start 167773707:10 len 92 2914 clvmd match_change 167773707:10 matches cg 3 2914 clvmd wait_messages cg 3 need 2 of 5 2914 clvmd receive_start 167773708:3 len 92 2914 clvmd match_change 167773708:3 matches cg 3 2914 clvmd wait_messages cg 3 need 1 of 5 2914 clvmd receive_start 167773709:14 len 92 2914 clvmd match_change 167773709:14 matches cg 3 2914 clvmd wait_messages cg 3 got all 5 2914 clvmd start_kernel cg 3 member_count 5 2914 dir_member 167773706 2914 dir_member 167773710 2914 dir_member 167773709 2914 dir_member 167773708 2914 dir_member 167773707 2914 dir_member 167773705 2914 set_members rmdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773705" 2914 write "1" to "/sys/kernel/dlm/clvmd/control" 2914 clvmd prepare_plocks 2914 clvmd set_plock_data_node from 167773705 to 167773706 2914 clvmd receive_plocks_done 167773706:2 flags 2 plocks_data 4199895776 need 0 save 0 2914 dlm:controld conf 5 0 1 memb 167773706 167773707 167773708 167773709 167773710 join left 167773705 2914 dlm:controld left reason nodedown 0 procdown 0 leave 1 2914 dlm:ls:clvmd ring 167773706:3656 5 memb 167773706 167773707 167773708 167773709 167773710 2914 dlm:controld ring 167773706:3656 5 memb 167773706 167773707 167773708 167773709 167773710 2914 fence work wait for cluster ringid 2914 fence work wait for cluster ringid 2914 cluster quorum 1 seq 3656 nodes 5 2914 cluster node 167773705 removed seq 3656 2914 del_configfs_node rmdir "/sys/kernel/config/dlm/cluster/comms/167773705" 3157 dlm:ls:clvmd ring 167773705:3660 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 3157 dlm:controld ring 167773705:3660 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 3157 fence work wait for cluster ringid 3157 fence work wait for cluster ringid 3157 cluster quorum 1 seq 3660 nodes 6 3157 cluster node 167773705 added seq 3660 3157 set_configfs_node 167773705 10.0.6.9 local 0 3157 dlm:controld conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773705 left 3157 daemon joined 167773705 3157 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.1 3157 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 3157 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.1 3157 receive_protocol 167773705 max 3.1.1.0 run 0.0.0.0 3157 daemon node 167773705 prot max 0.0.0.0 run 0.0.0.0 3157 daemon node 167773705 save max 3.1.1.0 run 0.0.0.0 3157 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.0 3157 daemon node 167773705 prot max 3.1.1.0 run 0.0.0.0 3157 daemon node 167773705 save max 3.1.1.0 run 3.1.1.0 3157 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 3157 daemon node 167773706 prot max 3.1.1.0 run 3.1.1.0 3157 daemon node 167773706 save max 3.1.1.0 run 3.1.1.1 3157 receive_fence_clear from 167773706 for 167773705 result 0 flags 6 3157 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.1 3158 dlm:ls:clvmd conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773705 left 3158 clvmd add_change cg 4 joined nodeid 167773705 3158 clvmd add_change cg 4 counts member 6 joined 1 remove 0 failed 0 3158 clvmd stop_kernel cg 4 3158 write "0" to "/sys/kernel/dlm/clvmd/control" 3158 clvmd check_ringid done cluster 3660 cpg 167773705:3660 3158 clvmd check_fencing disabled 3158 clvmd send_start 167773708:4 counts 3 6 1 0 0 3158 clvmd receive_start 167773709:15 len 96 3158 clvmd match_change 167773709:15 matches cg 4 3158 clvmd wait_messages cg 4 need 5 of 6 3158 clvmd receive_start 167773710:17 len 96 3158 clvmd match_change 167773710:17 matches cg 4 3158 clvmd wait_messages cg 4 need 4 of 6 3158 clvmd receive_start 167773705:1 len 96 3158 clvmd match_change 167773705:1 matches cg 4 3158 clvmd wait_messages cg 4 need 3 of 6 3158 clvmd receive_start 167773706:3 len 96 3158 clvmd match_change 167773706:3 matches cg 4 3158 clvmd wait_messages cg 4 need 2 of 6 3158 clvmd receive_start 167773707:11 len 96 3158 clvmd match_change 167773707:11 matches cg 4 3158 clvmd wait_messages cg 4 need 1 of 6 3158 clvmd receive_start 167773708:4 len 96 3158 clvmd match_change 167773708:4 matches cg 4 3158 clvmd wait_messages cg 4 got all 6 3158 clvmd start_kernel cg 4 member_count 6 3158 dir_member 167773706 3158 dir_member 167773710 3158 dir_member 167773709 3158 dir_member 167773708 3158 dir_member 167773707 3158 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773705" 3158 write "1" to "/sys/kernel/dlm/clvmd/control" 3158 clvmd prepare_plocks 3158 clvmd set_plock_data_node from 167773706 to 167773706 3158 clvmd receive_plocks_done 167773706:3 flags 2 plocks_data 0 need 0 save 0 13097 dlm:ls:clvmd conf 5 0 1 memb 167773705 167773706 167773708 167773709 167773710 join left 167773707 13097 clvmd add_change cg 5 remove nodeid 167773707 reason leave 13097 clvmd add_change cg 5 counts member 5 joined 0 remove 1 failed 0 13097 clvmd stop_kernel cg 5 13097 write "0" to "/sys/kernel/dlm/clvmd/control" 13097 clvmd purged 0 plocks for 167773707 13097 clvmd check_ringid done cluster 3660 cpg 167773705:3660 13097 clvmd check_fencing disabled 13097 clvmd send_start 167773708:5 counts 4 5 0 1 0 13097 clvmd receive_start 167773705:2 len 92 13097 clvmd match_change 167773705:2 matches cg 5 13097 clvmd wait_messages cg 5 need 4 of 5 13097 clvmd receive_start 167773708:5 len 92 13097 clvmd match_change 167773708:5 matches cg 5 13097 clvmd wait_messages cg 5 need 3 of 5 13097 clvmd receive_start 167773709:16 len 92 13097 clvmd match_change 167773709:16 matches cg 5 13097 clvmd wait_messages cg 5 need 2 of 5 13097 clvmd receive_start 167773710:18 len 92 13097 clvmd match_change 167773710:18 matches cg 5 13097 clvmd wait_messages cg 5 need 1 of 5 13097 clvmd receive_start 167773706:4 len 92 13097 clvmd match_change 167773706:4 matches cg 5 13097 clvmd wait_messages cg 5 got all 5 13097 clvmd start_kernel cg 5 member_count 5 13097 dir_member 167773705 13097 dir_member 167773706 13097 dir_member 167773710 13097 dir_member 167773709 13097 dir_member 167773708 13097 dir_member 167773707 13097 set_members rmdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773707" 13097 write "1" to "/sys/kernel/dlm/clvmd/control" 13097 clvmd prepare_plocks 13097 clvmd set_plock_data_node from 167773706 to 167773705 13097 clvmd receive_plocks_done 167773705:2 flags 2 plocks_data 2053243616 need 0 save 0 13098 dlm:controld conf 5 0 1 memb 167773705 167773706 167773708 167773709 167773710 join left 167773707 13098 dlm:controld left reason nodedown 0 procdown 0 leave 1 13098 dlm:ls:clvmd ring 167773705:3664 5 memb 167773705 167773706 167773708 167773709 167773710 13098 dlm:controld ring 167773705:3664 5 memb 167773705 167773706 167773708 167773709 167773710 13098 fence work wait for cluster ringid 13098 fence work wait for cluster ringid 13098 cluster quorum 1 seq 3664 nodes 5 13098 cluster node 167773707 removed seq 3664 13098 del_configfs_node rmdir "/sys/kernel/config/dlm/cluster/comms/167773707" 13380 dlm:ls:clvmd ring 167773705:3668 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 13380 dlm:controld ring 167773705:3668 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 13380 fence work wait for cluster ringid 13380 fence work wait for cluster ringid 13380 cluster quorum 1 seq 3668 nodes 6 13380 cluster node 167773707 added seq 3668 13380 set_configfs_node 167773707 10.0.6.11 local 0 13380 dlm:controld conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773707 left 13380 daemon joined 167773707 13380 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 13380 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.1 13380 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 13380 receive_protocol 167773707 max 3.1.1.0 run 0.0.0.0 13380 daemon node 167773707 prot max 0.0.0.0 run 0.0.0.0 13380 daemon node 167773707 save max 3.1.1.0 run 0.0.0.0 13380 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.0 13380 daemon node 167773707 prot max 3.1.1.0 run 0.0.0.0 13380 daemon node 167773707 save max 3.1.1.0 run 3.1.1.0 13380 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.1 13380 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 13380 daemon node 167773705 prot max 3.1.1.0 run 3.1.1.0 13380 daemon node 167773705 save max 3.1.1.0 run 3.1.1.1 13380 receive_fence_clear from 167773705 for 167773707 result 0 flags 6 13380 dlm:ls:clvmd conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773707 left 13380 clvmd add_change cg 6 joined nodeid 167773707 13380 clvmd add_change cg 6 counts member 6 joined 1 remove 0 failed 0 13380 clvmd stop_kernel cg 6 13380 write "0" to "/sys/kernel/dlm/clvmd/control" 13380 clvmd check_ringid done cluster 3668 cpg 167773705:3668 13380 clvmd check_fencing disabled 13380 clvmd send_start 167773708:6 counts 5 6 1 0 0 13380 clvmd receive_start 167773705:3 len 96 13380 clvmd match_change 167773705:3 matches cg 6 13380 clvmd wait_messages cg 6 need 5 of 6 13380 clvmd receive_start 167773706:5 len 96 13380 clvmd match_change 167773706:5 matches cg 6 13380 clvmd wait_messages cg 6 need 4 of 6 13380 clvmd receive_start 167773707:1 len 96 13380 clvmd match_change 167773707:1 matches cg 6 13380 clvmd wait_messages cg 6 need 3 of 6 13380 clvmd receive_start 167773708:6 len 96 13380 clvmd match_change 167773708:6 matches cg 6 13380 clvmd wait_messages cg 6 need 2 of 6 13380 clvmd receive_start 167773709:17 len 96 13380 clvmd match_change 167773709:17 matches cg 6 13380 clvmd wait_messages cg 6 need 1 of 6 13380 clvmd receive_start 167773710:19 len 96 13380 clvmd match_change 167773710:19 matches cg 6 13380 clvmd wait_messages cg 6 got all 6 13380 clvmd start_kernel cg 6 member_count 6 13380 dir_member 167773705 13380 dir_member 167773706 13380 dir_member 167773710 13380 dir_member 167773709 13380 dir_member 167773708 13380 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773707" 13380 write "1" to "/sys/kernel/dlm/clvmd/control" 13380 clvmd prepare_plocks 13380 clvmd set_plock_data_node from 167773705 to 167773705 13380 clvmd receive_plocks_done 167773705:3 flags 2 plocks_data 0 need 0 save 0 17969 dlm:ls:clvmd conf 5 0 1 memb 167773705 167773706 167773707 167773708 167773710 join left 167773709 17969 clvmd add_change cg 7 remove nodeid 167773709 reason leave 17969 clvmd add_change cg 7 counts member 5 joined 0 remove 1 failed 0 17969 clvmd stop_kernel cg 7 17969 write "0" to "/sys/kernel/dlm/clvmd/control" 17969 clvmd purged 0 plocks for 167773709 17969 clvmd check_ringid done cluster 3668 cpg 167773705:3668 17969 clvmd check_fencing disabled 17969 clvmd send_start 167773708:7 counts 6 5 0 1 0 17969 clvmd receive_start 167773707:2 len 92 17969 clvmd match_change 167773707:2 matches cg 7 17969 clvmd wait_messages cg 7 need 4 of 5 17969 clvmd receive_start 167773708:7 len 92 17969 clvmd match_change 167773708:7 matches cg 7 17969 clvmd wait_messages cg 7 need 3 of 5 17969 clvmd receive_start 167773710:20 len 92 17969 clvmd match_change 167773710:20 matches cg 7 17969 clvmd wait_messages cg 7 need 2 of 5 17969 clvmd receive_start 167773705:4 len 92 17969 clvmd match_change 167773705:4 matches cg 7 17969 clvmd wait_messages cg 7 need 1 of 5 17969 clvmd receive_start 167773706:6 len 92 17969 clvmd match_change 167773706:6 matches cg 7 17969 clvmd wait_messages cg 7 got all 5 17969 clvmd start_kernel cg 7 member_count 5 17969 dir_member 167773707 17969 dir_member 167773705 17969 dir_member 167773706 17969 dir_member 167773710 17969 dir_member 167773709 17969 dir_member 167773708 17969 set_members rmdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773709" 17969 write "1" to "/sys/kernel/dlm/clvmd/control" 17969 clvmd prepare_plocks 17969 clvmd set_plock_data_node from 167773705 to 167773705 17969 clvmd receive_plocks_done 167773705:4 flags 2 plocks_data 2053243616 need 0 save 0 17969 dlm:controld conf 5 0 1 memb 167773705 167773706 167773707 167773708 167773710 join left 167773709 17969 dlm:controld left reason nodedown 0 procdown 0 leave 1 17970 dlm:ls:clvmd ring 167773705:3672 5 memb 167773705 167773706 167773707 167773708 167773710 17970 dlm:controld ring 167773705:3672 5 memb 167773705 167773706 167773707 167773708 167773710 17970 fence work wait for cluster ringid 17970 fence work wait for cluster ringid 17970 cluster quorum 1 seq 3672 nodes 5 17970 cluster node 167773709 removed seq 3672 17970 del_configfs_node rmdir "/sys/kernel/config/dlm/cluster/comms/167773709" 18228 dlm:controld ring 167773705:3676 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 18228 fence work wait for cluster ringid 18228 dlm:ls:clvmd ring 167773705:3676 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 18228 fence work wait for cluster ringid 18228 cluster quorum 1 seq 3676 nodes 6 18228 cluster node 167773709 added seq 3676 18228 set_configfs_node 167773709 10.0.6.13 local 0 18228 dlm:controld conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773709 left 18228 daemon joined 167773709 18228 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 18228 receive_fence_clear from 167773705 for 167773709 result 0 flags 6 18228 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 18228 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.1 18228 daemon node 167773707 prot max 3.1.1.0 run 3.1.1.0 18228 daemon node 167773707 save max 3.1.1.0 run 3.1.1.1 18228 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.1 18228 receive_protocol 167773709 max 3.1.1.0 run 0.0.0.0 18228 daemon node 167773709 prot max 0.0.0.0 run 0.0.0.0 18228 daemon node 167773709 save max 3.1.1.0 run 0.0.0.0 18228 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.0 18228 daemon node 167773709 prot max 3.1.1.0 run 0.0.0.0 18228 daemon node 167773709 save max 3.1.1.0 run 3.1.1.0 18228 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.1 18228 dlm:ls:clvmd conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773709 left 18228 clvmd add_change cg 8 joined nodeid 167773709 18228 clvmd add_change cg 8 counts member 6 joined 1 remove 0 failed 0 18228 clvmd stop_kernel cg 8 18228 write "0" to "/sys/kernel/dlm/clvmd/control" 18228 clvmd check_ringid done cluster 3676 cpg 167773705:3676 18228 clvmd check_fencing disabled 18228 clvmd send_start 167773708:8 counts 7 6 1 0 0 18228 clvmd receive_start 167773708:8 len 96 18228 clvmd match_change 167773708:8 matches cg 8 18228 clvmd wait_messages cg 8 need 5 of 6 18228 clvmd receive_start 167773709:1 len 96 18228 clvmd match_change 167773709:1 matches cg 8 18228 clvmd wait_messages cg 8 need 4 of 6 18228 clvmd receive_start 167773710:21 len 96 18228 clvmd match_change 167773710:21 matches cg 8 18228 clvmd wait_messages cg 8 need 3 of 6 18228 clvmd receive_start 167773705:5 len 96 18228 clvmd match_change 167773705:5 matches cg 8 18228 clvmd wait_messages cg 8 need 2 of 6 18228 clvmd receive_start 167773706:7 len 96 18228 clvmd match_change 167773706:7 matches cg 8 18228 clvmd wait_messages cg 8 need 1 of 6 18228 clvmd receive_start 167773707:3 len 96 18228 clvmd match_change 167773707:3 matches cg 8 18228 clvmd wait_messages cg 8 got all 6 18228 clvmd start_kernel cg 8 member_count 6 18228 dir_member 167773707 18228 dir_member 167773705 18228 dir_member 167773706 18228 dir_member 167773710 18228 dir_member 167773708 18228 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773709" 18228 write "1" to "/sys/kernel/dlm/clvmd/control" 18228 clvmd prepare_plocks 18228 clvmd set_plock_data_node from 167773705 to 167773705 18228 clvmd receive_plocks_done 167773705:5 flags 2 plocks_data 0 need 0 save 0 22600 dlm:ls:clvmd conf 5 0 1 memb 167773705 167773706 167773707 167773708 167773709 join left 167773710 22600 clvmd add_change cg 9 remove nodeid 167773710 reason leave 22600 clvmd add_change cg 9 counts member 5 joined 0 remove 1 failed 0 22600 clvmd stop_kernel cg 9 22600 write "0" to "/sys/kernel/dlm/clvmd/control" 22600 clvmd purged 0 plocks for 167773710 22600 clvmd check_ringid done cluster 3676 cpg 167773705:3676 22600 clvmd check_fencing disabled 22600 clvmd send_start 167773708:9 counts 8 5 0 1 0 22600 clvmd receive_start 167773709:2 len 92 22600 clvmd match_change 167773709:2 matches cg 9 22600 clvmd wait_messages cg 9 need 4 of 5 22600 clvmd receive_start 167773705:6 len 92 22600 clvmd match_change 167773705:6 matches cg 9 22600 clvmd wait_messages cg 9 need 3 of 5 22600 clvmd receive_start 167773706:8 len 92 22600 clvmd match_change 167773706:8 matches cg 9 22600 clvmd wait_messages cg 9 need 2 of 5 22600 clvmd receive_start 167773707:4 len 92 22600 clvmd match_change 167773707:4 matches cg 9 22600 clvmd wait_messages cg 9 need 1 of 5 22600 clvmd receive_start 167773708:9 len 92 22600 clvmd match_change 167773708:9 matches cg 9 22600 clvmd wait_messages cg 9 got all 5 22600 clvmd start_kernel cg 9 member_count 5 22600 dir_member 167773709 22600 dir_member 167773707 22600 dir_member 167773705 22600 dir_member 167773706 22600 dir_member 167773710 22600 dir_member 167773708 22600 set_members rmdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773710" 22600 write "1" to "/sys/kernel/dlm/clvmd/control" 22600 clvmd prepare_plocks 22600 clvmd set_plock_data_node from 167773705 to 167773705 22600 clvmd receive_plocks_done 167773705:6 flags 2 plocks_data 2053243616 need 0 save 0 22601 dlm:controld conf 5 0 1 memb 167773705 167773706 167773707 167773708 167773709 join left 167773710 22601 dlm:controld left reason nodedown 0 procdown 0 leave 1 22601 dlm:ls:clvmd ring 167773705:3680 5 memb 167773705 167773706 167773707 167773708 167773709 22601 dlm:controld ring 167773705:3680 5 memb 167773705 167773706 167773707 167773708 167773709 22601 fence work wait for cluster ringid 22601 fence work wait for cluster ringid 22602 fence work wait for cluster ringid 22603 fence work wait for cluster ringid 22604 fence work wait for cluster ringid 22605 fence work wait for cluster ringid 22606 fence work wait for cluster ringid 22607 fence work wait for cluster ringid 22608 fence work wait for cluster ringid 22609 fence work wait for cluster ringid 22610 fence work wait for cluster ringid 22611 fence work wait for cluster ringid 22612 fence work wait for cluster ringid 22613 fence work wait for cluster ringid 22614 fence work wait for cluster ringid 22615 fence work wait for cluster ringid 22616 fence work wait for cluster ringid 22617 fence work wait for cluster ringid 22618 fence work wait for cluster ringid 22619 fence work wait for cluster ringid 22620 fence work wait for cluster ringid 22621 fence work wait for cluster ringid 22622 fence work wait for cluster ringid 22623 fence work wait for cluster ringid 22624 fence work wait for cluster ringid 22625 fence work wait for cluster ringid 22626 fence work wait for cluster ringid 22627 fence work wait for cluster ringid 22628 dlm:controld ring 167773705:3712 5 memb 167773705 167773706 167773707 167773708 167773709 22628 fence work wait for cluster ringid 22628 dlm:ls:clvmd ring 167773705:3712 5 memb 167773705 167773706 167773707 167773708 167773709 22628 fence work wait for cluster ringid 22628 cluster quorum 1 seq 3712 nodes 5 22628 cluster node 167773710 removed seq 3712 22628 del_configfs_node rmdir "/sys/kernel/config/dlm/cluster/comms/167773710" 22628 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 22628 receive_fence_clear from 167773705 for 167773708 result 0 flags 6 22628 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 22628 receive_fence_clear from 167773705 for 167773707 result 0 flags 6 22628 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 22628 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 22628 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 22628 daemon node 167773709 prot max 3.1.1.0 run 3.1.1.0 22628 daemon node 167773709 save max 3.1.1.0 run 3.1.1.1 22628 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 22628 clvmd receive_start 167773705:11 len 92 22628 clvmd find_change 167773705:11 no match 22628 clvmd receive_start 167773706:13 len 92 22628 clvmd find_change 167773706:13 no match 22628 clvmd receive_start 167773709:7 len 92 22628 clvmd find_change 167773709:7 no match 24470 dlm:ls:clvmd ring 167773705:3724 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 24470 dlm:controld ring 167773705:3724 6 memb 167773705 167773706 167773707 167773708 167773709 167773710 24470 fence work wait for cluster ringid 24470 fence work wait for cluster ringid 24470 cluster quorum 1 seq 3724 nodes 6 24470 cluster node 167773710 added seq 3724 24470 set_configfs_node 167773710 10.0.6.14 local 0 24470 dlm:controld conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773710 left 24470 daemon joined 167773710 24470 receive_protocol 167773707 max 3.1.1.0 run 3.1.1.1 24470 receive_protocol 167773708 max 3.1.1.0 run 3.1.1.1 24470 receive_protocol 167773709 max 3.1.1.0 run 3.1.1.1 24470 receive_protocol 167773710 max 3.1.1.0 run 0.0.0.0 24470 daemon node 167773710 prot max 0.0.0.0 run 0.0.0.0 24470 daemon node 167773710 save max 3.1.1.0 run 0.0.0.0 24470 receive_protocol 167773710 max 3.1.1.0 run 3.1.1.0 24470 daemon node 167773710 prot max 3.1.1.0 run 0.0.0.0 24470 daemon node 167773710 save max 3.1.1.0 run 3.1.1.0 24470 receive_protocol 167773705 max 3.1.1.0 run 3.1.1.1 24470 receive_protocol 167773706 max 3.1.1.0 run 3.1.1.1 24470 dlm:ls:clvmd conf 6 1 0 memb 167773705 167773706 167773707 167773708 167773709 167773710 join 167773710 left 24470 clvmd add_change cg 10 joined nodeid 167773710 24470 clvmd add_change cg 10 counts member 6 joined 1 remove 0 failed 0 24470 clvmd stop_kernel cg 10 24470 write "0" to "/sys/kernel/dlm/clvmd/control" 24470 clvmd check_ringid done cluster 3724 cpg 167773705:3724 24470 clvmd check_fencing disabled 24470 clvmd send_start 167773708:10 counts 9 6 1 0 0 24470 clvmd receive_start 167773709:8 len 96 24470 clvmd match_change 167773709:8 matches cg 10 24470 clvmd wait_messages cg 10 need 5 of 6 24470 clvmd receive_start 167773710:1 len 96 24470 clvmd match_change 167773710:1 matches cg 10 24470 clvmd wait_messages cg 10 need 4 of 6 24470 clvmd receive_start 167773705:12 len 96 24470 clvmd match_change 167773705:12 matches cg 10 24470 clvmd wait_messages cg 10 need 3 of 6 24470 clvmd receive_start 167773706:14 len 96 24470 clvmd match_change 167773706:14 matches cg 10 24470 clvmd wait_messages cg 10 need 2 of 6 24470 clvmd receive_start 167773707:5 len 96 24470 clvmd match_change 167773707:5 matches cg 10 24470 clvmd wait_messages cg 10 need 1 of 6 24470 clvmd receive_start 167773708:10 len 96 24470 clvmd match_change 167773708:10 matches cg 10 24470 clvmd wait_messages cg 10 got all 6 24470 clvmd start_kernel cg 10 member_count 6 24470 dir_member 167773709 24470 dir_member 167773707 24470 dir_member 167773705 24470 dir_member 167773706 24470 dir_member 167773708 24470 set_members mkdir "/sys/kernel/config/dlm/cluster/spaces/clvmd/nodes/167773710" 24470 write "1" to "/sys/kernel/dlm/clvmd/control" 24470 clvmd prepare_plocks 24470 clvmd set_plock_data_node from 167773705 to 167773705
-- Regards, Feri
_______________________________________________ Users mailing list: [email protected] https://lists.clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
