Hi, I am using Pacemaker-1.1 (devel: 7172b7323bb72c51999ce11c6fa5d3ff0a0a4b4f). The setting of "on-fail" does not become effective. For example, it becomes default action("restart") even if it specifies "stop".
[root@vm1 ~]# crm configure show | grep -A3 "primitive prmDummy1" primitive prmDummy1 ocf:pacemaker:Dummy \ op start interval="0" timeout="60s" on-fail="restart" \ op monitor interval="10s" timeout="60s" on-fail="stop" \ op stop interval="0" timeout="60s" on-fail="block" [root@vm1 ~]# [root@vm1 ~]# crm_mon -f1 ============ Last updated: Fri Apr 6 10:13:14 2012 Last change: Fri Apr 6 10:12:42 2012 via cibadmin on vm1 Stack: Heartbeat Current DC: vm1 (87e0eef1-0d86-4e8a-adfe-51f444a4054f) - partition with quorum Version: 1.1.7-7172b73 2 Nodes configured, unknown expected votes 1 Resources configured. ============ Online: [ vm1 vm2 ] prmDummy1 (ocf::pacemaker:Dummy): Started vm1 Migration summary: * Node vm1: * Node vm2: [root@vm1 ~]# [root@vm1 ~]# rm -f /var/run/Dummy-prmDummy1.state [root@vm1 ~]# crm_mon -f1 ============ Last updated: Fri Apr 6 10:13:33 2012 Last change: Fri Apr 6 10:12:42 2012 via cibadmin on vm1 Stack: Heartbeat Current DC: vm1 (87e0eef1-0d86-4e8a-adfe-51f444a4054f) - partition with quorum Version: 1.1.7-7172b73 2 Nodes configured, unknown expected votes 1 Resources configured. ============ Online: [ vm1 vm2 ] prmDummy1 (ocf::pacemaker:Dummy): Started vm2 Migration summary: * Node vm1: prmDummy1: migration-threshold=1 fail-count=1 * Node vm2: Failed actions: prmDummy1_monitor_10000 (node=vm1, call=4, rc=7, status=complete): not running [root@vm1 ~]# Attached gdb_pengine.log is a log of gdb at the time of monitor failure. Is it because the 2nd argument (variable 'key') of the find_rsc_op_entry() function is "prmDummy1_last_failure_0"? Thereby, it seems that "on-fail" cannot be identified. (L117~L205) Best Regards, Kazunori INOUE
[root@vm1 ~]# gdb /usr/libexec/pacemaker/pengine 27764 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-50.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/pacemaker/pengine...done. Attaching to program: /usr/libexec/pacemaker/pengine, process 27764 Reading symbols from /usr/lib64/libcib.so.1...done. Loaded symbols for /usr/lib64/libcib.so.1 Reading symbols from /usr/lib64/libcrmcommon.so.2...done. Loaded symbols for /usr/lib64/libcrmcommon.so.2 Reading symbols from /usr/lib64/libpe_status.so.3...done. Loaded symbols for /usr/lib64/libpe_status.so.3 Reading symbols from /usr/lib64/libpengine.so.3...done. Loaded symbols for /usr/lib64/libpengine.so.3 Reading symbols from /usr/lib64/liblrm.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/liblrm.so.2 Reading symbols from /usr/lib64/libpe_rules.so.2...done. Loaded symbols for /usr/lib64/libpe_rules.so.2 Reading symbols from /lib64/libncurses.so.5...(no debugging symbols found)...done. Loaded symbols for /lib64/libncurses.so.5 Reading symbols from /usr/lib64/libcpg.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcpg.so.4 Reading symbols from /usr/lib64/libcfg.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcfg.so.6 Reading symbols from /usr/lib64/libcmap.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcmap.so.4 Reading symbols from /usr/lib64/libquorum.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libquorum.so.5 Reading symbols from /usr/lib64/libgnutls.so.26...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libgnutls.so.26 Reading symbols from /usr/lib64/libcorosync_common.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcorosync_common.so.4 Reading symbols from /usr/lib64/libplumb.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libplumb.so.2 Reading symbols from /usr/lib64/libpils.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libpils.so.2 Reading symbols from /lib64/libbz2.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libbz2.so.1 Reading symbols from /usr/lib64/libxslt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libxslt.so.1 Reading symbols from /usr/lib64/libxml2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libxml2.so.2 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/libpam.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpam.so.0 Reading symbols from /lib64/libglib-2.0.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libglib-2.0.so.0 Reading symbols from /usr/lib64/libltdl.so.7...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libltdl.so.7 Reading symbols from /usr/lib64/libqb.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libqb.so.0 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libuuid.so.1 Reading symbols from /lib64/libtinfo.so.5...(no debugging symbols found)...done. Loaded symbols for /lib64/libtinfo.so.5 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /usr/lib64/libtasn1.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libtasn1.so.3 Reading symbols from /lib64/libgcrypt.so.11...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcrypt.so.11 Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libaudit.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libaudit.so.1 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /lib64/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libgpg-error.so.0 Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libfreebl3.so Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 0x0000003ad5cdc018 in poll () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install audit-libs-2.1.3-3.el6.x86_64 bzip2-libs-1.0.5-7.el6_0.x86_64 cluster-glue-libs-1.0.9-1.el6.x86_64 corosynclib-1.99.9-1.3.353e.el6.x86_64 glib2-2.22.5-6.el6.x86_64 glibc-2.12-1.47.el6.x86_64 gnutls-2.8.5-4.el6.x86_64 libgcrypt-1.4.5-9.el6.x86_64 libgpg-error-1.7-4.el6.x86_64 libqb-0.11.1-1.18.239d.dirty.el6.x86_64 libtasn1-2.3-3.el6.x86_64 libtool-ltdl-2.2.6-15.5.el6.x86_64 libuuid-2.17.2-12.4.el6.x86_64 libxml2-2.7.6-4.el6.x86_64 libxslt-1.1.26-2.el6.x86_64 ncurses-libs-5.7-3.20090208.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 pam-1.1.1-10.el6.x86_64 zlib-1.2.3-27.el6.x86_64 (gdb) b /DEV/pacemaker/lib/pengine/unpack.c:1595 Breakpoint 1 at 0x7fed120d7b23: file unpack.c, line 1595. (gdb) b /DEV/pacemaker/lib/pengine/utils.c:789 Breakpoint 2 at 0x7fed1230fa6b: file utils.c, line 789. (2 locations) (gdb) b /DEV/pacemaker/lib/pengine/unpack.c:2126 Breakpoint 3 at 0x7fed120d9ee8: file unpack.c, line 2126. (gdb) c Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); (gdb) disp task 1: task = 0x1200600 "start" (gdb) disp on_fail 2: on_fail = action_fail_ignore (gdb) c Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11ff960 "monitor" (gdb) c Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11f19b0 "monitor" (gdb) bt #0 unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1595 #1 0x00007fed120d7e9f in unpack_lrm_resources (node=0x1212fc0, lrm_rsc_list=0x11fdae0, data_set=0x7fff961bfd10) at unpack.c:1638 #2 0x00007fed120d3dce in unpack_status (status=0x120d080, data_set=0x7fff961bfd10) at unpack.c:695 #3 0x00007fed120d00ba in cluster_status (data_set=0x7fff961bfd10) at status.c:101 #4 0x00007fed11e81779 in stage0 (data_set=0x7fff961bfd10) at allocate.c:764 #5 0x00007fed11e7e364 in do_calculations (data_set=0x7fff961bfd10, xml_input=0x11f4370, now=0x0) at pengine.c:259 #6 0x00007fed11e7da40 in process_pe_message (msg=0x11f6470, xml_data=0x11f60b0, sender=0x11f03f0) at pengine.c:135 #7 0x00000000004013d0 in pe_msg_callback (client=0x11f03f0, user_data=0x0) at main.c:61 #8 0x0000003dbba1ecfa in G_CH_dispatch_int () from /usr/lib64/libplumb.so.2 #9 0x0000003ad5838f0e in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 #10 0x0000003ad583c938 in ?? () from /lib64/libglib-2.0.so.0 #11 0x0000003ad583cd55 in g_main_loop_run () from /lib64/libglib-2.0.so.0 #12 0x000000000040196a in main (argc=1, argv=0x7fff961c0248) at main.c:178 (gdb) c Continuing. Breakpoint 2, find_rsc_op_entry (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0") at utils.c:789 789 if (safe_str_eq(key, match_key)) { (gdb) bt #0 find_rsc_op_entry (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0") at utils.c:789 #1 0x00007fed120dca20 in custom_action (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0", task=0x11f19b0 "monitor", on_node=0x0, optional=1, save_action=0, data_set=0x7fff961bfd10) at utils.c:403 #2 0x00007fed120d92fa in unpack_rsc_op (rsc=0x1218fd0, node=0x1212fc0, xml_op=0x11f1590, next=0x0, on_fail=0x7fff961bfab8, data_set=0x7fff961bfd10) at unpack.c:1951 #3 0x00007fed120d7b56 in unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1595 #4 0x00007fed120d7e9f in unpack_lrm_resources (node=0x1212fc0, lrm_rsc_list=0x11fdae0, data_set=0x7fff961bfd10) at unpack.c:1638 #5 0x00007fed120d3dce in unpack_status (status=0x120d080, data_set=0x7fff961bfd10) at unpack.c:695 #6 0x00007fed120d00ba in cluster_status (data_set=0x7fff961bfd10) at status.c:101 #7 0x00007fed11e81779 in stage0 (data_set=0x7fff961bfd10) at allocate.c:764 #8 0x00007fed11e7e364 in do_calculations (data_set=0x7fff961bfd10, xml_input=0x11f4370, now=0x0) at pengine.c:259 #9 0x00007fed11e7da40 in process_pe_message (msg=0x11f6470, xml_data=0x11f60b0, sender=0x11f03f0) at pengine.c:135 #10 0x00000000004013d0 in pe_msg_callback (client=0x11f03f0, user_data=0x0) at main.c:61 #11 0x0000003dbba1ecfa in G_CH_dispatch_int () from /usr/lib64/libplumb.so.2 #12 0x0000003ad5838f0e in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 #13 0x0000003ad583c938 in ?? () from /lib64/libglib-2.0.so.0 #14 0x0000003ad583cd55 in g_main_loop_run () from /lib64/libglib-2.0.so.0 #15 0x000000000040196a in main (argc=1, argv=0x7fff961c0248) at main.c:178 (gdb) p key $1 = 0x1213660 "prmDummy1_last_failure_0" (gdb) disp match_key 3: match_key = 0x1219e70 "prmDummy1_start_0" (gdb) c Continuing. Breakpoint 2, find_rsc_op_entry (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0") at utils.c:789 789 if (safe_str_eq(key, match_key)) { 3: match_key = 0x1219e70 "prmDummy1_monitor_10000" (gdb) c Continuing. Breakpoint 2, find_rsc_op_entry (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0") at utils.c:789 789 if (safe_str_eq(key, match_key)) { 3: match_key = 0x1219e70 "prmDummy1_stop_0" (gdb) fin Run till exit from #0 find_rsc_op_entry (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0") at utils.c:789 0x00007fed120dca20 in custom_action (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0", task=0x11f19b0 "monitor", on_node=0x0, optional=1, save_action=0, data_set=0x7fff961bfd10) at utils.c:403 403 action->op_entry = find_rsc_op_entry(rsc, key); Value returned is $2 = (xmlNode *) 0x0 (gdb) n 405 unpack_operation(action, action->op_entry, data_set); (gdb) p action->op_entry $3 = (xmlNode *) 0x0 (gdb) d 2 (gdb) fin Run till exit from #0 custom_action (rsc=0x1218fd0, key=0x1213660 "prmDummy1_last_failure_0", task=0x11f19b0 "monitor", on_node=0x0, optional=1, save_action=0, data_set=0x7fff961bfd10) at utils.c:405 0x00007fed120d92fa in unpack_rsc_op (rsc=0x1218fd0, node=0x1212fc0, xml_op=0x11f1590, next=0x0, on_fail=0x7fff961bfab8, data_set=0x7fff961bfd10) at unpack.c:1951 1951 action = custom_action(rsc, crm_strdup(id), task, NULL, TRUE, FALSE, data_set); Value returned is $4 = (action_t *) 0x12199e0 (gdb) fin Run till exit from #0 0x00007fed120d92fa in unpack_rsc_op (rsc=0x1218fd0, node=0x1212fc0, xml_op=0x11f1590, next=0x0, on_fail=0x7fff961bfab8, data_set=0x7fff961bfd10) at unpack.c:1951 Breakpoint 3, unpack_rsc_op (rsc=0x1218fd0, node=0x1212fc0, xml_op=0x11f1590, next=0x0, on_fail=0x7fff961bfab8, data_set=0x7fff961bfd10) at unpack.c:2126 2126 if (*on_fail < action->on_fail) { (gdb) disp *on_fail 4: *on_fail = action_fail_ignore (gdb) n 2127 *on_fail = action->on_fail; 4: *on_fail = action_fail_ignore (gdb) n 2130 if (safe_str_eq(task, CRMD_ACTION_STOP)) { 4: *on_fail = action_fail_recover (gdb) fin Run till exit from #0 unpack_rsc_op (rsc=0x1218fd0, node=0x1212fc0, xml_op=0x11f1590, next=0x0, on_fail=0x7fff961bfab8, data_set=0x7fff961bfd10) at unpack.c:2130 unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1587 1587 for (gIter = sorted_op_list; gIter != NULL; gIter = gIter->next) { 2: on_fail = action_fail_recover 1: task = 0x11f19b0 "monitor" Value returned is $5 = 1 (gdb) bt #0 unpack_lrm_rsc_state (node=0x1212fc0, rsc_entry=0x11fdb80, data_set=0x7fff961bfd10) at unpack.c:1587 #1 0x00007fed120d7e9f in unpack_lrm_resources (node=0x1212fc0, lrm_rsc_list=0x11fdae0, data_set=0x7fff961bfd10) at unpack.c:1638 #2 0x00007fed120d3dce in unpack_status (status=0x120d080, data_set=0x7fff961bfd10) at unpack.c:695 #3 0x00007fed120d00ba in cluster_status (data_set=0x7fff961bfd10) at status.c:101 #4 0x00007fed11e81779 in stage0 (data_set=0x7fff961bfd10) at allocate.c:764 #5 0x00007fed11e7e364 in do_calculations (data_set=0x7fff961bfd10, xml_input=0x11f4370, now=0x0) at pengine.c:259 #6 0x00007fed11e7da40 in process_pe_message (msg=0x11f6470, xml_data=0x11f60b0, sender=0x11f03f0) at pengine.c:135 #7 0x00000000004013d0 in pe_msg_callback (client=0x11f03f0, user_data=0x0) at main.c:61 #8 0x0000003dbba1ecfa in G_CH_dispatch_int () from /usr/lib64/libplumb.so.2 #9 0x0000003ad5838f0e in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 #10 0x0000003ad583c938 in ?? () from /lib64/libglib-2.0.so.0 #11 0x0000003ad583cd55 in g_main_loop_run () from /lib64/libglib-2.0.so.0 #12 0x000000000040196a in main (argc=1, argv=0x7fff961c0248) at main.c:178 (gdb) d 3 (gdb) c Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x12133f0, rsc_entry=0x11ee940, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11ef2b0 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x12199e0, rsc_entry=0x11f53e0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11f9d30 "start" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x12199e0, rsc_entry=0x11f53e0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x120de30 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x12199e0, rsc_entry=0x11f53e0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x121cd60 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x11fcf80, rsc_entry=0x120d560, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x120f9d0 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x121a9d0, rsc_entry=0x11fdce0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x1200730 "start" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x121a9d0, rsc_entry=0x11fdce0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11fd5c0 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x121a9d0, rsc_entry=0x11fdce0, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11ff830 "monitor" (gdb) Continuing. Breakpoint 1, unpack_lrm_rsc_state (node=0x121ad60, rsc_entry=0x11f1c40, data_set=0x7fff961bfd10) at unpack.c:1595 1595 unpack_rsc_op(rsc, node, rsc_op, gIter->next, &on_fail, data_set); 2: on_fail = action_fail_ignore 1: task = 0x11ef170 "monitor" (gdb) Continuing.
_______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org