I have seen a similar behavior in #708. The standby controller did not join the
AMF cluster saying "invalid nodeid"
---
** [tickets:#638] node cannot join AMF cluster after restart**
**Status:** accepted
**Created:** Fri Nov 22, 2013 02:54 PM UTC by Hans Feldt
**Last Updated:** Thu Jan 16, 2014 03:20 AM UTC
**Owner:** A V Mahesh (AVM)
OpenSAF 4.2.2 changeset 3796, 79 extra patches
System: RHEL based, 2 node cluster, MDS/TIPC
After node reboot of the standby controller it cannot join the cluster again.
This can be seen in the syslog on the active controller:
Nov 17 17:15:20 notice atrcxb3166 osafamfd[6038]: Cold sync complete!
Nov 19 17:40:07 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' joined the cluster
Nov 19 17:42:08 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 19 17:42:28 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
<repeats>
Nov 21 16:24:21 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' left the cluster
Nov 21 16:29:04 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' joined the cluster
Nov 21 16:29:24 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:29:44 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:30:04 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:30:24 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:30:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:31:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:31:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:31:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:32:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:32:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:32:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:33:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:33:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:33:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:34:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:34:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:34:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:35:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:35:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:35:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:36:14 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:36:34 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 16:36:54 warning atrcxb3166 osafamfd[6712]: invalid node state 1 for
node 2020f
Nov 21 17:41:58 err atrcxb3166 osafamfd[6712]: avd_d2n_msg_dequeue: ncsmds_api
failed 2
Nov 21 17:42:08 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' left the cluster
Nov 21 17:42:18 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:42:38 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:42:58 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:43:18 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:43:39 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:43:59 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:44:19 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:44:39 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:44:59 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:45:19 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:45:39 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:45:59 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:46:19 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
node ID (2020f)
Nov 21 17:46:39 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
msg id 210, from 2020f should be 1
Nov 21 17:46:59 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
msg id 211, from 2020f should be 1
<repeats>
Nov 21 18:00:40 warning atrcxb3166 osafamfd[6712]: avd_msg_sanity_chk: invalid
msg id 252, from 2020f should be 1
Nov 21 18:01:00 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' left the cluster
Nov 22 11:44:37 notice atrcxb3166 osafamfd[6712]: Re-initializing with IMM
Nov 22 11:44:39 notice atrcxb3166 osafamfd[6712]: Finished re-initializing with
IMM
Nov 22 12:08:05 notice atrcxb3166 osafamfd[6712]: Node 'SC-2' joined the cluster
What is noteable here is that amfd cannot use imm during this problem state.
See http://sourceforge.net/p/opensaf/tickets/637/
After immnd was restart at Nov 22 11:44:37 and SC2 was rebooted the system
recovered.
---
Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Opensaf-tickets mailing list
Opensaf-tickets@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
