Hi Minh,

I have tested following scenarios till now and works well:
1. Faults under standalone system. Act and Std SUs are there and cluster went 
headless. Faults occurred in Act and Standby separately and together with 
recovery as restart, su f/o, node f/o, node s/o. Faults are also mixed with two 
SGs.
2. During headless, the escalations from comp restart->su restart->su f/o -> 
node f/o. Two SG was used for testings. When cluster recovers, the system works 
well as expected.

SG and node Auto repair was enabled and disabled in many test cases.

Further testing will continue:
1. Faults during assignments.
2. Faults during admin operations.

Thanks
-Nagu

> -----Original Message-----
> From: minh chau [mailto:minh.c...@dektech.com.au]
> Sent: 15 September 2016 12:27
> To: Nagendra Kumar; hans.nordeb...@ericsson.com; Praveen Malviya;
> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> Cc: opensaf-devel@lists.sourceforge.net
> Subject: Re: [devel] [PATCH 2 of 4] AMFND: Admin operation continuation if
> csi completes during headless [#1725 part 1] V1
> 
> Hi Nagu,
> 
> Yes you are right. The "component failover" is unstable, I think I will post 
> my
> analysis of component failover problem to #1902 after you have verified the
> other recoveries.
> 
> Thanks,
> Minh
> 
> On 15/09/16 16:51, Nagendra Kumar wrote:
> > Hi Minh,
> >             @2.a.) and @2.b.) are working except "Component Failover"
> as recovery. Other recovery like SU Failover, etc are working fine with
> 1725_pending_review.tgz and 07_no_recovery_if_no_pending_susi.diff.
> >
> > Please confirm.
> >
> > Thanks
> > -Nagu
> >
> >> -----Original Message-----
> >> From: Nagendra Kumar
> >> Sent: 15 September 2016 12:13
> >> To: minh chau; hans.nordeb...@ericsson.com; Praveen Malviya;
> >> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> >> Cc: opensaf-devel@lists.sourceforge.net
> >> Subject: Re: [devel] [PATCH 2 of 4] AMFND: Admin operation
> >> continuation if csi completes during headless [#1725 part 1] V1
> >>
> >> Hi Minh,
> >>>> If there's no any major problem, can we make SI Dep as last phase?
> >> Yes, absolutely. There is no problem.
> >>>> If I am right, I think you are testing @2.a) - and *fault* has just
> >>>> been as
> >> node reboot/powered-off by user during headless.
> >> Yes, you are right.
> >>
> >> Thanks
> >> -Nagu
> >>
> >>> -----Original Message-----
> >>> From: minh chau [mailto:minh.c...@dektech.com.au]
> >>> Sent: 14 September 2016 17:54
> >>> To: Nagendra Kumar; hans.nordeb...@ericsson.com; Praveen Malviya;
> >>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> >>> Cc: opensaf-devel@lists.sourceforge.net
> >>> Subject: Re: [devel] [PATCH 2 of 4] AMFND: Admin operation
> >>> continuation if csi completes during headless [#1725 part 1] V1
> >>>
> >>> Hi Nagu,
> >>>
> >>> I have proposed to change the order on 28 Jul:
> >>>
> >>> ==============
> >>>
> >>> I would like to change the above orders of implementation:
> >>> @0. We are here now: No admin op continuation, no recovery on faults
> >>> during headless.
> >>> Since componentRestart/suRestart has no impact on recovery after
> >>> headless, faults during headless here mean: failover escalation,
> >>> node reboot/powered-off by user during headless. Faults are
> >>> different phenomenons but they all result in loss of SUSI. Having
> >>> #1902 will remove the major impact of a node reboot due to immediate
> >>> escalation and AMF also has to deal with the loss of SUSI the same
> >>> as without
> >>> #1902 plus failover escalation
> >>>
> >>> @1. Admin op continuation without required recovery on faults during
> >>> headless
> >>> @1.a) All CSI(s) callback completes during headless, but SUSI states
> >>> are still QUIESCED/QUIESCING
> >>> @1.b) One of CSI(s) callback is still ongoing after headless (AMFD
> >>> would have to wait for it?)
> >>>
> >>> @2. Recovery on faults. (Doing fault recovery needs to consider
> >>> admin op continuation which would have been implemented in step @1)
> >>> Need
> >>> #1902
> >>> @2.a.) Faults in normal flow: No admin op continuation is required
> >>> after headless, but fault did happen during headless
> >>> @2.b.) Faults happen during admin operation while headless, after
> >>> headless AMFD needs to consider a recovery on fault together with
> >>> admin op continuation.
> >>>
> >>> @3. @1 + @2 + With SI Dep.
> >>>
> >>> ===============
> >>> I thought we have followed the above order so far? Because part 1
> >>> was acked, which is "@1. Admin op continuation without required
> >>> recovery on faults during headless"
> >>> If there's no any major problem, can we make SI Dep as last phase?
> >>> If I am right, I think you are testing @2.a) - and *fault* has just
> >>> been as node reboot/powered-off by user during headless.
> >>>
> >>> Thanks,
> >>> Minh
> >>>
> >>> On 14/09/16 21:48, Nagendra Kumar wrote:
> >>>> Hi Minh,
> >>>>          If it is not tested, then it is fine. But, we had added (#1) the
> >>> following in the ticket #1725 on 27 Jul :
> >>>> ===========================================
> >>>> Nagendra Kumar - 2016-07-27
> >>>>
> >>>> For 2N red model, implementation can be done in the following
> >>>> phased
> >>> manner.
> >>>> It has advantages of being logically segregated and it continues
> >>>> from
> >>> where we left in 5.0.
> >>>> (Phases #1, #2 and #3 is more related to ticket #1725 and phases #4
> >>>> and #5 are related to #1902)
> >>>>
> >>>> 1.    Node restart escalation (with and without SI Dep).
> >>>> 2.    Without Si Dep : Admin op (no faults/escalations).
> >>>> 3.    Without Si Dep : Admin Op + node restart faults/escalations during
> >>> headless.
> >>>> 4.    Without Si Dep :
> >>>>       a.) All faults in normal flows.
> >>>>       b.) All faults during admin operation(minus node reboot
> >>>> during headless
> >>> as covered in #3).
> >>>>    5.   With Si Dep : #2, #3 and #4.
> >>>>
> >>>> Since 5.0 already has immediate escalation model (component and
> >>>> node
> >>> restart/reboot), so #1, #2 and #3 completes left over portion of
> >>> headless contribution in 5.0 with that model.
> >>>> ======================================
> >>>>
> >>>> Thanks
> >>>> -Nagu
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: minh chau [mailto:minh.c...@dektech.com.au]
> >>>>> Sent: 14 September 2016 17:05
> >>>>> To: Nagendra Kumar; hans.nordeb...@ericsson.com; Praveen
> Malviya;
> >>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> >>>>> Cc: opensaf-devel@lists.sourceforge.net
> >>>>> Subject: Re: [devel] [PATCH 2 of 4] AMFND: Admin operation
> >>>>> continuation if csi completes during headless [#1725 part 1] V1
> >>>>>
> >>>>> Hi Nagu,
> >>>>>
> >>>>> SI Dep is the last phase of implementation of headless recovery,
> >>>>> its support is not included in all patches attached in ticket #1725.
> >>>>>
> >>>>> Thanks,
> >>>>> Minh
> >>>>>
> >>>>> On 14/09/16 21:21, Nagendra Kumar wrote:
> >>>>>> Hi Minh,
> >>>>>>                Have you tested Si Dep (2N Red model) for "node
> restart test
> >>>>> cases" ? I can't see it in the test case doc.
> >>>>>> Thanks
> >>>>>> -Nagu
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Nagendra Kumar
> >>>>>>> Sent: 13 September 2016 11:20
> >>>>>>> To: minh chau; hans.nordeb...@ericsson.com; Praveen Malviya;
> >>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> >>>>>>> Cc: opensaf-devel@lists.sourceforge.net
> >>>>>>> Subject: Re: [devel] [PATCH 2 of 4] AMFND: Admin operation
> >>>>>>> continuation if csi completes during headless [#1725 part 1] V1
> >>>>>>>
> >>>>>>> Hi Minh,
> >>>>>>>               I have tested these scenarios again and it works well.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>> -Nagu
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: minh chau [mailto:minh.c...@dektech.com.au]
> >>>>>>>> Sent: 12 September 2016 11:53
> >>>>>>>> To: Nagendra Kumar; hans.nordeb...@ericsson.com; Praveen
> >>> Malviya;
> >>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
> >>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
> >>>>>>>> Subject: Re: [PATCH 2 of 4] AMFND: Admin operation continuation
> >>>>>>>> if csi completes during headless [#1725 part 1] V1
> >>>>>>>>
> >>>>>>>> Hi Nagu,
> >>>>>>>>
> >>>>>>>> One bug get hit by your configuration, where the absent SUSIs
> >>>>>>>> are found after headless but no real SUSIs are available also.
> >>>>>>>> In this case I think that AMFD can do like a fresh assignment.
> >>>>>>>> I attach the patch to ticket #1725, please help to test again.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Minh
> >>>>>>>>
> >>>>>>>> On 12/09/16 11:09, minh chau wrote:
> >>>>>>>>> Hi Nagu,
> >>>>>>>>>
> >>>>>>>>> I'm running the tests with this configuration and will get
> >>>>>>>>> back to
> >> you.
> >>>>>>>>> Thanks,
> >>>>>>>>> Minh
> >>>>>>>>>
> >>>>>>>>> On 09/09/16 22:26, Nagendra Kumar wrote:
> >>>>>>>>>> Hi Minh,
> >>>>>>>>>> I am using 1725_pending_review.tgz
> >>>>>>>>>> (1725_02_V2_bugfix_01_resend_buffer_in_set_leds.diff,
> >>>>>>>>>> 1725_02_V2_bugfix_02_honor_clusterinit_nodesync_timer.diff,
> >>>>>>>>>> 1725_02_V2_bugfix_03_restore_ng_admin.diff,
> >>>>>>>>>> 1725_03_V4_failover_absent_susi_longDn.diff,
> >>>>>>>>>> 1725_04_V2_headless_validation.diff,
> >>>>>>>>>> 1725_05_V2_resend_oper_state.diff,
> >>>>>>>>>> 1725_06a_fullscope_escalation_headless.diff).
> >>>>>>>>>>
> >>>>>>>>>> I am doing basic node reboot validation testing with no faults.
> >>>>>>>>>>
> >>>>>>>>>> Configuration: SU1(act) and SU2(stanby) both on PL-3.
> >>>>>>>>>>
> >>>>>>>>>> TC #1: Start SC-1, PL-3 and PL-5: Unlock SU1 and SU2. Stop
> >>>>>>>>>> SC-1 and stop PL-3, start PL-3 and start SC-1.
> >>>>>>>>>> After SC-1 and PL-3 comes back, ideally SU1 and SU2 should
> >>>>>>>>>> get assignments as Act and Std, but no assignment are being
> >>>>>>>>>> given to SUs on PL-3 and it shows following in status:
> >>>>>>>>>>
> >>>>>>>>>> Only Su2 has Std assignment.
> >>>>>>>>>>
> >>>>>>>>>> safSISU=safSu=SC-
> >>>>>>>> 1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed1,safApp=O
> >>>>>>>>>> penSAF
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>> safSISU=safSu=PL-
> >>>>>>>> 5\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=O
> >>>>>>>>>> penSAF
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>>
> >>
> safSISU=safSu=SU2\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
> >>>>>>>> mo1,s
> >>>>>>>>>> afApp=AmfDemo1
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=STANDBY(2)
> >>>>>>>>>> safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-
> >>>>>>>> 2N,safApp=OpenSAF
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>> safSISU=safSu=PL-
> >>>>>>>> 3\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=O
> >>>>>>>>>> penSAF
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>>
> >>>>>>>>>> TC #2: Configuration same as TC#1. Stop PL-3 and don't start.
> >>>>>>>>>> The same issue:
> >>>>>>>>>> safSISU=safSu=PL-
> >>>>>>>> 5\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=O
> >>>>>>>>>> penSAF
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>>
> >>
> safSISU=safSu=SU2\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
> >>>>>>>> mo1,s
> >>>>>>>>>> afApp=AmfDemo1
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=STANDBY(2)
> >>>>>>>>>> safSISU=safSu=SC-
> >>>>>>>> 1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=O
> >>>>>>>>>> penSAF
> >>>>>>>>>>
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>> safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-
> >>>>>>>> 2N,safApp=OpenSAF
> >>>>>>>>>>             saAmfSISUHAState=ACTIVE(1)
> >>>>>>>>>>
> >>>>>>>>>> TC #3: Configured SU1(Act) on PL-3 and SU2(Std) on PL-4.
> >>>>>>>>>> Stop SC-1, stop PL-3 and PL-4, but PL-5 is running. start
> >>>>>>>>>> SC-1, the same issue.
> >>>>>>>>>>
> >>>>>>>>>> TC #4: Same as TC #3, but SU3 configured on PL-5 as spare.
> >>>>>>>>>> SU3 doesn't get any assignment and Sg is unstable.
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>> -Nagu
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Minh Hon Chau [mailto:minh.c...@dektech.com.au]
> >>>>>>>>>>> Sent: 18 August 2016 05:46
> >>>>>>>>>>> To: hans.nordeb...@ericsson.com; Nagendra Kumar; Praveen
> >>>>>>> Malviya;
> >>>>>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au;
> >>>>>>>>>>> minh.c...@dektech.com.au
> >>>>>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
> >>>>>>>>>>> Subject: [PATCH 2 of 4] AMFND: Admin operation continuation
> >>>>>>>>>>> if csi completes during headless [#1725 part 1] V1
> >>>>>>>>>>>
> >>>>>>>>>>>      osaf/services/saf/amf/amfnd/di.cc             |  199
> >>>>>>>>>>> +++++++++++++++++--------
> >>>>>>>>>>>      osaf/services/saf/amf/amfnd/include/avnd_di.h |    1 +
> >>>>>>>>>>>      2 files changed, 134 insertions(+), 66 deletions(-)
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> There're two options basically that AMFD can continue admin
> >>>>>>>>>>> operation wih completed csi(s)
> >>>>>>>>>>>
> >>>>>>>>>>> First: AMFD can use the sync SUSI fsm state as latest, AMFD
> >>>>>>>>>>> then has to explore its SUSI assignments with adminStates of
> >>>>>>>>>>> relevant entities to determine which SU should be on call of
> >>> susi_success().
> >>>>>>>>>>> Deeper level of exploration for csi addition. It also
> >>>>>>>>>>> depends on SG Fsm state which is being used variously in
> >>>>>>>>>>> different SG
> >> types.
> >>>>>>>>>>> Second: AMFD uses the SUSI fsm state read from IMM as
> >>>>>>>>>>> latest, and AMFND needs to resend susi_resp messages which
> >>>>>>>>>>> were deferred during headless so that AMFD can continue the
> >>>>>>>>>>> admin operation
> >>>>>>> sequence.
> >>>>>>>>>>> Both cases of csi completion [during or after] headless can
> >>>>>>>>>>> run in the same code flow.
> >>>>>>>>>>>
> >>>>>>>>>>> The patch buffers susi_resp_msg during headless stage and
> >>>>>>>>>>> resend it to AMFD after headless. There could be a chance
> >>>>>>>>>>> that AMFND sent out susi response message but AMFD could
> not
> >>> receive
> >>>>>>>>>>> or process it. This case could be seen as a defect, which
> >>>>>>>>>>> can be fixed by securing the result of sending susi_resp
> >>>>>>>>>>> message from AMFND toward
> >>>>>>> AMFD.
> >>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfnd/di.cc
> >>>>>>>>>>> b/osaf/services/saf/amf/amfnd/di.cc
> >>>>>>>>>>> --- a/osaf/services/saf/amf/amfnd/di.cc
> >>>>>>>>>>> +++ b/osaf/services/saf/amf/amfnd/di.cc
> >>>>>>>>>>> @@ -805,11 +805,6 @@ uint32_t
> >>> avnd_di_susi_resp_send(AVND_CB
> >>>>>>>>>>>          if (cb->term_state ==
> >>>>>>>>>>> AVND_TERM_STATE_OPENSAF_SHUTDOWN_STARTED)
> >>>>>>>>>>>              return rc;
> >>>>>>>>>>>
> >>>>>>>>>>> -    if (cb->is_avd_down == true) {
> >>>>>>>>>>> -        m_AVND_SU_ALL_SI_RESET(su);
> >>>>>>>>>>> -        return rc;
> >>>>>>>>>>> -    }
> >>>>>>>>>>> -
> >>>>>>>>>>>          // should be in assignment pending state to be here
> >>>>>>>>>>>          osafassert(m_AVND_SU_IS_ASSIGN_PEND(su));
> >>>>>>>>>>>
> >>>>>>>>>>> @@ -820,64 +815,76 @@ uint32_t
> >>>>> avnd_di_susi_resp_send(AVND_CB
> >>>>>>>>>>>          TRACE_ENTER2("Sending Resp su=%s, si=%s,
> >>>>>>>>>>> curr_state=%u, prv_state=%u", su->name.value,
> >>>>>>>>>>> curr_si->name.value,curr_si-
> >>>>>>>>>>>> curr_state,curr_si->prv_state);
> >>>>>>>>>>>          /* populate the susi resp msg */
> >>>>>>>>>>>          msg.info.avd = new AVSV_DND_MSG();
> >>>>>>>>>>> -        msg.type = AVND_MSG_AVD;
> >>>>>>>>>>> -        msg.info.avd->msg_type =
> >>>>>>> AVSV_N2D_INFO_SU_SI_ASSIGN_MSG;
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.msg_id =
> ++(cb-
> >>>>>>>>>>>> snd_msg_id);
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.node_id = cb-
> >>>>>>>>>>>> node_info.nodeId;
> >>>>>>>>>>> -        if (si) {
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.single_csi =
> >>>>>>>>>>> -                        ((si->single_csi_add_rem_in_si ==
> >>>>>>>>>>> AVSV_SUSI_ACT_BASE) ?
> >>>>>>>>>>> false : true);
> >>>>>>>>>>> -        }
> >>>>>>>>>>> -        TRACE("curr_assign_state '%u'", curr_si-
> >>> curr_assign_state);
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
> >>>>>>>>>>> -
> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> >> ||
> >>>>>>>>>>> -
> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si))
> >> ?
> >>>>>>>>>>> -                ((!curr_si->prv_state) ? AVSV_SUSI_ACT_ASGN :
> >>>>>>>>>>> AVSV_SUSI_ACT_MOD) : AVSV_SUSI_ACT_DEL;
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.su_name = su-
> >>>>>> name;
> >>>>>>>>>>> -        if (si) {
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.si_name = si-
> >name;
> >>>>>>>>>>> -                if (AVSV_SUSI_ACT_ASGN ==
> >>>>>>>>>>> si->single_csi_add_rem_in_si) {
> >>>>>>>>>>> -                        TRACE("si->curr_assign_state '%u'", 
> >>>>>>>>>>> curr_si-
> >>>>>>>>>>>> curr_assign_state);
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
> >>>>>>>>>>> -
> >>>>>>>>>>> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> ||
> >>>>>>>>>>> -
> >>>>>>>>>>>
> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si)) ?
> >>>>>>>>>>> -                                AVSV_SUSI_ACT_ASGN :
> >>>>>>>>>>> AVSV_SUSI_ACT_DEL;
> >>>>>>>>>>> -                }
> >>>>>>>>>>> -        }
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.ha_state =
> >>>>>>>>>>> -                (SA_AMF_HA_QUIESCING == curr_si->curr_state) ?
> >>>>>>>>>>> SA_AMF_HA_QUIESCED : curr_si->curr_state;
> >>>>>>>>>>> -        msg.info.avd->msg_info.n2d_su_si_assign.error =
> >>>>>>>>>>> -
> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> >> ||
> >>>>>>>>>>> -
> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_REMOVED(curr_si))
> >> ?
> >>>>>>>>>>> NCSCC_RC_SUCCESS : NCSCC_RC_FAILURE;
> >>>>>>>>>>> +    msg.type = AVND_MSG_AVD;
> >>>>>>>>>>> +    msg.info.avd->msg_type =
> >>>>> AVSV_N2D_INFO_SU_SI_ASSIGN_MSG;
> >>>>>>>>>>> +    msg.info.avd->msg_info.n2d_su_si_assign.node_id = cb-
> >>>>>>>>>>>> node_info.nodeId;
> >>>>>>>>>>> +    if (si) {
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.single_csi =
> >>>>>>>>>>> +                ((si->single_csi_add_rem_in_si ==
> >>>>>>>>>>> AVSV_SUSI_ACT_BASE) ? false : true);
> >>>>>>>>>>> +    }
> >>>>>>>>>>> +    TRACE("curr_assign_state '%u'", curr_si-
> >curr_assign_state);
> >>>>>>>>>>> +    msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> >>> ||
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si))
> >>> ?
> >>>>>>>>>>> +                ((!curr_si->prv_state) ?
> >>>>>>>>>>> AVSV_SUSI_ACT_ASGN : AVSV_SUSI_ACT_MOD) :
> >>>>>>> AVSV_SUSI_ACT_DEL;
> >>>>>>>>>>> +    msg.info.avd->msg_info.n2d_su_si_assign.su_name = su-
> >>>> name;
> >>>>>>>>>>> +    if (si) {
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.si_name =
> >>>>>>>>>>> + si-
> >>>>>>>>>>>> name;
> >>>>>>>>>>> +        if (AVSV_SUSI_ACT_ASGN ==
> >>>>>>>>>>> + si->single_csi_add_rem_in_si)
> >> {
> >>>>>>>>>>> +            TRACE("si->curr_assign_state '%u'", curr_si-
> >>>>>>>>>>>> curr_assign_state);
> >>>>>>>>>>> +                msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.msg_act =
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> >>> ||
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si))
> >>> ?
> >>>>>>>>>>> +                    AVSV_SUSI_ACT_ASGN :
> >>>>>>>>>>> AVSV_SUSI_ACT_DEL;
> >>>>>>>>>>> +        }
> >>>>>>>>>>> +    }
> >>>>>>>>>>> +    msg.info.avd->msg_info.n2d_su_si_assign.ha_state =
> >>>>>>>>>>> +            (SA_AMF_HA_QUIESCING == curr_si->curr_state) ?
> >>>>>>>>>>> SA_AMF_HA_QUIESCED : curr_si->curr_state;
> >>>>>>>>>>> +    msg.info.avd->msg_info.n2d_su_si_assign.error =
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
> >>> ||
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_REMOVED(curr_si))
> >>> ?
> >>>>>>>>>>> +NCSCC_RC_SUCCESS : NCSCC_RC_FAILURE;
> >>>>>>>>>>>
> >>>>>>>>>>> -        if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
> >>>>>>>>>>> AVSV_SUSI_ACT_ASGN)
> >>>>>>>>>>> -                osafassert(si);
> >>>>>>>>>>> +    if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
> >>>>>>>>>>> AVSV_SUSI_ACT_ASGN)
> >>>>>>>>>>> +        osafassert(si);
> >>>>>>>>>>>
> >>>>>>>>>>> -        /* send the msg to AvD */
> >>>>>>>>>>> -        TRACE("Sending. msg_id'%u', node_id'%u', msg_act'%u',
> >>>>>>>>>>> su'%s', si'%s',
> >>>>>>>>>>> ha_state'%u', error'%u', single_csi'%u'",
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_id,
> >>>>>>>>>>> msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.node_id,
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_act,
> >>>>>>>>>>> msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.su_name.value,
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.si_name.value,
> >>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.ha_state,
> >>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.error,
> >>>>>>>>>>> msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.single_csi);
> >>>>>>>>>>> +    /* send the msg to AvD */
> >>>>>>>>>>> +    TRACE("Sending. msg_id'%u', node_id'%u', msg_act'%u',
> >>>>>>>>>>> + su'%s',
> >>>>>>>>>>> si'%s', ha_state'%u', error'%u', single_csi'%u'",
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.msg_id,
> >>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.node_id,
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.msg_act,
> >>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.su_name.value,
> >>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.si_name.value,
> >>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.ha_state,
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.error,
> >>>>>>>>>>> +msg.info.avd->msg_info.n2d_su_si_assign.single_csi);
> >>>>>>>>>>>
> >>>>>>>>>>> -        if ((su->si_list.n_nodes > 1) && (si == nullptr)) {
> >>>>>>>>>>> -                if (msg.info.avd-
> >msg_info.n2d_su_si_assign.msg_act
> >> ==
> >>>>>>>>>>> AVSV_SUSI_ACT_DEL)
> >>>>>>>>>>> -                        LOG_NO("Removed 'all SIs' from '%s'",
> >>>>>>>>>>> su->name.value);
> >>>>>>>>>>> +    if ((su->si_list.n_nodes > 1) && (si == nullptr)) {
> >>>>>>>>>>> +        if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act
> >>>>>>>>>>> + ==
> >>>>>>>>>>> AVSV_SUSI_ACT_DEL)
> >>>>>>>>>>> +            LOG_NO("Removed 'all SIs' from '%s'", su-
> >>>>>>>>>>>> name.value);
> >>>>>>>>>>> -                if (msg.info.avd-
> >msg_info.n2d_su_si_assign.msg_act
> >> ==
> >>>>>>>>>>> AVSV_SUSI_ACT_MOD)
> >>>>>>>>>>> -                        LOG_NO("Assigned 'all SIs' %s of '%s'",
> >>>>>>>>>>> -                               ha_state[msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.ha_state],
> >>>>>>>>>>> -                               su->name.value);
> >>>>>>>>>>> -        }
> >>>>>>>>>>> +        if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act
> >>>>>>>>>>> + ==
> >>>>>>>>>>> AVSV_SUSI_ACT_MOD)
> >>>>>>>>>>> +            LOG_NO("Assigned 'all SIs' %s of '%s'",
> >>>>>>>>>>> +                    ha_state[msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.ha_state],
> >>>>>>>>>>> +                    su->name.value);
> >>>>>>>>>>> +    }
> >>>>>>>>>>>
> >>>>>>>>>>> -        rc = avnd_di_msg_send(cb, &msg);
> >>>>>>>>>>> -        if (NCSCC_RC_SUCCESS == rc)
> >>>>>>>>>>> -                msg.info.avd = 0;
> >>>>>>>>>>> -
> >>>>>>>>>>> -        /* we have completed the SU SI msg processing */
> >>>>>>>>>>> -        if (su_assign_state_is_stable(su))
> >>>>>>>>>>> -                m_AVND_SU_ASSIGN_PEND_RESET(su);
> >>>>>>>>>>> -        m_AVND_SU_ALL_SI_RESET(su);
> >>>>>>>>>>> +    if (cb->is_avd_down == true) {
> >>>>>>>>>>> +        // We are in headless, buffer this msg
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.msg_id = 0;
> >>>>>>>>>>> +        if (avnd_diq_rec_add(cb, &msg) == nullptr) {
> >>>>>>>>>>> +            rc = NCSCC_RC_FAILURE;
> >>>>>>>>>>> +        }
> >>>>>>>>>>> +        m_AVND_SU_ALL_SI_RESET(su);
> >>>>>>>>>>> +        LOG_NO("avnd_di_susi_resp_send() deferred as AMF
> >>>>>>>>>>> director is offline");
> >>>>>>>>>>> +    } else {
> >>>>>>>>>>> +        // We are in normal cluster, send msg to director
> >>>>>>>>>>> +        msg.info.avd->msg_info.n2d_su_si_assign.msg_id =
> >>>>>>>>>>> + ++(cb-
> >>>>>>>>>>>> snd_msg_id);
> >>>>>>>>>>> +        /* send the msg to AvD */
> >>>>>>>>>>> +        rc = avnd_di_msg_send(cb, &msg);
> >>>>>>>>>>> +        if (NCSCC_RC_SUCCESS == rc)
> >>>>>>>>>>> +            msg.info.avd = 0;
> >>>>>>>>>>> +        /* we have completed the SU SI msg processing */
> >>>>>>>>>>> +        if (su_assign_state_is_stable(su)) {
> >>>>>>>>>>> +            m_AVND_SU_ASSIGN_PEND_RESET(su);
> >>>>>>>>>>> +        }
> >>>>>>>>>>> +        m_AVND_SU_ALL_SI_RESET(su);
> >>>>>>>>>>> +    }
> >>>>>>>>>>>
> >>>>>>>>>>>          /* free the contents of avnd message */
> >>>>>>>>>>>          avnd_msg_content_free(cb, &msg); @@ -1256,14
> >>>>>>>>>>> +1263,7
> >>> @@
> >>>>>>> void
> >>>>>>>>>>> avnd_diq_rec_del(AVND_CB *cb, AVND_
> >>>>>>>>>>>          /* stop the AvD msg response timer */
> >>>>>>>>>>>          if (m_AVND_TMR_IS_ACTIVE(rec->resp_tmr)) {
> >>>>>>>>>>>              m_AVND_TMR_MSG_RESP_STOP(cb, *rec);
> >>>>>>>>>>> -        // Resend msgs from queue because amfd dropped during
> >>>>>>>>>>> sync
> >>>>>>>>>>> -        if ((cb->dnd_list.head != nullptr)) {
> >>>>>>>>>>> -            TRACE("retransmit message to amfd");
> >>>>>>>>>>> -            AVND_DND_MSG_LIST *pending_rec = 0;
> >>>>>>>>>>> -            for (pending_rec = cb->dnd_list.head; pending_rec !=
> >>>>>>>>>>> nullptr; pending_rec = pending_rec->next) {
> >>>>>>>>>>> -                avnd_diq_rec_send(cb, pending_rec);
> >>>>>>>>>>> -            }
> >>>>>>>>>>> -        }
> >>>>>>>>>>> +        avnd_diq_rec_send_buffered_msg(cb);
> >>>>>>>>>>>              /* resend pg start track */
> >>>>>>>>>>>              avnd_di_resend_pg_start_track(cb);
> >>>>>>>>>>>          }
> >>>>>>>>>>> @@ -1276,6 +1276,73 @@ void avnd_diq_rec_del(AVND_CB
> >> *cb,
> >>>>>>>> AVND_
> >>>>>>>>>>>          TRACE_LEAVE();
> >>>>>>>>>>>          return;
> >>>>>>>>>>>      }
> >>>>>>>>>>>
> >>
> +/************************************************************
> >>>>>>>>>>> ****************
> >>>>>>>>>>> +  Name          : avnd_diq_rec_send_buffered_msg
> >>>>>>>>>>> +
> >>>>>>>>>>> +  Description   : Resend buffered msg
> >>>>>>>>>>> +
> >>>>>>>>>>> +  Arguments     : cb  - ptr to the AvND control block
> >>>>>>>>>>> +
> >>>>>>>>>>> +  Return Values : None.
> >>>>>>>>>>> +
> >>>>>>>>>>> +  Notes         : None.
> >>>>>>>>>>>
> >>
> +*************************************************************
> >>>>>>>>>>> **********
> >>>>>>>>>>> +*******/ void avnd_diq_rec_send_buffered_msg(AVND_CB
> >> *cb)
> >>> {
> >>>>>>>>>>> +    TRACE_ENTER();
> >>>>>>>>>>> +    // Resend msgs from queue because amfnd dropped during
> >>>>>>> headless
> >>>>>>>>>>> +    // or headless-synchronization
> >>>>>>>>>>> +    if ((cb->dnd_list.head != nullptr)) {
> >>>>>>>>>>> +        AVND_DND_MSG_LIST *pending_rec = 0;
> >>>>>>>>>>> +        TRACE("Attach msg_id of buffered msg");
> >>>>>>>>>>> +        bool found = true;
> >>>>>>>>>>> +        while (found) {
> >>>>>>>>>>> +            found = false;
> >>>>>>>>>>> +            for (pending_rec = cb->dnd_list.head;
> >>>>>>>>>>> + pending_rec !=
> >>>>>>>>>>> nullptr; pending_rec = pending_rec->next) {
> >>>>>>>>>>> +                if (pending_rec->msg.type ==
> >>>>>>>>>>> AVND_MSG_AVD) {
> >>>>>>>>>>> +                    // At this moment, only oper_state
> >>>>>>>>>>> msg needs to report to director
> >>>>>>>>>>> +                    if (pending_rec->msg.info.avd-
> >>>>>>>>>>>> msg_type == AVSV_N2D_INFO_SU_SI_ASSIGN_MSG &&
> >>>>>>>>>>> +                        pending_rec->msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.msg_id == 0) {
> >>>>>>>>>>> +                        m_AVND_DIQ_REC_POP(cb,
> >>>>>>>>>>> pending_rec); #if 0
> >>>>>>>>>>> +                        // only resend if this SUSI
> >>>>>>>>>>> does exist
> >>>>>>>>>>> +                        AVND_SU *su =
> >>>>>>>>>>> m_AVND_SUDB_REC_GET(cb->sudb,
> >>>>>>>>>>> +                                pending_rec-
> >>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.su_name);
> >>>>>>>>>>> +                        if (su != nullptr && su-
> >>>>>>>>>>>> si_list.n_nodes > 0) { #endif
> >>>>>>>>>>> +                            pending_rec-
> >>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.msg_id =
> >>>>>>>>>>>> ++(cb->snd_msg_id);
> >>>>>>>>>>> +
> >>>>>>>>>>>        m_AVND_DIQ_REC_PUSH(cb, pending_rec);
> >>>>>>>>>>> +                            LOG_NO("Found and
> >>>>>>>>>>> resend buffered su_si_assign msg for SU:'%s', "
> >>>>>>>>>>> +
> >>>>>>>>>>>        "SI:'%s', ha_state:'%u', msg_act:'%u', single_csi:'%u', "
> >>>>>>>>>>> +
> >>>>>>>>>>>        "error:'%u', msg_id:'%u'",
> >>>>>>>>>>> +
> >>>>>>>>>>>        pending_rec->msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.su_name.value,
> >>>>>>>>>>> +
> >>>>>>>>>>>        pending_rec->msg.info.avd-
> >>>>>>>>>>>> msg_info.n2d_su_si_assign.si_name.value,
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >>>>>>>>>>> pending_rec->msg.info.avd-
> >>>> msg_info.n2d_su_si_assign.ha_state,
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >>>>>>>>>>> pending_rec->msg.info.avd-
> >>> msg_info.n2d_su_si_assign.msg_act,
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >>>>>>>>>>> pending_rec->msg.info.avd-
> >>>> msg_info.n2d_su_si_assign.single_csi
> >>>>>>>>>>> ,
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >>>>>>>>>>> pending_rec->msg.info.avd-
> >msg_info.n2d_su_si_assign.error,
> >>>>>>>>>>> +
> >>>>>>>>>>>
> >>>>>>>>>>> pending_rec->msg.info.avd-
> >>> msg_info.n2d_su_si_assign.msg_id);
> >>>>>>>>>>> +
> >>>>>>>>>>> +#if 0
> >>>>>>>>>>> +                        } else {
> >>>>>>>>>>> +
> >>>>>>>>>>>        avnd_msg_content_free(cb, &pending_rec->msg);
> >>>>>>>>>>> +                            delete pending_rec;
> >>>>>>>>>>> +                            pending_rec = cb-
> >>>>>>>>>>>> dnd_list.head;
> >>>>>>>>>>> +                        }
> >>>>>>>>>>> +#endif
> >>>>>>>>>>> +                        found = true;
> >>>>>>>>>>> +                    }
> >>>>>>>>>>> +                }
> >>>>>>>>>>> +            }
> >>>>>>>>>>> +        }
> >>>>>>>>>>> +        TRACE("retransmit message to amfd");
> >>>>>>>>>>> +         for (pending_rec = cb->dnd_list.head; pending_rec
> >>>>>>>>>>> +!= nullptr;
> >>>>>>>>>>> pending_rec = pending_rec->next) {
> >>>>>>>>>>> +             avnd_diq_rec_send(cb, pending_rec);
> >>>>>>>>>>> +         }
> >>>>>>>>>>> +    }
> >>>>>>>>>>> +    TRACE_LEAVE();
> >>>>>>>>>>> +    return;
> >>>>>>>>>>> +}
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>
> /*************************************************************
> >>>>>>>>>>> ***************
> >>>>>>>>>>>        Name          : avnd_diq_rec_send
> >>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfnd/include/avnd_di.h
> >>>>>>>>>>> b/osaf/services/saf/amf/amfnd/include/avnd_di.h
> >>>>>>>>>>> --- a/osaf/services/saf/amf/amfnd/include/avnd_di.h
> >>>>>>>>>>> +++ b/osaf/services/saf/amf/amfnd/include/avnd_di.h
> >>>>>>>>>>> @@ -79,6 +79,7 @@ void avnd_di_msg_ack_process(struct
> >> avnd
> >>>>> void
> >>>>>>>>>>> avnd_diq_del(struct avnd_cb_tag *);  AVND_DND_MSG_LIST
> >>>>>>>>>>> *avnd_diq_rec_add(struct avnd_cb_tag *cb, AVND_MSG
> *msg);
> >>> void
> >>>>>>>>>>> avnd_diq_rec_del(struct avnd_cb_tag *cb,
> >> AVND_DND_MSG_LIST
> >>>>>>> *rec);
> >>>>>>>>>>> +void avnd_diq_rec_send_buffered_msg(struct avnd_cb_tag
> >> *cb);
> >>>>>>>>>>>      uint32_t avnd_diq_rec_send(struct avnd_cb_tag *cb,
> >>>>>>>>>>> AVND_DND_MSG_LIST *rec);  uint32_t
> >>>>> avnd_di_reg_su_rsp_snd(struct
> >>>>>>>>>>> avnd_cb_tag *cb, SaNameT *su_name, uint32_t ret_code);
> >>>>>>>>>>> uint32_t avnd_di_ack_nack_msg_send(struct avnd_cb_tag
> *cb,
> >>>>>>>>>>> uint32_t rcv_id, uint32_t view_num);
> >>>>>>> ----------------------------------------------------------------
> >>>>>>> -
> >>>>>>> --
> >>>>>>> --
> >>>>>>> --------- _______________________________________________
> >>>>>>> Opensaf-devel mailing list
> >>>>>>> Opensaf-devel@lists.sourceforge.net
> >>>>>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
> >> ---------------------------------------------------------------------
> >> --------- _______________________________________________
> >> Opensaf-devel mailing list
> >> Opensaf-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
> 

------------------------------------------------------------------------------
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel

Reply via email to