Hi Praveen,
I think in all cases we need to restore @admin_ng, that can tell whether
an admin operation on ng was executing. Also, we need to restore
ng_using_saAmfSGAdminState in case 2N borrows SaAmfSGAdminState for
nodegroup. If we can restore exactly what was happening before headless,
then the operation continuation should work. However, there's still the
cases like:
case 1: Create nodegroup (PL4 + PL5) with LOCKED, lock PL5, lock PL4,
delay quiesced csi cbk, stop SC, restart SC.
case 2: Create nodegroup (PL4 + PL5) with LOCKED, lock nodegroup, delay
quiesced csi cbk, stop SC, restart SC.
case 3: Create nodegroup (PL4 + PL5) with LOCKED, lock PL5, lock
nodegroup, delay quiesced csi cbk, stop SC, restart SC.
Case 1 and 2 has been cleared out previously, case 1 and 3 leave all
states the same: SG_SU_OPER, node: LOCKED, ng: LOCKED. But if AMF views
case 3 in the shape of case 1, then nodegroup operation continuation
also works fine. I have sent out the patch for reivew, please check.
Thanks,
Minh
On 01/09/16 18:11, praveen malviya wrote:
> Hi Minh,
> Please see response inline with [Praveen]
>
> Thanks,
> Praveen
>
>
> On 31-Aug-16 11:19 AM, minh chau wrote:
>> Hi Praveen
>>
>> I think @admin_ng needs to be restore as well as
>> @ng_using_saAmfSGAdminState, since just found that @admin_ng check is
>> required in avd_node_down_appl_susi_failover(), which happens if node
>> having pending csi callback reboot.
> [Praveen] Inside avd_node_down_appl_susi_failover(), reason of calling
> process_su_si_response_for_ng() in non-headless case is :
> -If this happens to be the last node in the nodegroup where node group
> operation was going on, then migrate node/NG from SHUTTING_DOWN to
> locked,clear ng_using_saAmfSGAdminState and respond to IMM because we
> will not get any su_si_assign() event to trigger it agian.
> -Even if this is not the last node, then also this nodes should move
> from SHUTTING_DOWN to locked state again because there will not be any
> further trigger (when whole SG is mapped in NG).
>
> Can we think of a possibility of making it OR with
> @ng_using_saAmfSGAdminState here also in
> avd_node_down_appl_susi_failover() ?
>
> A)With this small fix in headless case: After headless if again some
> node faults then AMFD will still be calling
> process_su_si_response_for_ng() for whole SG mapped case.
>
> But what about other case when whole SG is not mapped.In this case
> node_fail_su_oper() would have marked atleast node from SHUTTING_DOWN
> to LOCKED state. Then how to mark NG from SHUTTING_DOWN to LOCKED
> because admin_ng is NULL and ng_using_saAMFSGAdminState is not set for
> this case. I think this can be done deductively using headless state
> vairable and other facts like node->admin_node_pend_cbk.invocation and
> ng->admin_ng_pend_cbk.admin_oper after headless to cross verify
> whether this is really the context of admin operation before headless
> or after headless. Based on this deduction, we need to mark node from
> SHUTING_DON to LOCKED.
>
> b)With this small fix in non-headless case: There should not be any
> impact because admin_ng is not NULL and AMFD was already making a call.
>
>
> Thanks,
> Praveen
>> I will use SG_FSM_ADMIN to differentiate cases 1 and 2.
>>
>> Thanks,
>> Minh
>> On 29/08/16 16:25, praveen malviya wrote:
>>> Hi Minh,
>>>
>>> Please see inline with [Praveen]
>>>
>>> Thanks,
>>> Praveen
>>>
>>> On 29-Aug-16 5:57 AM, minh chau wrote:
>>>> Hi Praveen,
>>>>
>>>> Thanks for looking through the patch.
>>>> The potential problem of restoring nodegroup because nodegroup
>>>> allows to
>>>> be created in LOCKED while the SUs are having assignment, this could
>>>> cause an ambiguity for AMFD after headless. For example:
>>>> Suppose having SU4 hosted on PL4, SU5 hosted on PL5, SU4 has active
>>>> assignment, SU5 has standby assignment.
>>>> case 1: Create nodegroup (PL4 + PL5) with LOCKED, lock PL5, lock PL4,
>>>> delay quiesced csi cbk, stop SC, restart SC.
>>> [Praveen] In this case, after headless state SG fsm will not be in
>>> SG_ADMIN state because payload are being locked one by one. So in this
>>> case it is distinguishable that it is a not a NG operation case
>>> as SG is not in SG_ADMIN state even though SG is fully assigned in NG.
>>>> case 2: Create nodegroup (PL4 + PL5) with LOCKED, lock nodegroup,
>>>> delay
>>>> quiesced csi cbk, stop SC, restart SC.
>>> [Praveen] In this case we have following information after headless
>>> state:
>>> -SG is in SG_ADMIN state.
>>> -NG is in SHUTTING_DOWN or LOCKED state.
>>> -Nodes in SHUTTING_DOWN or LOCKED state.
>>> -SG FSM remains in SG_ADMIN state only in case of admin operation
>>> on SG. But after headless SG is not found in UNLOCKED state and one NG
>>> is found in LOCKED/SHUTTING down state and its nodes.
>>> I think with above information, AMFD can set
>>> @ng_using_saAmfSGAdminState and set SG admin state to SHUTTING_DOWN or
>>> LOCKED. Restoring admin_ng is not required as in su_si_assign(), there
>>> is an OR condition between @ng_using_saAmfSGAdminState and @admin_ng
>>> for calling process_su_si_response_for_ng(). Also checks on admin_ng
>>> is used only for updating counters related to completion of admin
>>> opeations which is not required after headless.
>>>
>>>>
>>>> if case 2 actually happened before headless, then @admin_ng and
>>>> @ng_using_saAmfSGAdminState needs to be restored, otherwise
>>>> process_su_si_response_for_ng() won't be called and saAmfSGAdminState
>>>> remains LOCKED and SG is still not STABLE state.
>>>>
>>>> But in both cases, after headless, AMFD sees all PLs are LOCKED,
>>>> nodegroup is LOCKED, SU4 has pending quiesced csi cbk, thus they are
>>>> running into the same code flow. In case 1, @admin_ng and
>>>> @ng_using_saAmfSGAdminState should not bet set since case1 was not
>>>> nodegroup operation before headless.
>>>> I have run a test of both cases, they are working with the patch
>>>> attached in ticket, but it still looks a potential problem since all
>>>> cases are not transparent to AMFD after headless, the @admin_ng and
>>>> @ng_using_asAmfSGAdminState maybe get hit in some points in case 1
>>> [Praveen] Besides above cases, there remains only one case: when
>>> operation was initiated on NG and SG is partially mapped in NG. In
>>> this case, after headless state we can get only two states of SG
>>> either SU_OPER or SG_REALIGN. In both the cases I think we do not
>>> require to restore @ng_using_saAmfSGAdminState and @admin_ng because
>>> we do not require to enter in process_su_si_response_for_ng().In
>>> sg_2n_fsm, it marks Node from SHUTTING_DOWN to locked in
>>> susi_success_su_oper().
>>>
>>> Have I missed any other case?
>>>
>>>>
>>>> If case 1 looks ok to you from nodegroup point of view, then I will
>>>> float the patch for review.
>>>>
>>>> Thanks,
>>>> Minh
>>>>
>>>>
>>>> On 26/08/16 16:08, praveen malviya wrote:
>>>>> Hi,
>>>>>
>>>>> I have gone through amfd traces. Also patch for NG seems to be ok but
>>>>> some minor can be done.
>>>>>
>>>>> As pointed by Minh, when whole SG is mapped in NG (say case a), AMFD
>>>>> uses SG_ADMIN flow and SG admin state without exposing it to the user
>>>>> through IMM for 2N model. In the other case when only one SU is
>>>>> assigned in NG (say case b) there should not be any problem because
>>>>> operation fully depends on NG admin state. Since other case b) does
>>>>> not use SG admin state and ng_using_saAMfSGAdminState, it should work
>>>>> fine.
>>>>>
>>>>> I think we can take the help of following facts and functions to
>>>>> improve the patch and with that restoring ng_using_saAmfSGAdminState
>>>>> from IMM may not be required:
>>>>> 1)In normal cluster, if controller switchover/failover happens
>>>>> when NG
>>>>> operation is going on then standby controller continues admin
>>>>> operation with information that it gets through CKPT updates in
>>>>> dec_sg_admin_state() and dec_ng_admin_state(). Active controller
>>>>> never
>>>>> checkpoints ng_using_saAmfSGAdminState and deduce it in these
>>>>> functions.The situation after headless is almost like that.
>>>>> I think, in case a when shutdown operation is going on, admin
>>>>> state of NG is still SHUTTING_DOWN and system becomes headless,
>>>>> requires more params and not the lock operation. In shutdown
>>>>> operation, AMFD has to ensure transition of NG and Nodes to LOCKED
>>>>> state.
>>>>>
>>>>> 2)Like controller fail-over/switch-over after headless also, we are
>>>>> not bound to reply to IMM for admin operation completion. So we need
>>>>> to analyse if we require to restore node->admin_ng. Half of the code
>>>>> in process_su_si_response_for_ng() is for tracking the state of admin
>>>>> operation so that AMFD replies to IMM for admin operation and this is
>>>>> not required after headless state.
>>>>>
>>>>> I think problem is not that much complex as it is valid for only 2N
>>>>> models and only in case a).
>>>>>
>>>>> Thanks,
>>>>> Praveen
>>>>>
>>>>> On 25-Aug-16 6:38 PM, minh chau wrote:
>>>>>> Hi,
>>>>>>
>>>>>> The test failed because two reasons:
>>>>>> 1. There are two places that nodegroup operation borrows 2N SG FSM,
>>>>>> but
>>>>>> the AdminState of SG is not stored to IMM
>>>>>> saAmfSGAdminState = ng->saAmfNGAdminState;
>>>>>> ...
>>>>>> su->sg_of_su->saAmfSGAdminState = SA_AMF_ADMIN_UNLOCKED;
>>>>>>
>>>>>> This setting needs to be called by AVD_SG::set_admin_state()
>>>>>>
>>>>>> 2. After receives su_si assignment response after headless,
>>>>>> @admin_ng,
>>>>>> @ng_using_saAmfSGAdminState have not been restored.
>>>>>> They need to be restored by somehow. Since nodegroup allows to be
>>>>>> created at any adminState. So there should be the case nodegroup's
>>>>>> AdminState is created with LOCKED but the belonging SUs are still
>>>>>> having
>>>>>> assignment, so adminState of nodegroup can't be used.
>>>>>> The admin_ng, ng_using_saAmfSGAdminState seem need to be stored to
>>>>>> IMM?
>>>>>> @Praveen: any suggestions?
>>>>>>
>>>>>> } else if ((su->sg_of_su->sg_ncs_spec == false) &&
>>>>>> ((su->su_on_node->admin_ng != nullptr) ||
>>>>>> (su->sg_of_su->ng_using_saAmfSGAdminState == true))) {
>>>>>> AVD_AMF_NG *ng = su->su_on_node->admin_ng;
>>>>>> //Got response from AMFND for assignments decrement
>>>>>> su_cnt_admin_oper.
>>>>>> if ((ng != nullptr) &&
>>>>>> (((((ng->admin_ng_pend_cbk.admin_oper ==
>>>>>> SA_AMF_ADMIN_SHUTDOWN) ||
>>>>>> (ng->admin_ng_pend_cbk.admin_oper ==
>>>>>> SA_AMF_ADMIN_LOCK)) &&
>>>>>> (su->saAmfSUNumCurrActiveSIs == 0) &&
>>>>>> (su->saAmfSUNumCurrStandbySIs == 0) &&
>>>>>> (AVSV_SUSI_ACT_DEL ==
>>>>>> n2d_msg->msg_info.n2d_su_si_assign.msg_act))) ||
>>>>>> (ng->admin_ng_pend_cbk.admin_oper ==
>>>>>> SA_AMF_ADMIN_UNLOCK))) {
>>>>>> su->su_on_node->su_cnt_admin_oper--;
>>>>>> TRACE("node:'%s', su_cnt_admin_oper:%u",
>>>>>> su->su_on_node->name.c_str(),su->su_on_node->su_cnt_admin_oper);
>>>>>> }
>>>>>> process_su_si_response_for_ng(su, SA_AIS_OK);
>>>>>>
>>>>>> On 25/08/16 21:36, Nagendra Kumar wrote:
>>>>>>> Further testing results:
>>>>>>> Node group lock has resulted in SG unstable. Logs and configuration
>>>>>>> file attached.
>>>>>>>
>>>>>>> Configuration : SC-1, PL-3 and PL-4.
>>>>>>>
>>>>>>> Steps:
>>>>>>>
>>>>>>> 1. Unlock SU1(on PL-3), SU2 and SU3 (Both on PL-4).
>>>>>>> 2. Create node group of PL-3 and PL-4:
>>>>>>> 3. Lock the node group.
>>>>>>> amf-adm lock safAmfNodeGroup=nagu,safAmfCluster=myAmfCluster
>>>>>>> 4. Keep gdb in csi set callback, stop SC-1 and start respond OK
>>>>>>> from
>>>>>>> csi set callback and start SC-1.
>>>>>>>
>>>>>>> SG becomes unstable if you try to unlock the Node group:
>>>>>>> Aug 25 16:57:06 PM_SC-1 osafamfd[2166]: NO
>>>>>>> 'safSg=AmfDemo_2N,safApp=AmfDemo1' is in unstable/transition state
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> -Nagu
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Nagendra Kumar
>>>>>>>> Sent: 24 August 2016 16:58
>>>>>>>> To: Minh Hon Chau; hans.nordeb...@ericsson.com; Praveen Malviya;
>>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
>>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
>>>>>>>> Subject: Re: [devel] [PATCH 2 of 2] AMFND: Admin operation
>>>>>>>> continuation if
>>>>>>>> csi callback completes during headless [#1725 part 1] V1
>>>>>>>>
>>>>>>>> The below is the assignments after the test case (SU2 has standby
>>>>>>>> assignment):
>>>>>>>>
>>>>>>>> PM_SC-1:/home/nagu/views/staging-1725 # /etc/init.d/opensafd
>>>>>>>> status
>>>>>>>> safSISU=safSu=SU2\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
>>>>>>>> mo1,safApp=AmfDemo1
>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>> safSISU=safSu=PL-
>>>>>>>> 3\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>> safSISU=safSu=PL-
>>>>>>>> 4\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>> safSISU=safSu=SC-
>>>>>>>> 2\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed4,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>> safSISU=safSu=SC-
>>>>>>>> 1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed1,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>> safSISU=safSu=SC-2\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>> safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Nagu
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Nagendra Kumar
>>>>>>>>> Sent: 24 August 2016 16:55
>>>>>>>>> To: Minh Hon Chau; hans.nordeb...@ericsson.com; Praveen Malviya;
>>>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
>>>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
>>>>>>>>> Subject: Re: [devel] [PATCH 2 of 2] AMFND: Admin operation
>>>>>>>>> continuation if csi callback completes during headless [#1725
>>>>>>>>> part 1]
>>>>>>>>> V1
>>>>>>>>>
>>>>>>>>> Hi Minh,
>>>>>>>>> With 1725_phase_1_V2.tgz, the below email TC has failed.
>>>>>>>>> Please
>>>>>>>> find
>>>>>>>>> the traces attached along with the configuration in the ticket.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> -Nagu
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Nagendra Kumar
>>>>>>>>>> Sent: 23 August 2016 15:15
>>>>>>>>>> To: Minh Hon Chau; hans.nordeb...@ericsson.com; Praveen Malviya;
>>>>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au
>>>>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
>>>>>>>>>> Subject: Re: [devel] [PATCH 2 of 2] AMFND: Admin operation
>>>>>>>>>> continuation if csi callback completes during headless [#1725
>>>>>>>>>> part
>>>>>>>>>> 1]
>>>>>>>>>> V1
>>>>>>>>>>
>>>>>>>>>> Hi Minh,
>>>>>>>>>> The following SU lock case is not working. This issue will
>>>>>>>>>> exist
>>>>>>>>>> for all the flows, so please check.
>>>>>>>>>>
>>>>>>>>>> Configuration and traces attached in the ticket.
>>>>>>>>>>
>>>>>>>>>> Steps:
>>>>>>>>>> 1. Start SC-1, SC-2, PL-3 and PL-4. Run the following command:
>>>>>>>>>> immcfg -f /tmp/AppConfig-2N-1725.xml amf-adm unlock-in
>>>>>>>>>> safSu=SU1,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> amf-adm unlock-in safSu=SU2,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> amf-adm unlock-in safSu=SU3,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> amf-adm unlock safSu=SU1,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> amf-adm unlock safSu=SU2,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> amf-adm unlock safSu=SU3,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>>
>>>>>>>>>> Assignments are:
>>>>>>>>>> PM_SC-1:/home/nagu/views/staging-1725 # /etc/init.d/opensafd
>>>>>>>>>> status
>>>>>>>>>> safSISU=safSu=SC-
>>>>>>>>>> 1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed1,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=SC-
>>>>>>>>>> 2\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=SC-2\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>>>> safSISU=safSu=PL-
>>>>>>>>>> 4\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=PL-
>>>>>>>>>> 3\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed4,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>>
>>>>>>>> safSISU=safSu=SU2\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
>>>>>>>>>> mo1,safApp=AmfDemo1
>>>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>>>>
>>>>>>>> safSISU=safSu=SU1\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
>>>>>>>>>> mo1,safApp=AmfDemo1
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>>
>>>>>>>>>> 2. Issue lock on SU1.
>>>>>>>>>> amf-adm lock safSu=SU1,safSg=AmfDemo_2N,safApp=AmfDemo1
>>>>>>>>>> And keep gdb in csi_set callback. Stop SC-1 and SC-2.
>>>>>>>>>> Send Ok from csi_set callback.
>>>>>>>>>>
>>>>>>>>>> 3. Start SC-1 and SC-2.
>>>>>>>>>>
>>>>>>>>>> 4. Assignment to components of SU2 is not given and
>>>>>>>>>> assignments of
>>>>>>>>>> SU2 still shows Standby.
>>>>>>>>>> PM_SC-1:/home/nagu/views/staging-1725 # /etc/init.d/opensafd
>>>>>>>>>> status
>>>>>>>>>>
>>>>>>>> safSISU=safSu=SU2\,safSg=AmfDemo_2N\,safApp=AmfDemo1,safSi=AmfDe
>>>>>>>>>> mo1,safApp=AmfDemo1
>>>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>>>> safSISU=safSu=SC-2\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=STANDBY(2)
>>>>>>>>>> safSISU=safSu=SC-
>>>>>>>>>> 1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed1,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=PL-
>>>>>>>>>> 4\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=PL-
>>>>>>>>>> 3\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=SC-
>>>>>>>>>> 2\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed4,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>> safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-
>>>>>>>>>> 2N,safApp=OpenSAF
>>>>>>>>>> saAmfSISUHAState=ACTIVE(1)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> -Nagu
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Minh Hon Chau [mailto:minh.c...@dektech.com.au]
>>>>>>>>>>> Sent: 05 August 2016 02:50
>>>>>>>>>>> To: hans.nordeb...@ericsson.com; Nagendra Kumar; Praveen
>>>>>>>>>>> Malviya;
>>>>>>>>>>> gary....@dektech.com.au; long.hb.ngu...@dektech.com.au;
>>>>>>>>>>> minh.c...@dektech.com.au
>>>>>>>>>>> Cc: opensaf-devel@lists.sourceforge.net
>>>>>>>>>>> Subject: [PATCH 2 of 2] AMFND: Admin operation continuation if
>>>>>>>>>>> csi
>>>>>>>>>> callback
>>>>>>>>>>> completes during headless [#1725 part 1] V1
>>>>>>>>>>>
>>>>>>>>>>> osaf/services/saf/amf/amfnd/di.cc | 199
>>>>>>>>>>> +++++++++++++++++-
>>>>>>>> ---
>>>>>>>>> --
>>>>>>>>>> --
>>>>>>>>>>> osaf/services/saf/amf/amfnd/include/avnd_di.h | 1 +
>>>>>>>>>>> 2 files changed, 134 insertions(+), 66 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The patch buffers susi_resp_msg during headless stage and
>>>>>>>>>>> resend
>>>>>>>>>>> it to AMFD after headless.
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfnd/di.cc
>>>>>>>>>>> b/osaf/services/saf/amf/amfnd/di.cc
>>>>>>>>>>> --- a/osaf/services/saf/amf/amfnd/di.cc
>>>>>>>>>>> +++ b/osaf/services/saf/amf/amfnd/di.cc
>>>>>>>>>>> @@ -804,11 +804,6 @@ uint32_t avnd_di_susi_resp_send(AVND_CB
>>>>>>>>>>> if (cb->term_state ==
>>>>>>>>>>> AVND_TERM_STATE_OPENSAF_SHUTDOWN_STARTED)
>>>>>>>>>>> return rc;
>>>>>>>>>>>
>>>>>>>>>>> - if (cb->is_avd_down == true) {
>>>>>>>>>>> - m_AVND_SU_ALL_SI_RESET(su);
>>>>>>>>>>> - return rc;
>>>>>>>>>>> - }
>>>>>>>>>>> -
>>>>>>>>>>> // should be in assignment pending state to be here
>>>>>>>>>>> osafassert(m_AVND_SU_IS_ASSIGN_PEND(su));
>>>>>>>>>>>
>>>>>>>>>>> @@ -819,64 +814,76 @@ uint32_t avnd_di_susi_resp_send(AVND_CB
>>>>>>>>>>> TRACE_ENTER2("Sending Resp su=%s, si=%s, curr_state=%u,
>>>>>>>>>>> prv_state=%u", su->name.value, curr_si->name.value,curr_si-
>>>>>>>>>>>> curr_state,curr_si->prv_state);
>>>>>>>>>>> /* populate the susi resp msg */
>>>>>>>>>>> msg.info.avd = new AVSV_DND_MSG();
>>>>>>>>>>> - msg.type = AVND_MSG_AVD;
>>>>>>>>>>> - msg.info.avd->msg_type =
>>>>>>>>>>> AVSV_N2D_INFO_SU_SI_ASSIGN_MSG;
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_id = ++(cb-
>>>>>>>>>>>> snd_msg_id);
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.node_id = cb-
>>>>>>>>>>>> node_info.nodeId;
>>>>>>>>>>> - if (si) {
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.single_csi =
>>>>>>>>>>> - ((si->single_csi_add_rem_in_si ==
>>>>>>>>>>> AVSV_SUSI_ACT_BASE)
>>>>>>>> ?
>>>>>>>>>>> false : true);
>>>>>>>>>>> - }
>>>>>>>>>>> - TRACE("curr_assign_state '%u'",
>>>>>>>>>>> curr_si->curr_assign_state);
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
>>>>>>>>>>> - (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
>>>>>>>>> ||
>>>>>>>>>>> -
>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si))
>>>>>>>>> ?
>>>>>>>>>>> - ((!curr_si->prv_state) ? AVSV_SUSI_ACT_ASGN :
>>>>>>>>>>> AVSV_SUSI_ACT_MOD) : AVSV_SUSI_ACT_DEL;
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.su_name = su->name;
>>>>>>>>>>> - if (si) {
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.si_name =
>>>>>>>>>>> si-
>>>>>>>>> name;
>>>>>>>>>>> - if (AVSV_SUSI_ACT_ASGN ==
>>>>>>>>>>> si->single_csi_add_rem_in_si) {
>>>>>>>>>>> - TRACE("si->curr_assign_state '%u'", curr_si-
>>>>>>>>>>>> curr_assign_state);
>>>>>>>>>>> -
>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
>>>>>>>>>>> -
>>>>>>>>>>> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si) ||
>>>>>>>>>>> -
>>>>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si)) ?
>>>>>>>>>>> - AVSV_SUSI_ACT_ASGN :
>>>>>>>>>>> AVSV_SUSI_ACT_DEL;
>>>>>>>>>>> - }
>>>>>>>>>>> - }
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.ha_state =
>>>>>>>>>>> - (SA_AMF_HA_QUIESCING == curr_si->curr_state) ?
>>>>>>>>>>> SA_AMF_HA_QUIESCED : curr_si->curr_state;
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.error =
>>>>>>>>>>> - (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si)
>>>>>>>>> ||
>>>>>>>>>>> -
>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_REMOVED(curr_si))
>>>>>>>>> ?
>>>>>>>>>>> NCSCC_RC_SUCCESS : NCSCC_RC_FAILURE;
>>>>>>>>>>> + msg.type = AVND_MSG_AVD;
>>>>>>>>>>> + msg.info.avd->msg_type = AVSV_N2D_INFO_SU_SI_ASSIGN_MSG;
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.node_id = cb-
>>>>>>>>>>>> node_info.nodeId;
>>>>>>>>>>> + if (si) {
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.single_csi =
>>>>>>>>>>> + ((si->single_csi_add_rem_in_si ==
>>>>>>>>>>> AVSV_SUSI_ACT_BASE) ? false : true);
>>>>>>>>>>> + }
>>>>>>>>>>> + TRACE("curr_assign_state '%u'",
>>>>>>>>>>> curr_si->curr_assign_state);
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.msg_act =
>>>>>>>>>>> +
>>>>>>>>>>> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si) ||
>>>>>>>>>>> +
>>>>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si)) ?
>>>>>>>>>>> + ((!curr_si->prv_state) ?
>>>>>>>>>>> AVSV_SUSI_ACT_ASGN : AVSV_SUSI_ACT_MOD) : AVSV_SUSI_ACT_DEL;
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.su_name = su->name;
>>>>>>>>>>> + if (si) {
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.si_name = si-
>>>>>>>>>>>> name;
>>>>>>>>>>> + if (AVSV_SUSI_ACT_ASGN ==
>>>>>>>>>>> si->single_csi_add_rem_in_si) {
>>>>>>>>>>> + TRACE("si->curr_assign_state '%u'", curr_si-
>>>>>>>>>>>> curr_assign_state);
>>>>>>>>>>> + msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.msg_act =
>>>>>>>>>>> +
>>>>>>>>>>> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si) ||
>>>>>>>>>>> +
>>>>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNING(curr_si)) ?
>>>>>>>>>>> + AVSV_SUSI_ACT_ASGN :
>>>>>>>>>>> AVSV_SUSI_ACT_DEL;
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.ha_state =
>>>>>>>>>>> + (SA_AMF_HA_QUIESCING == curr_si->curr_state) ?
>>>>>>>>>>> SA_AMF_HA_QUIESCED : curr_si->curr_state;
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.error =
>>>>>>>>>>> +
>>>>>>>>>>> (m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_ASSIGNED(curr_si) ||
>>>>>>>>>>> +
>>>>>>>>>>> m_AVND_SU_SI_CURR_ASSIGN_STATE_IS_REMOVED(curr_si)) ?
>>>>>>>>>>> +NCSCC_RC_SUCCESS : NCSCC_RC_FAILURE;
>>>>>>>>>>>
>>>>>>>>>>> - if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_ASGN)
>>>>>>>>>>> - osafassert(si);
>>>>>>>>>>> + if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_ASGN)
>>>>>>>>>>> + osafassert(si);
>>>>>>>>>>>
>>>>>>>>>>> - /* send the msg to AvD */
>>>>>>>>>>> - TRACE("Sending. msg_id'%u', node_id'%u', msg_act'%u',
>>>>>>>>>>> su'%s',
>>>>>>>>>> si'%s',
>>>>>>>>>>> ha_state'%u', error'%u', single_csi'%u'",
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_id,
>>>>>>>>> msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.node_id,
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.msg_act,
>>>>>>>>>> msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.su_name.value,
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.si_name.value,
>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.ha_state,
>>>>>>>>>>> - msg.info.avd->msg_info.n2d_su_si_assign.error,
>>>>>>>> msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.single_csi);
>>>>>>>>>>> + /* send the msg to AvD */
>>>>>>>>>>> + TRACE("Sending. msg_id'%u', node_id'%u', msg_act'%u',
>>>>>>>>>>> su'%s',
>>>>>>>>>>> si'%s', ha_state'%u', error'%u', single_csi'%u'",
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.msg_id,
>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.node_id,
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.msg_act,
>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.su_name.value,
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.si_name.value,
>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.ha_state,
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.error,
>>>>>>>>>>> +msg.info.avd->msg_info.n2d_su_si_assign.single_csi);
>>>>>>>>>>>
>>>>>>>>>>> - if ((su->si_list.n_nodes > 1) && (si == nullptr)) {
>>>>>>>>>>> - if
>>>>>>>>>>> (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_DEL)
>>>>>>>>>>> - LOG_NO("Removed 'all SIs' from '%s'",
>>>>>>>>>>> su->name.value);
>>>>>>>>>>> + if ((su->si_list.n_nodes > 1) && (si == nullptr)) {
>>>>>>>>>>> + if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_DEL)
>>>>>>>>>>> + LOG_NO("Removed 'all SIs' from '%s'", su-
>>>>>>>>>>>> name.value);
>>>>>>>>>>> - if
>>>>>>>>>>> (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_MOD)
>>>>>>>>>>> - LOG_NO("Assigned 'all SIs' %s of
>>>>>>>>>>> '%s'",
>>>>>>>>>>> - ha_state[msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.ha_state],
>>>>>>>>>>> - su->name.value);
>>>>>>>>>>> - }
>>>>>>>>>>> + if (msg.info.avd->msg_info.n2d_su_si_assign.msg_act ==
>>>>>>>>>>> AVSV_SUSI_ACT_MOD)
>>>>>>>>>>> + LOG_NO("Assigned 'all SIs' %s of '%s'",
>>>>>>>>>>> + ha_state[msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.ha_state],
>>>>>>>>>>> + su->name.value);
>>>>>>>>>>> + }
>>>>>>>>>>>
>>>>>>>>>>> - rc = avnd_di_msg_send(cb, &msg);
>>>>>>>>>>> - if (NCSCC_RC_SUCCESS == rc)
>>>>>>>>>>> - msg.info.avd = 0;
>>>>>>>>>>> -
>>>>>>>>>>> - /* we have completed the SU SI msg processing */
>>>>>>>>>>> - if (su_assign_state_is_stable(su))
>>>>>>>>>>> - m_AVND_SU_ASSIGN_PEND_RESET(su);
>>>>>>>>>>> - m_AVND_SU_ALL_SI_RESET(su);
>>>>>>>>>>> + if (cb->is_avd_down == true) {
>>>>>>>>>>> + // We are in headless, buffer this msg
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.msg_id = 0;
>>>>>>>>>>> + if (avnd_diq_rec_add(cb, &msg) == nullptr) {
>>>>>>>>>>> + rc = NCSCC_RC_FAILURE;
>>>>>>>>>>> + }
>>>>>>>>>>> + m_AVND_SU_ALL_SI_RESET(su);
>>>>>>>>>>> + LOG_NO("avnd_di_susi_resp_send() deferred as AMF
>>>>>>>>>>> director is offline");
>>>>>>>>>>> + } else {
>>>>>>>>>>> + // We are in normal cluster, send msg to director
>>>>>>>>>>> + msg.info.avd->msg_info.n2d_su_si_assign.msg_id = ++(cb-
>>>>>>>>>>>> snd_msg_id);
>>>>>>>>>>> + /* send the msg to AvD */
>>>>>>>>>>> + rc = avnd_di_msg_send(cb, &msg);
>>>>>>>>>>> + if (NCSCC_RC_SUCCESS == rc)
>>>>>>>>>>> + msg.info.avd = 0;
>>>>>>>>>>> + /* we have completed the SU SI msg processing */
>>>>>>>>>>> + if (su_assign_state_is_stable(su)) {
>>>>>>>>>>> + m_AVND_SU_ASSIGN_PEND_RESET(su);
>>>>>>>>>>> + }
>>>>>>>>>>> + m_AVND_SU_ALL_SI_RESET(su);
>>>>>>>>>>> + }
>>>>>>>>>>>
>>>>>>>>>>> /* free the contents of avnd message */
>>>>>>>>>>> avnd_msg_content_free(cb, &msg); @@ -1255,14 +1262,7 @@
>>>>>>>>>>> void
>>>>>>>>>>> avnd_diq_rec_del(AVND_CB *cb, AVND_
>>>>>>>>>>> /* stop the AvD msg response timer */
>>>>>>>>>>> if (m_AVND_TMR_IS_ACTIVE(rec->resp_tmr)) {
>>>>>>>>>>> m_AVND_TMR_MSG_RESP_STOP(cb, *rec);
>>>>>>>>>>> - // Resend msgs from queue because amfd dropped during
>>>>>>>>>>> sync
>>>>>>>>>>> - if ((cb->dnd_list.head != nullptr)) {
>>>>>>>>>>> - TRACE("retransmit message to amfd");
>>>>>>>>>>> - AVND_DND_MSG_LIST *pending_rec = 0;
>>>>>>>>>>> - for (pending_rec = cb->dnd_list.head;
>>>>>>>>>>> pending_rec !=
>>>>>>>>>>> nullptr; pending_rec = pending_rec->next) {
>>>>>>>>>>> - avnd_diq_rec_send(cb, pending_rec);
>>>>>>>>>>> - }
>>>>>>>>>>> - }
>>>>>>>>>>> + avnd_diq_rec_send_buffered_msg(cb);
>>>>>>>>>>> /* resend pg start track */
>>>>>>>>>>> avnd_di_resend_pg_start_track(cb);
>>>>>>>>>>> }
>>>>>>>>>>> @@ -1275,6 +1275,73 @@ void avnd_diq_rec_del(AVND_CB *cb,
>>>>>>>> AVND_
>>>>>>>>>>> TRACE_LEAVE();
>>>>>>>>>>> return;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>> +/************************************************************
>>>>>>>>>>> ****************
>>>>>>>>>>> + Name : avnd_diq_rec_send_buffered_msg
>>>>>>>>>>> +
>>>>>>>>>>> + Description : Resend buffered msg
>>>>>>>>>>> +
>>>>>>>>>>> + Arguments : cb - ptr to the AvND control block
>>>>>>>>>>> +
>>>>>>>>>>> + Return Values : None.
>>>>>>>>>>> +
>>>>>>>>>>> + Notes : None.
>>>>>>>>>>>
>>>>>>>> +*************************************************************
>>>>>>>>>>> **********
>>>>>>>>>>> +*******/ void avnd_diq_rec_send_buffered_msg(AVND_CB *cb) {
>>>>>>>>>>> + TRACE_ENTER();
>>>>>>>>>>> + // Resend msgs from queue because amfnd dropped during
>>>>>>>>>>> headless
>>>>>>>>>>> + // or headless-synchronization
>>>>>>>>>>> + if ((cb->dnd_list.head != nullptr)) {
>>>>>>>>>>> + AVND_DND_MSG_LIST *pending_rec = 0;
>>>>>>>>>>> + TRACE("Attach msg_id of buffered msg");
>>>>>>>>>>> + bool found = true;
>>>>>>>>>>> + while (found) {
>>>>>>>>>>> + found = false;
>>>>>>>>>>> + for (pending_rec = cb->dnd_list.head;
>>>>>>>>>>> pending_rec !=
>>>>>>>>>>> nullptr; pending_rec = pending_rec->next) {
>>>>>>>>>>> + if (pending_rec->msg.type ==
>>>>>>>>>>> AVND_MSG_AVD) {
>>>>>>>>>>> + // At this moment, only oper_state
>>>>>>>>>>> msg needs to report to director
>>>>>>>>>>> + if (pending_rec->msg.info.avd-
>>>>>>>>>>>> msg_type == AVSV_N2D_INFO_SU_SI_ASSIGN_MSG &&
>>>>>>>>>>> + pending_rec->msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.msg_id == 0) {
>>>>>>>>>>> + m_AVND_DIQ_REC_POP(cb,
>>>>>>>>>>> pending_rec); #if 0
>>>>>>>>>>> + // only resend if this SUSI
>>>>>>>>>>> does exist
>>>>>>>>>>> + AVND_SU *su =
>>>>>>>>>>> m_AVND_SUDB_REC_GET(cb->sudb,
>>>>>>>>>>> + pending_rec-
>>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.su_name);
>>>>>>>>>>> + if (su != nullptr && su-
>>>>>>>>>>>> si_list.n_nodes > 0) { #endif
>>>>>>>>>>> + pending_rec-
>>>>>>>>>>>> msg.info.avd->msg_info.n2d_su_si_assign.msg_id =
>>>>>>>>>>>> ++(cb->snd_msg_id);
>>>>>>>>>>> +
>>>>>>>>>>> m_AVND_DIQ_REC_PUSH(cb, pending_rec);
>>>>>>>>>>> + LOG_NO("Found and
>>>>>>>>>>> resend buffered su_si_assign msg for SU:'%s', "
>>>>>>>>>>> +
>>>>>>>>>>> "SI:'%s', ha_state:'%u', msg_act:'%u', single_csi:'%u', "
>>>>>>>>>>> +
>>>>>>>>>>> "error:'%u', msg_id:'%u'",
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.su_name.value,
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd-
>>>>>>>>>>>> msg_info.n2d_su_si_assign.si_name.value,
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd->msg_info.n2d_su_si_assign.ha_state,
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd->msg_info.n2d_su_si_assign.msg_act,
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd->msg_info.n2d_su_si_assign.single_csi,
>>>>>>>>>>>
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd->msg_info.n2d_su_si_assign.error,
>>>>>>>>>>> +
>>>>>>>>>>> pending_rec->msg.info.avd->msg_info.n2d_su_si_assign.msg_id);
>>>>>>>>>>> +
>>>>>>>>>>> +#if 0
>>>>>>>>>>> + } else {
>>>>>>>>>>> +
>>>>>>>>>>> avnd_msg_content_free(cb, &pending_rec->msg);
>>>>>>>>>>> + delete pending_rec;
>>>>>>>>>>> + pending_rec = cb-
>>>>>>>>>>>> dnd_list.head;
>>>>>>>>>>> + }
>>>>>>>>>>> +#endif
>>>>>>>>>>> + found = true;
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> + TRACE("retransmit message to amfd");
>>>>>>>>>>> + for (pending_rec = cb->dnd_list.head; pending_rec !=
>>>>>>>>>>> nullptr;
>>>>>>>>>>> pending_rec = pending_rec->next) {
>>>>>>>>>>> + avnd_diq_rec_send(cb, pending_rec);
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> + TRACE_LEAVE();
>>>>>>>>>>> + return;
>>>>>>>>>>> +}
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>> /*************************************************************
>>>>>>>>>>> ***************
>>>>>>>>>>> Name : avnd_diq_rec_send
>>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfnd/include/avnd_di.h
>>>>>>>>>>> b/osaf/services/saf/amf/amfnd/include/avnd_di.h
>>>>>>>>>>> --- a/osaf/services/saf/amf/amfnd/include/avnd_di.h
>>>>>>>>>>> +++ b/osaf/services/saf/amf/amfnd/include/avnd_di.h
>>>>>>>>>>> @@ -79,6 +79,7 @@ void avnd_di_msg_ack_process(struct avnd
>>>>>>>>>>> void
>>>>>>>>>>> avnd_diq_del(struct avnd_cb_tag *); AVND_DND_MSG_LIST
>>>>>>>>>>> *avnd_diq_rec_add(struct avnd_cb_tag *cb, AVND_MSG *msg); void
>>>>>>>>>>> avnd_diq_rec_del(struct avnd_cb_tag *cb, AVND_DND_MSG_LIST
>>>>>>>> *rec);
>>>>>>>>>>> +void avnd_diq_rec_send_buffered_msg(struct avnd_cb_tag *cb);
>>>>>>>>>>> uint32_t avnd_diq_rec_send(struct avnd_cb_tag *cb,
>>>>>>>>>> AVND_DND_MSG_LIST
>>>>>>>>>>> *rec); uint32_t avnd_di_reg_su_rsp_snd(struct avnd_cb_tag *cb,
>>>>>>>>>>> SaNameT *su_name, uint32_t ret_code); uint32_t
>>>>>>>>>> avnd_di_ack_nack_msg_send(struct
>>>>>>>>>>> avnd_cb_tag *cb, uint32_t rcv_id, uint32_t view_num);
>>>>>>>>>> --------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> -------- _______________________________________________
>>>>>>>>>> Opensaf-devel mailing list
>>>>>>>>>> Opensaf-devel@lists.sourceforge.net
>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------- _______________________________________________
>>>>>>>>> Opensaf-devel mailing list
>>>>>>>>> Opensaf-devel@lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Opensaf-devel mailing list
>>>>>>>> Opensaf-devel@lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>>>>>>
>>>>>
>>>>
>>>
>>
>
------------------------------------------------------------------------------
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
