Hi Minh,
I am pushing this README_NODEGROUP in amf directory.
Any comment, will be updated post FC.
Do you agree?
Thanks,
Praveen
On 29-Aug-16 2:46 PM, praveen malviya wrote:
> Hi Minh,
>
> Following content will go in Readme:
>
> High Level Implementation Notes for LOCK and SHUTDOWN operation on NG:
> ===========================================================
>
> In 2N model, there are broadly two cases for LOCK and SHUTDOWN operation
> on NG:
> A) Whole 2N SG is mapped in NG which means all assigned SU are hosted on
> the nodes of NG. Or
> B) SG is partially mapped in NG i.e either active SU or standby SU is
> hosted on one of the nodes of NG.
>
> Currently 2N model supports SI dep with in SU. So in LOCK and SHUTDOWN
> operations quiesced and quiescing HA states, should be given honoring SI
> deps respectively.
> In case A), AVD_SG_FSM_SG_ADMIN is used in case A) as it honors si dep
> while giving quiesced or quiescing assignments.
> Case B) becomes the case of either lock of standby Node/SU or active
> Node/Su which still needs to be handled honoring SI dep. Here
> AVD_SG_FSM_SU_OPER fsm state is used via internal function
> su_admin_down() as it handles SI dep.
> So in 2N model, AMFD always uses internal FSM functions by calling
> them inside the wrapper function ng_admin().
>
> Other red models do not support SI deps within SU as of now. So in these
> models, AMFD uses AVD_SG_FSM_SG_REALIGN by keeping multiple SUs in
> operation list while performing LOCK and SHUTDOWN operation on NG. Once
> SI dep is fully supported in these modles, AMFD can use internal SG FSM
> state like 2N.
>
> However there is one case, where AMFD can still use AVD_SG_FSM_SG_ADMIN
> state for these red models when whole SG is mapped in NG (all assigned
> SUs are hosted on on the nodes of NG). But possibility of such a case
> is more in 2N model where only two SUs can be assigned anytime. In other
> red models, there can be many assigned SUs so possibility of whole SG is
> mapped in NG is very less. So, as of now, in other models
> AVD_SG_FSM_SG_REALIGN states is used by keeping multiple SUs in oper
> list for this case also.
> But when SI deps, is completely supported with in SU in these models,
> then AMFD cannot use realign state and it will have to use internal FSM
> code as it will be enhanced for that.
>
>
> Thanks,
> Praveemn
>
>
>
> On 29-Aug-16 12:44 PM, minh chau wrote:
>> Hi Praveen,
>>
>> Yes it's for #1454 and #1608
>>
>> Thanks,
>> Minh
>>
>> On 29/08/16 17:08, praveen malviya wrote:
>>> Hi Minh,
>>>
>>> Thanks for reviewing and testing.
>>> I guess the ack is for both #1454 and #1608.
>>> I will add a note.
>>>
>>>
>>> Thanks,
>>> Praveen
>>>
>>> On 29-Aug-16 12:21 PM, minh chau wrote:
>>>> Hi Praveen,
>>>>
>>>> Ack (normal test only), the abnormal failover is supposed to work in
>>>> REALIGN in NpM and Nway I think
>>>> Maybe you can add a note in README to revisit the consistency of using
>>>> SG FSM for nodegroup in all SGs?
>>>>
>>>> Thanks,
>>>> Minh
>>>>
>>>> On 24/08/16 19:02, praveen malviya wrote:
>>>>> Hi Minh,
>>>>>
>>>>> Please responses inline with [Praveen]
>>>>>
>>>>> Thanks,
>>>>> Praveen
>>>>>
>>>>> On 24-Aug-16 2:01 PM, minh chau wrote:
>>>>>> Hi Praveen,
>>>>>>
>>>>>> I have tested the patches in case all SUs of NpM/Nway belong to one
>>>>>> nodegroup, it works for me. Will try the case that SUs are partially
>>>>>> in/out of nodegroup tomorrow.
>>>>>> But please, if you have time, can you try to make NpM's ng_admin under
>>>>>> FSM_SG_ADMIN. Addition to my previous comment, and also as you
>>>>>> knew, in
>>>>>> #1725 it's hard to deduce the states (sg fsm state is one of those)
>>>>>> because different SG is using FSM differently in some cases. This
>>>>>> inconsistency causes a difficult, instead of applying a generic
>>>>>> solution
>>>>>> for all SG, then each SG has to be treated different way. I think we
>>>>>> could also see this kind of problem in future.
>>>>> [Praveen] In 5.1, #1725 is targeting only 2N model.In future when
>>>>> #1725 will support other red models (upcoming releases), I would do
>>>>> any changes that is required in other red models for NG operations.
>>>>>
>>>>>
>>>>>> Thanks,
>>>>>> Minh
>>>>>>
>>>>>> On 23/08/16 16:58, praveen malviya wrote:
>>>>>>> Hi Minh,
>>>>>>>
>>>>>>> I have attached patches for #1454 and #1608 in the ticket #1454.
>>>>>>> Please apply them in order.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Praveen
>>>>>>>
>>>>>>> On 23-Aug-16 11:56 AM, minh chau wrote:
>>>>>>>> Hi Praveen,
>>>>>>>>
>>>>>>>> Since AMF longDn has been pushed, can you please attach a longDn
>>>>>>>> rebased
>>>>>>>> version to ticket (both #1454 + #1608) so we can do some test?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Minh
>>>>>>>>
>>>>>>>> On 23/08/16 15:56, praveen malviya wrote:
>>>>>>>>> Hi Minh,
>>>>>>>>>
>>>>>>>>> Thanks for reviewing the patch.
>>>>>>>>> Please see inline with [Praveen]
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Praveen
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 23-Aug-16 5:53 AM, minh chau wrote:
>>>>>>>>>> Hi Praveen,
>>>>>>>>>>
>>>>>>>>>> One comment in line with [Minh]
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> Minh
>>>>>>>>>>
>>>>>>>>>> On 20/07/16 18:57, praveen.malv...@oracle.com wrote:
>>>>>>>>>>> osaf/services/saf/amf/amfd/include/sg.h | 1 +
>>>>>>>>>>> osaf/services/saf/amf/amfd/nodegroup.cc | 4 +-
>>>>>>>>>>> osaf/services/saf/amf/amfd/sg_npm_fsm.cc | 62
>>>>>>>>>>> ++++++++++++++++++++++++++++++-
>>>>>>>>>>> 3 files changed, 62 insertions(+), 5 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Currently 2N, N-Way Active and NoRed models are supported for
>>>>>>>>>>> lock,
>>>>>>>>>>> shutdown,
>>>>>>>>>>> lock-in and unlock-in admin operations on NGs.
>>>>>>>>>>>
>>>>>>>>>>> This patch supports NplusM model also.
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfd/include/sg.h
>>>>>>>>>>> b/osaf/services/saf/amf/amfd/include/sg.h
>>>>>>>>>>> --- a/osaf/services/saf/amf/amfd/include/sg.h
>>>>>>>>>>> +++ b/osaf/services/saf/amf/amfd/include/sg.h
>>>>>>>>>>> @@ -507,6 +507,7 @@ public:
>>>>>>>>>>> uint32_t susi_failed(AVD_CL_CB *cb, AVD_SU *su,
>>>>>>>>>>> struct avd_su_si_rel_tag *susi, AVSV_SUSI_ACT act,
>>>>>>>>>>> SaAmfHAStateT state);
>>>>>>>>>>> void node_fail_si_oper(AVD_CL_CB *cb, AVD_SU *su);
>>>>>>>>>>> + void ng_admin(AVD_SU *su, AVD_AMF_NG *ng);
>>>>>>>>>>> private:
>>>>>>>>>>> uint32_t su_fault_su_oper(AVD_CL_CB *cb, AVD_SU *su);
>>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfd/nodegroup.cc
>>>>>>>>>>> b/osaf/services/saf/amf/amfd/nodegroup.cc
>>>>>>>>>>> --- a/osaf/services/saf/amf/amfd/nodegroup.cc
>>>>>>>>>>> +++ b/osaf/services/saf/amf/amfd/nodegroup.cc
>>>>>>>>>>> @@ -687,6 +687,7 @@ void avd_ng_admin_state_set(AVD_AMF_NG*
>>>>>>>>>>> avd_send_admin_state_chg_ntf(&ng->name,
>>>>>>>>>>> (SaAmfNotificationMinorIdT)SA_AMF_NTFID_NG_ADMIN_STATE,
>>>>>>>>>>> old_state, ng->saAmfNGAdminState);
>>>>>>>>>>> + TRACE_LEAVE();
>>>>>>>>>>> }
>>>>>>>>>>> /**
>>>>>>>>>>> * @brief Verify if Node is stable for admin operation on
>>>>>>>>>>> Nodegroup
>>>>>>>>>>> etc.
>>>>>>>>>>> @@ -749,8 +750,7 @@ static SaAisErrorT check_red_model_servi
>>>>>>>>>>> LOG_NO("service outage for '%s' because of
>>>>>>>>>>> shutdown/lock "
>>>>>>>>>>> "on '%s'",sg->name.value,ng->name.value);
>>>>>>>>>>> - if ((sg->sg_redundancy_model ==
>>>>>>>>>>> SA_AMF_N_WAY_REDUNDANCY_MODEL) ||
>>>>>>>>>>> - (sg->sg_redundancy_model ==
>>>>>>>>>>> SA_AMF_NPM_REDUNDANCY_MODEL)) {
>>>>>>>>>>> + if (sg->sg_redundancy_model ==
>>>>>>>>>>> SA_AMF_N_WAY_REDUNDANCY_MODEL) {
>>>>>>>>>>> LOG_NO("Admin op on '%s' hosting SUs of '%s' with
>>>>>>>>>>> redundancy '%u' "
>>>>>>>>>>> "is not supported",ng->name.value,
>>>>>>>>>>> sg->name.value,
>>>>>>>>>>> sg->sg_redundancy_model);
>>>>>>>>>>> diff --git a/osaf/services/saf/amf/amfd/sg_npm_fsm.cc
>>>>>>>>>>> b/osaf/services/saf/amf/amfd/sg_npm_fsm.cc
>>>>>>>>>>> --- a/osaf/services/saf/amf/amfd/sg_npm_fsm.cc
>>>>>>>>>>> +++ b/osaf/services/saf/amf/amfd/sg_npm_fsm.cc
>>>>>>>>>>> @@ -120,16 +120,16 @@ static AVD_SU_SI_REL *avd_sg_npm_su_othr
>>>>>>>>>>> if (i_susi->si->list_of_sisu != i_susi) {
>>>>>>>>>>> o_susi = i_susi->si->list_of_sisu;
>>>>>>>>>>> if (o_susi->fsm != AVD_SU_SI_STATE_UNASGN)
>>>>>>>>>>> - return o_susi;
>>>>>>>>>>> + break;
>>>>>>>>>>> } else if (i_susi->si->list_of_sisu->si_next !=
>>>>>>>>>>> AVD_SU_SI_REL_NULL) {
>>>>>>>>>>> o_susi = i_susi->si->list_of_sisu->si_next;
>>>>>>>>>>> if (o_susi->fsm != AVD_SU_SI_STATE_UNASGN)
>>>>>>>>>>> - return o_susi;
>>>>>>>>>>> + break;
>>>>>>>>>>> }
>>>>>>>>>>> i_susi = i_susi->su_next;
>>>>>>>>>>> }
>>>>>>>>>>> -
>>>>>>>>>>> + TRACE_LEAVE2("o_susi:'%p'",o_susi);
>>>>>>>>>>> return o_susi;
>>>>>>>>>>> }
>>>>>>>>>>> @@ -4452,6 +4452,62 @@ uint32_t SG_NPM::sg_admin_down(AVD_CL_CB
>>>>>>>>>>> return NCSCC_RC_SUCCESS;
>>>>>>>>>>> }
>>>>>>>>>>> +/*
>>>>>>>>>>> + * @brief Handles modification of assignments in SU of
>>>>>>>>>>> NpM SG
>>>>>>>>>>> + * because of lock or shutdown operation on Node
>>>>>>>>>>> group.
>>>>>>>>>>> + * If SU does not have any SIs assigned to it, AMF
>>>>>>>>>>> will
>>>>>>>>>>> try
>>>>>>>>>>> + * to instantiate new SUs in the SG. If SU has
>>>>>>>>>>> assignments,
>>>>>>>>>>> + * then depending upon lock or shutdown operation,
>>>>>>>>>>> quiesced
>>>>>>>>>>> + * or quiescing state will be sent for active SIs
>>>>>>>>>>> in SU.
>>>>>>>>>>> + * If SU has only standby assignments then remove the
>>>>>>>>>>> assignments.
>>>>>>>>>>> + *
>>>>>>>>>>> + * @param[in] ptr to SU
>>>>>>>>>>> + * @param[in] ptr to nodegroup AVD_AMF_NG.
>>>>>>>>>>> + */
>>>>>>>>>>> +void SG_NPM::ng_admin(AVD_SU *su, AVD_AMF_NG *ng)
>>>>>>>>>>> +{
>>>>>>>>>>> + SaAmfHAStateT ha_state;
>>>>>>>>>>> +
>>>>>>>>>>> + TRACE_ENTER2("'%s', sg_fsm_state:%u",su->name.value,
>>>>>>>>>>> + su->sg_of_su->sg_fsm_state);
>>>>>>>>>>> +
>>>>>>>>>>> + if (su->list_of_susi == nullptr) {
>>>>>>>>>>> + avd_sg_app_su_inst_func(avd_cb, su->sg_of_su);
>>>>>>>>>>> + return;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if (ng->saAmfNGAdminState == SA_AMF_ADMIN_SHUTTING_DOWN)
>>>>>>>>>>> + ha_state = SA_AMF_HA_QUIESCING;
>>>>>>>>>>> + else
>>>>>>>>>>> + ha_state = SA_AMF_HA_QUIESCED;
>>>>>>>>>>> +
>>>>>>>>>>> + if (su->list_of_susi->state == SA_AMF_HA_ACTIVE) {
>>>>>>>>>>> + if (avd_sg_su_si_mod_snd(avd_cb, su, ha_state) ==
>>>>>>>>>>> NCSCC_RC_FAILURE) {
>>>>>>>>>>> + LOG_ER("quiescing/quiesced state transtion failed for
>>>>>>>>>>> '%s'",su->name.value);
>>>>>>>>>>> + goto done;
>>>>>>>>>>> + }
>>>>>>>>>>> + } else {
>>>>>>>>>>> + if (avd_sg_su_si_del_snd(avd_cb, su) == NCSCC_RC_FAILURE) {
>>>>>>>>>>> + LOG_ER("removal of standby assignment failed for
>>>>>>>>>>> '%s'",su->name.value);
>>>>>>>>>>> + goto done;
>>>>>>>>>>> + }
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + avd_sg_su_oper_list_add(avd_cb, su, false);
>>>>>>>>>>> + su->sg_of_su->set_fsm_state(AVD_SG_FSM_SG_REALIGN);
>>>>>>>>>> [Minh]: For ng_admin operation, 2N is using fsm
>>>>>>>>>> AVD_SG_FSM_SG_ADMIN, the
>>>>>>>>>> NoRed, NwayActive are using AVD_SG_FSM_SG_REALIGN. Now the
>>>>>>>>>> support for
>>>>>>>>>> NpM and Nway are using AVD_SG_FSM_SG_REALIGN also.
>>>>>>>>>> Quickly browsing through ng_admin() of NoRed, NwayActive, NpM,
>>>>>>>>>> Nway,
>>>>>>>>>> they are very similar to the existing of those sg_admin_down(),
>>>>>>>>>> respectively. I think the NoRed, Nway, NwayACtive, NpM should be
>>>>>>>>>> using
>>>>>>>>>> AVD_SG_FSM_SG_ADMIN in the same way as 2N. The reason is the same
>>>>>>>>>> admin
>>>>>>>>>> operation should be treated in the same SG FSM state/code
>>>>>>>>>> between SG
>>>>>>>>>> types, and if sg_admin_down() works, we should reuse them.
>>>>>>>>>
>>>>>>>>> In 2N model, there are only two cases:
>>>>>>>>> A) Whole SG is mapped in NG or
>>>>>>>>> B) SG is partially mapped in NG. One of the SU is having active or
>>>>>>>>> standby assignments in one of the nodes of NG.
>>>>>>>>>
>>>>>>>>> So in 2N model, AVD_SG_FSM_SG_ADMIN is used in case A) because 2N
>>>>>>>>> model supports SI dep within SU. Because of this quiesced/quiescing
>>>>>>>>> assignments must be given honoring si dep. AVD_SG_FSM_SG_ADMIN
>>>>>>>>> honors
>>>>>>>>> that while giving quiesced/quiescing assignments.
>>>>>>>>> Case B) becomes the case of either lock of standby Node/su or
>>>>>>>>> active
>>>>>>>>> node/su which still needs to be handled honoring SI dep in
>>>>>>>>> su_admin_down().
>>>>>>>>>
>>>>>>>>> Other red models do not support, SI deps within SU. Once it is
>>>>>>>>> supported we will have to use internal SG FSM state. But without SI
>>>>>>>>> deps, other red models can still use AVD_SG_FSM_SG_ADMIN when
>>>>>>>>> whole SG
>>>>>>>>> is mapped in NG. But possibility of such a case is more in 2N model
>>>>>>>>> where only two SUs can be assigned anytime. In other red models,
>>>>>>>>> there
>>>>>>>>> can be many assigned SUs so possibility of whole SG is mapped in
>>>>>>>>> NG is
>>>>>>>>> very less. So in other models AVD_SG_FSM_SG_REALIGN states is
>>>>>>>>> used by
>>>>>>>>> keeping multiple SUs in oper list because these red models are
>>>>>>>>> handling switchover situation in REALIGN state (removal of
>>>>>>>>> standby is
>>>>>>>>> always handled in realign state). But when SI deps, is completely
>>>>>>>>> supported within SU in these models, then we cannot use realign
>>>>>>>>> state
>>>>>>>>> and we will have to use internal FSM code because internal FSM code
>>>>>>>>> will be enhanced for that.
>>>>>>>>> In 2N model, switchover is being done honoring SI dep but that
>>>>>>>>> is in
>>>>>>>>> some bug fix cases.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Praveen
>>>>>>>>>>> +
>>>>>>>>>>> + //Increment node counter for tracking status of ng operation.
>>>>>>>>>>> + if ((su->any_susi_fsm_in_modify() == true) ||
>>>>>>>>>>> + (su->any_susi_fsm_in_unasgn() == true)) {
>>>>>>>>>>> + su->su_on_node->su_cnt_admin_oper++;
>>>>>>>>>>> + TRACE("node:%s, su_cnt_admin_oper:%u",
>>>>>>>>>>> su->su_on_node->name.value,
>>>>>>>>>>> + su->su_on_node->su_cnt_admin_oper);
>>>>>>>>>>> + }
>>>>>>>>>>> +done:
>>>>>>>>>>> + TRACE_LEAVE();
>>>>>>>>>>> + return;
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> SG_NPM::~SG_NPM() {
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Opensaf-devel mailing list
> Opensaf-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/opensaf-devel
>
------------------------------------------------------------------------------
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
