- **status**: accepted --> duplicate
- **Milestone**: future --> never
- **Comment**:
SC-1 AMFD trace:
1) SG lock time:
Apr 14 17:04:24.432554 osafamfd [4634:lga_api.c:0903] << saLogWriteLogAsync
Apr 14 17:04:24.432561 osafamfd [4634:avd_sg.c:0994] >> sg_admin_op_cb:
'safSg=SGONE,safApp=TWONAPP', 2
Apr 14 17:04:24.432572 osafamfd [4634:avd_sg.c:1343] >> avd_sg_admin_state_set:
safSg=SGONE,safApp=TWONAPP AdmState UNLOCKED => LOCKED
2)During lock AMFD sends quiesced assignments to dependents honoring SI dep to
SU1 on SC-1 and sends assignment removal to SU2 on SC-2
During these assignments fault occurs in both SU1 and SU2. which leads to
deletion of assignment in both.
But in SU1 faults finally escalated to nodefailover
Apr 14 17:04:25 SC-1 osafamfnd[4644]: NO
'safComp=COMP2SU1TWONAPP,safSu=SU1,safSg=SGONE,safApp=TWONAPP' faulted due to
'csiRemovecallbackFailed' : Recovery is 'nodeFailover'
and on SC-2 it leads to component failover only
Apr 14 17:04:27 SC-2 osafamfnd[5284]: NO
'safComp=COMP2SU2TWONAPP,safSu=SU2,safSg=SGONE,safApp=TWONAPP' faulted due to
'csiRemovecallbackFailed' : Recovery is 'componentFailover'
3)When AMFD gets removal of assignment from SU1, it deletes it SUSIs
Apr 14 17:04:26.968522 osafamfd [4634:avd_sgproc.c:0542] >>
avd_su_si_assign_evh: id:151, node:2010f, act:4,
'safSu=SU1,safSg=SGONE,safApp=TWONAPP', '', ha:3, err:1, single:0
Apr 14 17:04:26.968966 osafamfd [4634:avd_siass.c:0433] >>
avd_susi_delete: safSu=SU1,safSg=SGONE,safApp=TWONAPP
safSi=TWONSI1,safApp=TWONAPP
Apr 14 17:04:26.969524 osafamfd [4634:avd_siass.c:0433] >>
avd_susi_delete: safSu=SU1,safSg=SGONE,safApp=TWONAPP
afSi=TWONSI2,safApp=TWONAPP
Apr 14 17:04:26.970076 osafamfd [4634:avd_siass.c:0433] >>
avd_susi_delete: safSu=SU1,safSg=SGONE,safApp=TWONAPP
safSi=TWONSI3,safApp=TWONAPP
Apr 14 17:04:26.970645 osafamfd [4634:avd_siass.c:0433] >>
avd_susi_delete: safSu=SU1,safSg=SGONE,safApp=TWONAPP
safSi=TWONSI4,safApp=TWONAPP
Apr 14 17:04:26.971190 osafamfd [4634:avd_siass.c:0433] >>
avd_susi_delete: safSu=SU1,safSg=SGONE,safApp=TWONAPP
safSi=TWONSI5,safApp=TWONAPP
4)But AMFD was able to complete IMM update for SUSI deletion only for SI1:
Apr 14 17:04:26.988208 osafamfd [4634:avd_imm.c:1561] >>
job_exec_imm_objdelete: Delete
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI1,safApp=TWONAPP
5)In the nodefailover request from AMFND, AMFD sends reboot message:
Apr 14 17:04:26.999598 osafamfd [4634:avd_util.c:1719] TR Sending REBOOT
MSG to 2010f
6)Now AMFD could not update IMM for all the other SUSIs SI2-SI5. This is the
reason why user sees SUSIs for SI2-SI5 for SU1 even after SG unlock when SC-2
is the active controller:
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI1,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI1,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
7)The SG is stable after SG unlock with SC-2 is active controller.
Apr 14 17:05:09.511351 osafamfd [5274:avd_sg2Nfsm.c:0590] <<
avd_sg_2n_act_susi: act: 'safSu=SU2,safSg=SGONE,safApp=TWONAPP', stdby:
'safSu=SU3,safSg=SGONE,safApp=TWONAPP'
Apr 14 17:05:09.511365 osafamfd [5274:avd_sg2Nfsm.c:0749] <<
avd_sg_2n_su_chose_asgn: '(null)'
Apr 14 17:05:09.511381 osafamfd [5274:avd_sg2Nfsm.c:2007] TR sg_fsm_state 1 => 0
The only problem is extra runtime objects for SU1 assignnets in IMM.
Such problem of missing IMM update of runtime objects (SUSI) in the case when
node failover is escalated
by an appilcation hosted on active controller has been reported in #494.
Due to this marking this ticket as duplicate of #494.
---
** [tickets:#853] Few SUSI assignments stuck in quiesced state**
**Status:** duplicate
**Milestone:** never
**Created:** Mon Apr 14, 2014 11:38 AM UTC by surender khetavath
**Last Updated:** Thu May 01, 2014 05:12 AM UTC
**Owner:** Praveen
changeset : 5143
model : 2n
configuration : 1App,1SG,5SUs with 3comps each, 5SIs with 3CSIs each
si-si deps configured as SI1 sponsor for SI2,3,4 resp
SU1 mapped to SC-1,SU2 to SC-2,SU3 to pl-3 and SU4,5 to PL-4.
case:
1) SU1 was active and SU2 standby.
2) Lock sg. In the remove cbk which has active CSI assignment respond with
ERR_FAILED_OP.
3) Error escalated to Node failover and SU1/SC-1 went for reboot.
4) After unlock of SG, few SUSI assginments stuck in quiesced state as shown
below.
SUSI Assignments:
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI1,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI4,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=PL-3\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed5,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=PL-5\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed3,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=PL-4\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed4,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SC-2\,safSg=2N\,safApp=OpenSAF,safSi=SC-2N,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SC-2\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed1,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI2,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU2\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI1,safApp=TWONAPP
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SU1\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI5,safApp=TWONAPP
saAmfSISUHAState=QUIESCED(3)
safSISU=safSu=SU3\,safSg=SGONE\,safApp=TWONAPP,safSi=TWONSI3,safApp=TWONAPP
saAmfSISUHAState=STANDBY(2)
safSISU=safSu=SC-1\,safSg=NoRed\,safApp=OpenSAF,safSi=NoRed2,safApp=OpenSAF
saAmfSISUHAState=ACTIVE(1)
safSISU=safSu=SC-1\,safSg=2N\,safApp=OpenSAF,safSi=SC-2N,safApp=OpenSAF
saAmfSISUHAState=STANDBY(2)
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos. Get
unparalleled scalability from the best Selenium testing platform available.
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
