During node lock operation quiescing assignments and failover of SU was
performed honoring
SI dependency. Unlock of payload failed because AMF at AMFD did not reset the
admin operation flag admin_node_pend_cbk.admin_oper during shutdown operation.
Since this flag was set, admin operation for unlock rejected with TRY_AGAIN and
"Node undergoing admin operation. This analysis is applicable to both cases
mentioned in the ticket.
A patch will be floated which will ensure to reset the flag.
---
** [tickets:#663] unlock of payload node fails with time out**
**Status:** unassigned
**Created:** Tue Dec 17, 2013 11:28 AM UTC by surender khetavath
**Last Updated:** Tue Dec 17, 2013 01:34 PM UTC
**Owner:** nobody
changeset : 4733
model : 2n
configuration : 1App,1SG,2SUs with 4comps each, 4SIs with 1 CSI each
si-si deps configured as SI1 sponsor for SI2,3,4 resp
SU1 is active, SU2 is standby.
SU1 is mapped to pl-3 and SU2 to pl-4
saAmfSGAutoRepair=1(True)
SuFailover=1(True)
scenario:
1. shutdown of pl-3 ie where the active SU resides
2. make sure that SI rejects the Remove cbk
3. unlock the node.
/var/log/messages show:
Dec 17 16:06:41 PL-3 osafamfnd[2712]: NO Assigning
'safSi=SI4,safApp=test2nApp' QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:41 PL-3 osafamfnd[2712]: NO Assigned 'safSi=SI3,safApp=test2nApp'
QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:41 PL-3 osafamfnd[2712]: NO Assigned 'safSi=SI2,safApp=test2nApp'
QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:41 PL-3 osafamfnd[2712]: NO Assigned 'safSi=SI4,safApp=test2nApp'
QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Assigning
'safSi=SI1,safApp=test2nApp' QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Assigned 'safSi=SI1,safApp=test2nApp'
QUIESCING to 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Removing 'all (4) SIs' from
'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Removing 'safSi=SI1,safApp=test2nApp'
from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Removing 'safSi=SI2,safApp=test2nApp'
from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Removing 'safSi=SI3,safApp=test2nApp'
from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Removing
'safSi=SI4,safApp=test2nApp' from 'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO saAmfSUFailover is true for
'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO
'safComp=COMP1,safSu=SU1,safSg=SG,safApp=test2nApp' faulted due to
'csiRemovecallbackFailed' : Recovery is 'suFailover'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Terminating components of
'safSu=SU1,safSg=SG,safApp=test2nApp'(abruptly & unordered)
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp'
Presence State INSTANTIATED => TERMINATING
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd cleanup for
safComp=COMP4,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd cleanup for
safComp=COMP3,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd cleanup for
safComp=COMP1,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd cleanup for
safComp=COMP2,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp'
Presence State TERMINATING => UNINSTANTIATED
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Terminated all components in
'safSu=SU1,safSg=SG,safApp=test2nApp'
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO Informing director of sufailover
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp'
Presence State UNINSTANTIATED => UNINSTANTIATED
Dec 17 16:06:42 PL-3 osafamfnd[2712]: ER cannot unlink failed state file
/var/run/opensaf/amf_failed_state: No such file or directory
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp'
Presence State UNINSTANTIATED => INSTANTIATING
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd instantiate for
safComp=COMP4,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd instantiate for
safComp=COMP3,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd instantiate for
safComp=COMP2,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 logger: CLC-CLI spawnd instantiate for
safComp=COMP1,safSu=SU1,safSg=SG,safApp=test2nApp
Dec 17 16:06:42 PL-3 osafamfnd[2712]: NO 'safSu=SU1,safSg=SG,safApp=test2nApp'
Presence State INSTANTIATING => INSTANTIATED
console output:
amf-adm unlock safAmfNode=PL-3,safAmfCluster=myAmfCluster
error - command timed out (alarm)
amf-adm unlock safAmfNode=PL-3,safAmfCluster=myAmfCluster
error - command timed out (alarm)
states:
safAmfNode=PL-3,safAmfCluster=myAmfCluster
saAmfNodeAdminState=LOCKED(2)
saAmfNodeOperState=ENABLED(1)
safSu=SU1,safSg=SG,safApp=test2nApp
saAmfSUAdminState=UNLOCKED(1)
saAmfSUOperState=ENABLED(1)
saAmfSUPresenceState=INSTANTIATED(3)
saAmfSUReadinessState=OUT-OF-SERVICE(1)
safSu=SU2,safSg=SG,safApp=test2nApp
saAmfSUAdminState=UNLOCKED(1)
saAmfSUOperState=ENABLED(1)
saAmfSUPresenceState=INSTANTIATED(3)
saAmfSUReadinessState=IN-SERVICE(2)
---
Sent from sourceforge.net because [email protected] is
subscribed to https://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
https://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
