Hi Hung Nguyen, Thanks a lot , I spend lot of time debugging IMM but didn't get much clue
Hi Zoran/Neel, can you Please send the patch to separate Imm tools in to a separate RPM and make part of Payload. -AVM On 2/17/2016 12:06 PM, Hung Nguyen wrote: > Hi Mahesh, > > Did you include the osafimmloadd/osafimmpbed in the RPMS for payload? > Normally, we don't need them for payload but in case of headless, the > immnd coordinator will be in payload. > > BR, > > Hung Nguyen - DEK Technologies > > > -------------------------------------------------------------------------------- > > > From: Mahesh Valla mahesh.va...@oracle.com > Sent: Wednesday, February 17, 2016 12:28PM > To: Opensaf-devel, Zoran Milinkovic, Neelakanta Reddy > opensaf-devel@lists.sourceforge.net, > zoran.milinko...@ericsson.com, reddy.neelaka...@oracle.com > Cc: > Subject: Re: [devel] [PATCH 1 of 5] imm: add common library > support for cloud resilience feature [#1625] > > > Hi Zoran/Neel, > > I am still facing the problem , the only difference between other > successful testing and my testing is , > i am using RPMS and others are using `make install` > > > ================================== > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process > at node 2030f old epoch: 20 new epoch:21 > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted > nodes:4 KnownVeteran:0 doReply:0 > Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO ABT REQUESTING SYNC > Feb 17 10:52:14 SC-1 osafimmd[9450]: WA IMMND on controller (not > currently coord) requests sync > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured > (900) => IMMND coord at payload node:2040f dest567416481644548 > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Node 2010f request sync > sync-pid:9461 epoch:0 > Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO SERVER STATE: > IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING > Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_ISOLATED > Feb 17 10:52:14 SC-1 osafimmd[9450]: WA IMMND on controller (not > currently coord) requests sync > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured > (900) => IMMND coord at payload node:2040f dest567416481644548 > Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Node 2020f request sync > sync-pid:8283 epoch:0 > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Successfully announced sync. New > ruling epoch:22 > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> > IMM_NODE_W_AVAILABLE > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO SERVER STATE: > IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process > at node 2040f old epoch: 21 new epoch:22 > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted > nodes:4 KnownVeteran:0 doReply:0 > Feb 17 10:52:18 SC-1 osafimmd[9450]: WA Successfully aborted sync. > Epoch:22 > Feb 17 10:52:18 SC-1 osafimmnd[9461]: WA Global ABORT SYNC received for > epoch 22 > Feb 17 10:52:18 SC-1 osafimmnd[9461]: WA SERVER STATE: > IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_UNKNOW > 2624 > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO Abort sync: Discarding synced > objects > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO Abort sync: Discarding synced > classes > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process > at node 2030f old epoch: 21 new epoch:22 > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted > nodes:4 KnownVeteran:0 doReply:0 > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO ABT REQUESTING SYNC > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO SERVER STATE: > IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING > Feb 17 10:52:18 SC-1 osafimmd[9450]: WA IMMND on controller (not > currently coord) requests sync > Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_ISOLATED > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured > (900) => IMMND coord at payload node:2040f dest567416481644548 > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Node 2010f request sync > sync-pid:9461 epoch:0 > Feb 17 10:52:18 SC-1 osafimmd[9450]: WA IMMND on controller (not > currently coord) requests sync > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured > (900) => IMMND coord at payload node:2040f dest567416481644548 > Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Node 2020f request sync > sync-pid:8283 epoch:0 > > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> > IMM_NODE_W_AVAILABLE > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO SERVER STATE: > IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT > Feb 17 10:52:04 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process > at node 2040f old epoch: 17 new epoch:18 > Feb 17 10:52:04 SC-2 osafimmd[8272]: NO IMMND coord at 2040f > Feb 17 10:52:04 SC-2 osafimmd[8272]: NO IMMND coord at 2040f > Feb 17 10:52:04 SC-2 osafimmnd[8283]: WA Global ABORT SYNC received for > epoch 18 > Feb 17 10:52:04 SC-2 osafimmnd[8283]: WA SERVER STATE: > IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_UNKNOW > 2624 > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced > objects > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced > classes > Feb 17 10:52:04 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process > at node 2030f old epoch: 17 new epoch:18 > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO ABT REQUESTING SYNC > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO SERVER STATE: > IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING > Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_ISOLATED > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: Ruling epoch noted as:19 > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> > IMM_NODE_W_AVAILABLE > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO SERVER STATE: > IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process > at node 2040f old epoch: 18 new epoch:19 > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f > Feb 17 10:52:07 SC-2 osafimmnd[8283]: WA Global ABORT SYNC received for > epoch 19 > Feb 17 10:52:07 SC-2 osafimmnd[8283]: WA SERVER STATE: > IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_UNKNOW > 2624 > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced > objects > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced > classes > Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process > at node 2030f old epoch: 18 new epoch:19 > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO ABT REQUESTING SYNC > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO SERVER STATE: > IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING > Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_ISOLATED > > > Feb 17 10:51:12 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:17 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 7 > Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO Epoch set to 7 in ImmModel > Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:21 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 8 > Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO Epoch set to 8 in ImmModel > Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:25 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 9 > Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO Epoch set to 9 in ImmModel > Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:30 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 10 > Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO Epoch set to 10 in ImmModel > Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:34 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 11 > Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO Epoch set to 11 in ImmModel > Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:38 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for > epoch 12 > Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO Epoch set to 12 in ImmModel > Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:43 PL-3 osafimmnd[5293]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > > > > Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:20 PL-4 osafimmnd[5381]: ER osafimmnd failed to exec sync, > error 2, exiting > Feb 17 10:51:20 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 > Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO -SERVER STATE: > IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY > Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO Epoch set to 8 in ImmModel > Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, > epoch:8 > Feb 17 10:51:20 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for > epoch 8 > Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO Announce sync, epoch:9 > Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO SERVER STATE: IMM_SERVER_READY > --> IMM_SERVER_SYNC_SERVER > Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:24 PL-4 osafimmnd[5386]: ER osafimmnd failed to exec sync, > error 2, exiting > Feb 17 10:51:25 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 > Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO -SERVER STATE: > IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY > Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO Epoch set to 9 in ImmModel > Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, > epoch:9 > Feb 17 10:51:25 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for > epoch 9 > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Announce sync, epoch:10 > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO SERVER STATE: IMM_SERVER_READY > --> IMM_SERVER_SYNC_SERVER > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_R_AVAILABLE > Feb 17 10:51:29 PL-4 osafimmnd[5387]: ER osafimmnd failed to exec sync, > error 2, exiting > Feb 17 10:51:29 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO -SERVER STATE: > IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO NODE STATE-> > IMM_NODE_FULLY_AVAILABLE (2610) > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Epoch set to 10 in ImmModel > Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, > epoch:10 > Feb 17 10:51:29 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for > epoch 10 > ============================================== > > On 2/12/2016 10:53 AM, A V Mahesh wrote: > >> Hi Zoran/Neel, >> >> As part of reviewing/testing CPSV headless state V2 [#1621] , we bring >> down the both SC`s and >> bring up both SC`s in reverse order ( old active as new standby , old >> standby as new active) >> which is piratical case where you cannot control heads joining sequence, >> as a result heads never joined the cluster and cluster went for >> reboot. >> >> I s this supported or not in headless state ? >> >> ============================================== >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO SERVER STATE: >> IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT >> Feb 12 10:41:13 SC-2 osafimmd[29890]: NO SBY: New Epoch for IMMND >> process at node 2040f old epoch: 198 new epoch:199 >> Feb 12 10:41:13 SC-2 osafimmd[29890]: NO IMMND coord at 2040f >> Feb 12 10:41:13 SC-2 osafimmd[29890]: NO IMMND coord at 2040f >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: WA Global ABORT SYNC received >> for epoch 199 >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: WA SERVER STATE: >> IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO NODE STATE-> IMM_NODE_UNKNOW >> 2621 >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO Abort sync: Discarding >> synced objects >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO Abort sync: Discarding >> synced classes >> Feb 12 10:41:13 SC-2 osafimmd[29890]: NO SBY: New Epoch for IMMND >> process at node 2030f old epoch: 198 new epoch:199 >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO ABT REQUESTING SYNC >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO SERVER STATE: >> IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING >> Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO NODE STATE-> IMM_NODE_ISOLATED >> Feb 12 10:41:17 SC-2 kernel: [71593.180556] tipc: Resetting link >> <1.1.2:eth2-1.1.3:eth1>, changeover initiated by peer >> Feb 12 10:41:17 SC-2 kernel: [71593.180562] tipc: Lost link >> <1.1.2:eth2-1.1.3:eth1> on network plane B >> Feb 12 10:41:17 SC-2 kernel: [71593.192851] tipc: Resetting link >> <1.1.2:eth2-1.1.4:eth2>, changeover initiated by peer >> Feb 12 10:41:17 SC-2 kernel: [71593.192857] tipc: Lost link >> <1.1.2:eth2-1.1.4:eth2> on network plane B >> Feb 12 10:41:19 SC-2 osafimmd[29890]: WA Message count:11144 + 1 != >> 11144 >> Feb 12 10:41:19 SC-2 kernel: [71595.365041] tipc: Resetting link >> <1.1.2:eth3-1.1.3:eth4>, peer not responding >> Feb 12 10:41:19 SC-2 kernel: [71595.365047] tipc: Lost link >> <1.1.2:eth3-1.1.3:eth4> on network plane A >> Feb 12 10:41:19 SC-2 kernel: [71595.365051] tipc: Lost contact with >> <1.1.3> >> Feb 12 10:41:19 SC-2 kernel: [71595.372092] tipc: Resetting link >> <1.1.2:eth3-1.1.4:eth0>, peer not responding >> Feb 12 10:41:19 SC-2 kernel: [71595.372100] tipc: Lost link >> <1.1.2:eth3-1.1.4:eth0> on network plane A >> Feb 12 10:41:19 SC-2 kernel: [71595.372105] tipc: Lost contact with >> <1.1.4> >> Feb 12 10:41:19 SC-2 osafimmd[29890]: ER Standby IMMD recieved reset >> message. All IMMNDs will restart. >> Feb 12 10:41:19 SC-2 osafimmd[29890]: ER IMM RELOAD with NO persistent >> back end => ensure cluster restart by IMMD exit at both SCs, exiting >> Feb 12 10:41:20 SC-2 osafimmnd[30074]: WA SC Absence IS allowed:900 >> IMMD service is DOWN >> Feb 12 10:41:20 SC-2 osafimmnd[30074]: WA IMMND was being synced or >> loaded (6), has to restart >> Feb 12 10:41:20 SC-2 opensafd[29839]: ER Could Not RESPAWN IMMND >> Feb 12 10:41:20 SC-2 opensafd[29839]: ER Failed DESC:IMMND >> Feb 12 10:41:20 SC-2 opensafd[29839]: ER Trying To RESPAWN >> /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #2 >> Feb 12 10:41:20 SC-2 opensafd[29839]: ER Sending SIGABRT to IMMND, >> pid=30074, (origin parent pid=30068) >> Feb 12 10:41:35 SC-2 osafimmnd[30187]: Started >> ============================================== >> >> >> >> On 2/3/2016 5:05 PM, Hung Nguyen wrote: >>> Hi Zoran, >>> >>> Please find my comments inline. >>> >>> >>> BR, >>> >>> Hung Nguyen - DEK Technologies >>> >>> >>> -------------------------------------------------------------------------------- >>> >>> >>> >>> From: Zoran Milinkovic zoran.milinko...@ericsson.com >>> Sent: Tuesday, December 22, 2015 9:14PM >>> To: Neelakanta Reddy >>> reddy.neelaka...@oracle.com >>> Cc: Opensaf-devel >>> opensaf-devel@lists.sourceforge.net >>> Subject: [devel] [PATCH 1 of 5] imm: add common library support for >>> cloud resilience feature [#1625] >>> >>> >>> osaf/libs/common/immsv/immpbe_dump.cc | 378 >>> +++++++++++++++++++++++++- >>> osaf/libs/common/immsv/immsv_evt.c | 43 ++ >>> osaf/libs/common/immsv/include/immpbe_dump.hh | 2 +- >>> osaf/libs/common/immsv/include/immsv_api.h | 1 + >>> osaf/libs/common/immsv/include/immsv_evt.h | 17 +- >>> 5 files changed, 430 insertions(+), 11 deletions(-) >>> >>> >>> The patch contains IMM common library code for cloud resilience feature >>> >>> diff --git a/osaf/libs/common/immsv/immpbe_dump.cc >>> b/osaf/libs/common/immsv/immpbe_dump.cc >>> --- a/osaf/libs/common/immsv/immpbe_dump.cc >>> +++ b/osaf/libs/common/immsv/immpbe_dump.cc >>> @@ -1185,6 +1185,359 @@ bailout: >>> exit(1); >>> } >>> +static bool verifyAttributeValuePBE(const char *attrVal, >>> + SaImmAttrValueT value, SaImmValueTypeT attrType) { >>> + >>> + if(!attrVal && !value) { >>> + return true; >>> + } >>> + if(!attrVal || !value) { >>> + return false; >>> + } >>> + >>> + switch(attrType) { >>> + case SA_IMM_ATTR_SADOUBLET: { >>> + char val[50]; >>> + snprintf(val, 50, "%.15g", *(double *)value); >>> + return strtod(attrVal, NULL) == strtod(val, NULL); >>> + } >>> + >>> + case SA_IMM_ATTR_SAFLOATT: >>> + return strtof(attrVal, NULL) == *(float *)value; >>> + >>> + case SA_IMM_ATTR_SAUINT32T: >>> + return strtoul(attrVal, NULL, 10) == *(SaUint32T *)value; >>> + >>> + case SA_IMM_ATTR_SAINT32T: >>> + return strtol(attrVal, NULL, 10) == *(SaInt32T *)value; >>> + >>> + case SA_IMM_ATTR_SAUINT64T: >>> + return strtoull(attrVal, NULL, 10) == *(SaUint64T *)value; >>> + >>> + case SA_IMM_ATTR_SATIMET: >>> + case SA_IMM_ATTR_SAINT64T: >>> + return strtoll(attrVal, NULL, 10) == *(SaInt64T *)value; >>> + >>> + case SA_IMM_ATTR_SASTRINGT: >>> + return strcmp(attrVal, *(SaStringT *)value) == 0; >>> + >>> + case SA_IMM_ATTR_SANAMET: >>> + return strcmp(attrVal, >>> osaf_extended_name_borrow((SaNameT *)value)) == 0; >>> + >>> + case SA_IMM_ATTR_SAANYT: { >>> + std::ostringstream ost; >>> + SaAnyT *any = (SaAnyT *)value; >>> + >>> + for (unsigned int i = 0; i < any->bufferSize; i++) { >>> + ost << std::hex >>> + << (((int)any->bufferAddr[i] < 0x10)? "0":"") >>> + << (int)any->bufferAddr[i]; >>> + } >>> + >>> + return strcmp(ost.str().c_str(), attrVal) == 0; >>> + } >>> + >>> + default: >>> + LOG_ER("UNKNOWN ATTRIBUTE TYPE: %d", attrType); >>> + abort(); >>> + } >>> + >>> + return false; >>> +} >>> + >>> +static bool verifyObjectAttributePBE(const char *attrVal, >>> SaImmAttrValuesT_2 *attr) { >>> + unsigned int i; >>> + >>> + if(!attrVal && attr->attrValuesNumber == 0) { >>> + return true; >>> + } >>> + >>> + for(i=0; i<attr->attrValuesNumber; i++) { >>> + if(verifyAttributeValuePBE(attrVal, attr->attrValues[i], >>> attr->attrValueType)) { >>> + return true; >>> + } >>> + } >>> + >>> + return false; >>> +} >>> + >>> +static bool verifyClassObjectsPBE(std::string classNameString, >>> + SaImmHandleT immHandle, >>> + void* db_handle) >>> +{ >>> + sqlite3* dbHandle = (sqlite3 *) db_handle; >>> + SaImmSearchHandleT searchHandle = 0; >>> + SaImmSearchParametersT_2 searchParam; >>> + const char *className = classNameString.c_str(); >>> + SaAisErrorT err; >>> + std::string sql; >>> + char **result = NULL; >>> + int nrow; >>> + int ncol; >>> + char *errmsg; >>> + int n, c, r, i; >>> + int attrPos; >>> + int objectCounter = 0; >>> + std::map<std::string, int> rdnMap; // <RDN, row in the >>> result> >>> + std::map<std::string, int>::iterator rmi; >>> + SaStringT objectName; >>> + SaImmAttrValuesT_2 **attributes; >>> + SaImmAttrDefinitionT_2 **attrDefinitions = NULL; >>> + SaImmClassCategoryT classCategory; >>> + int rdnAttrDefIndex = -1; >>> + bool ok = true; >>> + int rc; >>> + >>> + TRACE_ENTER(); >>> + >>> + err = saImmOmClassDescriptionGet_2(immHandle, >>> (SaImmClassNameT)className, &classCategory, &attrDefinitions); >>> + if(err != SA_AIS_OK) { >>> + LOG_ER("Cannot fetch class description for class '%s': %d", >>> className, err); >>> + TRACE_LEAVE(); >>> + return false; >>> + } >>> + >>> + // Skip runtime classes >>> + if(classCategory == SA_IMM_CLASS_RUNTIME) { >>> + saImmOmClassDescriptionMemoryFree_2(immHandle, >>> attrDefinitions); >>> + TRACE_LEAVE(); >>> + return true; >>> + } >>> + >>> + for(i=0; attrDefinitions[i]; i++) { >>> + if(attrDefinitions[i]->attrFlags & SA_IMM_ATTR_RDN) { >>> + rdnAttrDefIndex = i; >>> + } >>> + } >>> + >>> + if(attrDefinitions[i]) { >>> + // This should never happen >>> + LOG_ER("Cannot find RDN attribute in search result for class >>> '%s'", className); >>> + abort(); >>> + } >>> + >>> + searchParam.searchOneAttr.attrName = >>> (SaImmAttrNameT)"SaImmAttrClassName"; >>> + searchParam.searchOneAttr.attrValue = &className; >>> + searchParam.searchOneAttr.attrValueType = SA_IMM_ATTR_SASTRINGT; >>> + >>> + err = saImmOmSearchInitialize_2(immHandle, NULL, SA_IMM_SUBTREE, >>> + SA_IMM_SEARCH_ONE_ATTR | SA_IMM_SEARCH_GET_ALL_ATTR | >>> SA_IMM_SEARCH_PERSISTENT_ATTRS, >>> + &searchParam, NULL, &searchHandle); >>> + if(err != SA_AIS_OK) { >>> + LOG_ER("Cannot fetch objects of class '%s' from IMM: %d", >>> className, err); >>> + goto bailout; >>> + } >>> + >>> + sql = "SELECT o.dn, c.* FROM objects o, "; >>> + sql.append(className); >>> + sql.append(" c WHERE o.obj_id = c.obj_id"); >>> + rc = sqlite3_get_table(dbHandle, sql.c_str(), &result, &nrow, >>> &ncol, &errmsg); >>> + if(rc) { >>> + LOG_ER("SQL statement ('%s') failed. Reason: %s", >>> sql.c_str(), errmsg); >>> + goto bailout; >>> + } >>> + >>> + // Map object DNs for fast search >>> + for(n=1; n<=nrow; n++) { >>> + rdnMap[result[n * ncol]] = n; >>> + } >>> + >>> + // Iterate through IMM search result and check attributes >>> + while((err = saImmOmSearchNext_o3(searchHandle, &objectName, >>> &attributes)) == SA_AIS_OK) { >>> + // Increase object counter >>> + objectCounter++; >>> + >>> + // Skip checking IMM object >>> + if(!strcmp(objectName, >>> "opensafImm=opensafImm,safApp=safImmService") || >>> + !strcmp(objectName, >>> "safRdn=immManagement,safApp=safImmService")) { >>> + continue; >>> + } >>> + >>> + // Find object index in the fetched result >>> + rmi = rdnMap.find(objectName); >>> + if(rmi == rdnMap.end()) { >>> + LOG_ER("IMM object '%s' cannot be found in PBE >>> database", objectName); >>> + goto bailout; >>> + } >>> + >>> + // Check each attribute >>> + for(i=0; attrDefinitions[i]; i++) { >>> + // Don't check RDN attribute >>> + if(i == rdnAttrDefIndex) { >>> + continue; >>> + } >>> + >>> + /* Skip persistent attributes. >>> + * TODO: Should persistent runtime data be checked ??? >>> + */ >>> + if(attrDefinitions[i]->attrFlags & SA_IMM_ATTR_RUNTIME) { >>> + continue; >>> + } >>> + >>> + // Skip default attributes >>> + if(!strcmp(attrDefinitions[i]->attrName, >>> "SaImmAttrImplementerName") || >>> + !strcmp(attrDefinitions[i]->attrName, >>> "SaImmAttrClassName") || >>> + !strcmp(attrDefinitions[i]->attrName, >>> "SaImmAttrAdminOwnerName")) { >>> + continue; >>> + } >>> + >>> + // Find attribute from the search result >>> + for(attrPos=0; attributes[attrPos]; attrPos++) { >>> + if(!strcmp(attributes[attrPos]->attrName, >>> attrDefinitions[i]->attrName)) { >>> + break; >>> + } >>> + } >>> + if(!attributes[attrPos]) { >>> + LOG_ER("Cannot find attribute '%s' in object '%s' in >>> the search result", >>> + attrDefinitions[i]->attrName, objectName); >>> + goto bailout; >>> + } >>> + >>> + if(attrDefinitions[i]->attrFlags & >>> SA_IMM_ATTR_MULTI_VALUE) { >>> + // Check multi value attribute >>> + char **res; >>> + int nr, nc; // number of rows and columns >>> + >>> + // Create SQL statements to fetch multi values >>> + switch(attrDefinitions[i]->attrValueType) { >>> + case SA_IMM_ATTR_SADOUBLET: >>> + case SA_IMM_ATTR_SAFLOATT: >>> + sql = "SELECT real_val FROM >>> objects_real_multi WHERE obj_id = "; >>> + sql.append(result[rmi->second * ncol + 1]); >>> + sql.append(" AND attr_name = '"); >>> + sql.append(attrDefinitions[i]->attrName); >>> + sql.append("'"); >>> + break; >>> + >>> + case SA_IMM_ATTR_SAINT32T: >>> + case SA_IMM_ATTR_SAINT64T: >>> + case SA_IMM_ATTR_SAUINT32T: >>> + case SA_IMM_ATTR_SAUINT64T: >>> + case SA_IMM_ATTR_SATIMET: >>> + sql = "SELECT int_val FROM objects_int_multi >>> WHERE obj_id = "; >>> + sql.append(result[rmi->second * ncol + 1]); >>> + sql.append(" AND attr_name = '"); >>> + sql.append(attrDefinitions[i]->attrName); >>> + sql.append("'"); >>> + break; >>> + >>> + case SA_IMM_ATTR_SASTRINGT: >>> + case SA_IMM_ATTR_SANAMET: >>> + sql = "SELECT text_val FROM >>> objects_text_multi WHERE obj_id = "; >>> + sql.append(result[rmi->second * ncol + 1]); >>> + sql.append(" AND attr_name = '"); >>> + sql.append(attrDefinitions[i]->attrName); >>> + sql.append("'"); >>> + break; >>> + >>> + case SA_IMM_ATTR_SAANYT: >>> + sql = "SELECT real_val FROM >>> objects_real_multi WHERE obj_id = "; >>> + sql.append(result[rmi->second * ncol + 1]); >>> + sql.append(" AND attr_name = '"); >>> + sql.append(attrDefinitions[i]->attrName); >>> + sql.append("'"); >>> + break; >>> + >>> + default: >>> + LOG_ER("UNKOWN attribute type: %d", >>> attrDefinitions[i]->attrValueType); >>> + abort(); >>> + } >>> + >>> + // Execute SQL >>> + if(sqlite3_get_table(dbHandle, sql.c_str(), &res, >>> &nr, &nc, &errmsg) != SQLITE_OK) { >>> + LOG_ER("Cannot fetch values for attribute '%s' >>> in class '%s'. Reason: %s", >>> + attrDefinitions[i]->attrName, className, >>> errmsg); >>> + goto bailout; >>> + } >>> + >>> + // First multi value check. Check number of values >>> + if(nr != (int)attributes[attrPos]->attrValuesNumber) { >>> + LOG_ER("Mismatch in number of multi value >>> attributes in attribute '%s' in object '%s'", >>> + attributes[attrPos]->attrName, >>> objectName); >>> + ok = false; >>> + goto finish; >>> + } >>> + >>> + for(r=1; r<=nr; r++) { >>> + // First loop. res vs. attributes >>> + if(!verifyObjectAttributePBE(res[r], >>> attributes[attrPos])) { >>> + LOG_ER("Cannot match value for multi value >>> attribute '%s' in object '%s'", >>> + attributes[attrPos]->attrName, objectName); >>> + // Values do not match >>> + ok = false; >>> + goto finish; >>> + } >>> + // Second loop. attributes vs. res >>> + for(n=0; n<nr; n++) { >>> + if(verifyAttributeValuePBE(res[r], >>> attributes[attrPos]->attrValues[n], >>> attributes[attrPos]->attrValueType)) { >>> + // Found value >>> + break; >>> + } >>> + } >>> + if(n == nr) { >>> + LOG_ER("Cannot match value for multi value >>> attribute '%s' in object '%s'.", >>> + attributes[attrPos]->attrName, objectName); >>> + // Values didn't match in the second loop >>> + ok = false; >>> + goto finish; >>> + } >>> + } >>> + >>> +finish: >>> + sqlite3_free_table(res); >>> + if(!ok) { >>> + goto bailout; >>> + } >>> + } else { // Check non-multi value attribute >>> + // Find column in 'result' >>> + for(c=2; c<ncol; c++) { >>> + if(!strcmp(attrDefinitions[i]->attrName, >>> result[c])) { >>> + break; >>> + } >>> + } >>> + >>> + // Verify fetched value against value from the >>> search result >>> + if(!verifyObjectAttributePBE(result[rmi->second * ncol + c], >>> attributes[attrPos])) { >>> + LOG_ER("Cannot match attribute value for >>> attribute '%s' in object '%s'", >>> + attributes[attrPos]->attrName, >>> objectName); >>> + goto bailout; >>> + } >>> + } >>> + } >>> + } >>> + >>> + saImmOmSearchFinalize(searchHandle); >>> + sqlite3_free_table(result); >>> + saImmOmClassDescriptionMemoryFree_2(immHandle, attrDefinitions); >>> + >>> + if(objectCounter != nrow) { >>> + LOG_ER("Mismatch in number of objects between IMM database >>> and PBE database"); >>> + TRACE_LEAVE(); >>> + return false; >>> + } >>> + >>> + TRACE_LEAVE(); >>> + >>> + return true; >>> + >>> +bailout: >>> + if(result) { >>> + sqlite3_free_table(result); >>> + } >>> + >>> + if(searchHandle) { >>> + saImmOmSearchFinalize(searchHandle); >>> + } >>> + >>> + if(attrDefinitions) { >>> + saImmOmClassDescriptionMemoryFree_2(immHandle, >>> attrDefinitions); >>> + } >>> + >>> + TRACE_LEAVE(); >>> + >>> + return false; >>> +} >>> + >>> static ClassInfo* verifyClassPBE(std::string classNameString, >>> SaImmHandleT immHandle, >>> void* db_handle) >>> @@ -1199,11 +1552,9 @@ static ClassInfo* verifyClassPBE(std::st >>> SaImmAttrDefinitionT_2 **attrDefinitions; >>> SaAisErrorT errorCode; >>> /* >>> - This function does very little verification right now. It >>> should do more. >>> - Reason is we are re-attaching to a DB file after a gap during >>> which we do not know >>> - what has happened to that file. We should at least verify that >>> the number of classes >>> - and number of objects match. Each object should really have a >>> checksum. >>> - Verification of objects should be a separate function >>> verifyObjectPBE. >>> + We are re-attaching to a DB file after a gap during which we >>> do not know >>> + what has happened to that file. We should at least verify that >>> the number >>> + of classes and number of objects match. >>> */ >>> TRACE_ENTER(); >>> @@ -2509,7 +2860,7 @@ bool dumpClassesToPbe(SaImmHandleT immHa >>> return false; >>> } >>> -int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, >>> void* db_handle) >>> +int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, >>> void* db_handle, bool pbeVerifyObjects) >>> { >>> /* Function used only when re-connecting to an already >>> existing DB file. */ >>> std::list<std::string> classNameList; >>> @@ -2524,6 +2875,8 @@ int verifyPbeState(SaImmHandleT immHandl >>> int nrows=0; >>> int ncols=0; >>> bool badfile=false; >>> + bool verified; >>> + >>> TRACE_ENTER(); >>> classNameList = getClassNames(immHandle); >>> @@ -2542,11 +2895,20 @@ int verifyPbeState(SaImmHandleT immHandl >>> ClassInfo* cl_info = verifyClassPBE((*it), immHandle, >>> dbHandle); >>> if(cl_info) { >>> (*classIdMap)[(*it)] = cl_info; >>> - it++; >>> } else { >>> badfile=true; >>> goto bailout; >>> } >>> + >>> + if(pbeVerifyObjects) { >>> + verified = verifyClassObjectsPBE((*it), immHandle, >>> dbHandle); >>> + if(!verified) { >>> + badfile=true; >>> + goto bailout; >>> + } >>> + } >>> + >>> + it++; >>> } >>> rc = sqlite3_get_table(dbHandle, sqlQ.c_str(), &result, >>> &nrows, &ncols, &qErr); >>> @@ -3816,7 +4178,7 @@ SaAisErrorT getCcbOutcomeFromPbe(void* d >>> return SA_AIS_ERR_LIBRARY; >>> } >>> -int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, >>> void* db_handle) >>> +int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, >>> void* db_handle, bool pbeVerifyObjects) >>> { >>> abort(); >>> return 0; >>> diff --git a/osaf/libs/common/immsv/immsv_evt.c >>> b/osaf/libs/common/immsv/immsv_evt.c >>> --- a/osaf/libs/common/immsv/immsv_evt.c >>> +++ b/osaf/libs/common/immsv/immsv_evt.c >>> @@ -2908,6 +2908,27 @@ static uint32_t immsv_evt_enc_toplevel(I >>> ncs_enc_claim_space(o_ub, 4); >>> /* immdevt->info.ctrl_msg.pbeFile.buf encoded by >>> sublevel */ >>> } >>> + >>> + if((immdevt->info.ctrl_msg.refresh == 2) && >>> + (immdevt->type == IMMD_EVT_ND2D_INTRO)) { /* Intro >>> after IMMD restart. */ >>> + >>> + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 8); >>> + ncs_encode_64bit(&p8, >>> immdevt->info.ctrl_msg.fevs_count); >>> + ncs_enc_claim_space(o_ub, 8); >>> + >>> + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); >>> + ncs_encode_32bit(&p8, >>> immdevt->info.ctrl_msg.admo_id_count); >>> + ncs_enc_claim_space(o_ub, 4); >>> + >>> + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); >>> + ncs_encode_32bit(&p8, >>> immdevt->info.ctrl_msg.ccb_id_count); >>> + ncs_enc_claim_space(o_ub, 4); >>> + >>> + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); >>> + ncs_encode_32bit(&p8, >>> immdevt->info.ctrl_msg.impl_count); >>> + ncs_enc_claim_space(o_ub, 4); >>> + } >>> + >>> break; >>> case IMMD_EVT_ND2D_SYNC_FEVS_BASE: >>> @@ -4283,6 +4304,28 @@ static uint32_t immsv_evt_dec_toplevel(N >>> ncs_dec_skip_space(i_ub, 4); >>> /* immdevt->info.ctrl_msg.pbeFile.buf decoded by >>> sublevel */ >>> } >>> + >>> + >>> + if((immdevt->info.ctrl_msg.refresh == 2) && >>> + (immdevt->type == IMMD_EVT_ND2D_INTRO)) { /* Intro >>> after IMMD restart. */ >>> + >>> + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 8); >>> + immdevt->info.ctrl_msg.fevs_count = >>> ncs_decode_64bit(&p8); >>> + ncs_dec_skip_space(i_ub, 8); >>> + >>> + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); >>> + immdevt->info.ctrl_msg.admo_id_count = >>> ncs_decode_32bit(&p8); >>> + ncs_dec_skip_space(i_ub, 4); >>> + >>> + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); >>> + immdevt->info.ctrl_msg.ccb_id_count = >>> ncs_decode_32bit(&p8); >>> + ncs_dec_skip_space(i_ub, 4); >>> + >>> + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); >>> + immdevt->info.ctrl_msg.impl_count = >>> ncs_decode_32bit(&p8); >>> + ncs_dec_skip_space(i_ub, 4); >>> + } >>> + >>> break; >>> case IMMD_EVT_ND2D_SYNC_FEVS_BASE: >>> diff --git a/osaf/libs/common/immsv/include/immpbe_dump.hh >>> b/osaf/libs/common/immsv/include/immpbe_dump.hh >>> --- a/osaf/libs/common/immsv/include/immpbe_dump.hh >>> +++ b/osaf/libs/common/immsv/include/immpbe_dump.hh >>> @@ -81,7 +81,7 @@ void deleteClassToPBE(std::string classN >>> ClassInfo* theClass); >>> int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, >>> - void* db_handle); >>> + void* db_handle, bool pbeVerifyObjects); >>> int dumpObjectsToPbe(SaImmHandleT immHandle, ClassMap* >>> classIdMap, >>> void* db_handle); >>> diff --git a/osaf/libs/common/immsv/include/immsv_api.h >>> b/osaf/libs/common/immsv/include/immsv_api.h >>> --- a/osaf/libs/common/immsv/include/immsv_api.h >>> +++ b/osaf/libs/common/immsv/include/immsv_api.h >>> @@ -61,6 +61,7 @@ extern "C" { >>> #define OPENSAF_IMM_LONG_DNS_ALLOWED "longDnsAllowed" >>> #define OPENSAF_IMM_ACCESS_CONTROL_MODE "accessControlMode" >>> #define OPENSAF_IMM_AUTHORIZED_GROUP "authorizedGroup" >>> +#define OPENSAF_IMM_SC_ABSENCE_ALLOWED "scAbsenceAllowed" >>> typedef enum { >>> ACCESS_CONTROL_DISABLED = 0, >>> diff --git a/osaf/libs/common/immsv/include/immsv_evt.h >>> b/osaf/libs/common/immsv/include/immsv_evt.h >>> --- a/osaf/libs/common/immsv/include/immsv_evt.h >>> +++ b/osaf/libs/common/immsv/include/immsv_evt.h >>> @@ -412,7 +412,9 @@ typedef struct immsv_d2nd_control { >>> IMMSV_OCTET_STRING dir; >>> IMMSV_OCTET_STRING xmlFile; >>> - IMMSV_OCTET_STRING pbeFile; >>> + IMMSV_OCTET_STRING pbeFile; >>> + >>> + >>> } IMMSV_D2ND_CONTROL; >>> /**************************************************************************** >>> >>> >>> @@ -422,7 +424,7 @@ typedef struct immsv_d2nd_control { >>> typedef struct immsv_nd2d_control { >>> SaUint32T ndExecPid; >>> SaUint32T epoch; >>> - uint8_t refresh; //TRUE=> This is a refresh of epoch. >>> + uint8_t refresh; /* refresh>0 => Refresh of epoch; refresh==2 => >>> IMMD restart intro.*/ >>> uint8_t pbeEnabled;/* OpenSaf4.4: >>> 2:not-enabled-not-configured can be convertred >>> to 0 in immd. >>> 3:not-enabled-configured >>> @@ -435,6 +437,17 @@ typedef struct immsv_nd2d_control { >>> IMMSV_OCTET_STRING dir; >>> IMMSV_OCTET_STRING xmlFile; >>> IMMSV_OCTET_STRING pbeFile; >>> + >>> + /* OpenSaf4.6: At restart of IMMD sv-id, IMMND gives feeedback on: >>> [Hung] Not OpenSaf4.6 :) >>> >>> + - fevs_count >>> + - admo_id_count >>> + - ccb_id_count >>> + - impl_count >>> + */ >>> + SaUint64T fevs_count; //Max received at IMMND for Fevs Count >>> + SaUint32T admo_id_count;//Max received at IMMND for AdminOwner ID >>> + SaUint32T ccb_id_count; //Max received at IMMND for CCB ID >>> + SaUint32T impl_count; //Max received at IMMND for >>> Implementer ID >>> } IMMSV_ND2D_CONTROL; >>> typedef struct immsv_nd2d_2_pbe { >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> >>> _______________________________________________ >>> Opensaf-devel mailing list >>> Opensaf-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> >>> Site24x7 APM Insight: Get Deep Visibility into Application Performance >>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month >>> Monitor end-to-end web transactions and take corrective actions now >>> Troubleshoot faster and improve end-user experience. Signup Now! >>> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 >>> _______________________________________________ >>> Opensaf-devel mailing list >>> Opensaf-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/opensaf-devel > > ------------------------------------------------------------------------------ > > > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > Opensaf-devel mailing list > Opensaf-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/opensaf-devel > > > ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel