Hi Mahesh, The problem is in executing osafimmloadd. Check if the file is corrupted or is missing, as Hung stated.
BR, Zoran From: A V Mahesh [mailto:mahesh.va...@oracle.com] Sent: Wednesday, February 17, 2016 6:28 AM To: opensaf-devel@lists.sourceforge.net; Zoran Milinkovic; reddy.neelaka...@oracle.com Subject: Re: [devel] [PATCH 1 of 5] imm: add common library support for cloud resilience feature [#1625] Hi Zoran/Neel, I am still facing the problem , the only difference between other successful testing and my testing is , i am using RPMS and others are using `make install` ================================== Feb 17 10:52:14 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process at node 2030f old epoch: 20 new epoch:21 Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted nodes:4 KnownVeteran:0 doReply:0 Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO ABT REQUESTING SYNC Feb 17 10:52:14 SC-1 osafimmd[9450]: WA IMMND on controller (not currently coord) requests sync Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured (900) => IMMND coord at payload node:2040f dest567416481644548 Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Node 2010f request sync sync-pid:9461 epoch:0 Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING Feb 17 10:52:14 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_ISOLATED Feb 17 10:52:14 SC-1 osafimmd[9450]: WA IMMND on controller (not currently coord) requests sync Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured (900) => IMMND coord at payload node:2040f dest567416481644548 Feb 17 10:52:14 SC-1 osafimmd[9450]: NO Node 2020f request sync sync-pid:8283 epoch:0 Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Successfully announced sync. New ruling epoch:22 Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_W_AVAILABLE Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT Feb 17 10:52:18 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process at node 2040f old epoch: 21 new epoch:22 Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted nodes:4 KnownVeteran:0 doReply:0 Feb 17 10:52:18 SC-1 osafimmd[9450]: WA Successfully aborted sync. Epoch:22 Feb 17 10:52:18 SC-1 osafimmnd[9461]: WA Global ABORT SYNC received for epoch 22 Feb 17 10:52:18 SC-1 osafimmnd[9461]: WA SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_UNKNOW 2624 Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO Abort sync: Discarding synced objects Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO Abort sync: Discarding synced classes Feb 17 10:52:18 SC-1 osafimmd[9450]: NO ACT: New Epoch for IMMND process at node 2030f old epoch: 21 new epoch:22 Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Attached Nodes:4 Accepted nodes:4 KnownVeteran:0 doReply:0 Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO ABT REQUESTING SYNC Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING Feb 17 10:52:18 SC-1 osafimmd[9450]: WA IMMND on controller (not currently coord) requests sync Feb 17 10:52:18 SC-1 osafimmnd[9461]: NO NODE STATE-> IMM_NODE_ISOLATED Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured (900) => IMMND coord at payload node:2040f dest567416481644548 Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Node 2010f request sync sync-pid:9461 epoch:0 Feb 17 10:52:18 SC-1 osafimmd[9450]: WA IMMND on controller (not currently coord) requests sync Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Sc Absence Allowed is configured (900) => IMMND coord at payload node:2040f dest567416481644548 Feb 17 10:52:18 SC-1 osafimmd[9450]: NO Node 2020f request sync sync-pid:8283 epoch:0 Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_W_AVAILABLE Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT Feb 17 10:52:04 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process at node 2040f old epoch: 17 new epoch:18 Feb 17 10:52:04 SC-2 osafimmd[8272]: NO IMMND coord at 2040f Feb 17 10:52:04 SC-2 osafimmd[8272]: NO IMMND coord at 2040f Feb 17 10:52:04 SC-2 osafimmnd[8283]: WA Global ABORT SYNC received for epoch 18 Feb 17 10:52:04 SC-2 osafimmnd[8283]: WA SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_UNKNOW 2624 Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced objects Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced classes Feb 17 10:52:04 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process at node 2030f old epoch: 17 new epoch:18 Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO ABT REQUESTING SYNC Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING Feb 17 10:52:04 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_ISOLATED Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: Ruling epoch noted as:19 Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_W_AVAILABLE Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process at node 2040f old epoch: 18 new epoch:19 Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f Feb 17 10:52:07 SC-2 osafimmd[8272]: NO IMMND coord at 2040f Feb 17 10:52:07 SC-2 osafimmnd[8283]: WA Global ABORT SYNC received for epoch 19 Feb 17 10:52:07 SC-2 osafimmnd[8283]: WA SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_UNKNOW 2624 Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced objects Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO Abort sync: Discarding synced classes Feb 17 10:52:07 SC-2 osafimmd[8272]: NO SBY: New Epoch for IMMND process at node 2030f old epoch: 18 new epoch:19 Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO ABT REQUESTING SYNC Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING Feb 17 10:52:07 SC-2 osafimmnd[8283]: NO NODE STATE-> IMM_NODE_ISOLATED Feb 17 10:51:12 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:17 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 7 Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO Epoch set to 7 in ImmModel Feb 17 10:51:17 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:21 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 8 Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO Epoch set to 8 in ImmModel Feb 17 10:51:21 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:25 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 9 Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO Epoch set to 9 in ImmModel Feb 17 10:51:25 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:30 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 10 Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO Epoch set to 10 in ImmModel Feb 17 10:51:30 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:34 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 11 Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO Epoch set to 11 in ImmModel Feb 17 10:51:34 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:38 PL-3 osafimmnd[5293]: WA Global ABORT SYNC received for epoch 12 Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO Epoch set to 12 in ImmModel Feb 17 10:51:38 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:43 PL-3 osafimmnd[5293]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:20 PL-4 osafimmnd[5381]: ER osafimmnd failed to exec sync, error 2, exiting Feb 17 10:51:20 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO -SERVER STATE: IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO Epoch set to 8 in ImmModel Feb 17 10:51:20 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, epoch:8 Feb 17 10:51:20 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for epoch 8 Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO Announce sync, epoch:9 Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO SERVER STATE: IMM_SERVER_READY --> IMM_SERVER_SYNC_SERVER Feb 17 10:51:24 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:24 PL-4 osafimmnd[5386]: ER osafimmnd failed to exec sync, error 2, exiting Feb 17 10:51:25 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO -SERVER STATE: IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO Epoch set to 9 in ImmModel Feb 17 10:51:25 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, epoch:9 Feb 17 10:51:25 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for epoch 9 Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Announce sync, epoch:10 Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO SERVER STATE: IMM_SERVER_READY --> IMM_SERVER_SYNC_SERVER Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_R_AVAILABLE Feb 17 10:51:29 PL-4 osafimmnd[5387]: ER osafimmnd failed to exec sync, error 2, exiting Feb 17 10:51:29 PL-4 osafimmnd[5276]: ER SYNC APPARENTLY FAILED status:1 Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO -SERVER STATE: IMM_SERVER_SYNC_SERVER --> IMM_SERVER_READY Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO NODE STATE-> IMM_NODE_FULLY_AVAILABLE (2610) Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Epoch set to 10 in ImmModel Feb 17 10:51:29 PL-4 osafimmnd[5276]: NO Coord broadcasting ABORT_SYNC, epoch:10 Feb 17 10:51:29 PL-4 osafimmnd[5276]: WA Global ABORT SYNC received for epoch 10 ============================================== On 2/12/2016 10:53 AM, A V Mahesh wrote: Hi Zoran/Neel, As part of reviewing/testing CPSV headless state V2 [#1621] , we bring down the both SC`s and bring up both SC`s in reverse order ( old active as new standby , old standby as new active) which is piratical case where you cannot control heads joining sequence, as a result heads never joined the cluster and cluster went for reboot. I s this supported or not in headless state ? ============================================== Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO SERVER STATE: IMM_SERVER_SYNC_PENDING --> IMM_SERVER_SYNC_CLIENT Feb 12 10:41:13 SC-2 osafimmd[29890]: NO SBY: New Epoch for IMMND process at node 2040f old epoch: 198 new epoch:199 Feb 12 10:41:13 SC-2 osafimmd[29890]: NO IMMND coord at 2040f Feb 12 10:41:13 SC-2 osafimmd[29890]: NO IMMND coord at 2040f Feb 12 10:41:13 SC-2 osafimmnd[30074]: WA Global ABORT SYNC received for epoch 199 Feb 12 10:41:13 SC-2 osafimmnd[30074]: WA SERVER STATE: IMM_SERVER_SYNC_CLIENT --> IMM_SERVER_LOADING_PENDING (sync aborted) Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO NODE STATE-> IMM_NODE_UNKNOW 2621 Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO Abort sync: Discarding synced objects Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO Abort sync: Discarding synced classes Feb 12 10:41:13 SC-2 osafimmd[29890]: NO SBY: New Epoch for IMMND process at node 2030f old epoch: 198 new epoch:199 Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO ABT REQUESTING SYNC Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO SERVER STATE: IMM_SERVER_LOADING_PENDING --> IMM_SERVER_SYNC_PENDING Feb 12 10:41:13 SC-2 osafimmnd[30074]: NO NODE STATE-> IMM_NODE_ISOLATED Feb 12 10:41:17 SC-2 kernel: [71593.180556] tipc: Resetting link <1.1.2:eth2-1.1.3:eth1>, changeover initiated by peer Feb 12 10:41:17 SC-2 kernel: [71593.180562] tipc: Lost link <1.1.2:eth2-1.1.3:eth1> on network plane B Feb 12 10:41:17 SC-2 kernel: [71593.192851] tipc: Resetting link <1.1.2:eth2-1.1.4:eth2>, changeover initiated by peer Feb 12 10:41:17 SC-2 kernel: [71593.192857] tipc: Lost link <1.1.2:eth2-1.1.4:eth2> on network plane B Feb 12 10:41:19 SC-2 osafimmd[29890]: WA Message count:11144 + 1 != 11144 Feb 12 10:41:19 SC-2 kernel: [71595.365041] tipc: Resetting link <1.1.2:eth3-1.1.3:eth4>, peer not responding Feb 12 10:41:19 SC-2 kernel: [71595.365047] tipc: Lost link <1.1.2:eth3-1.1.3:eth4> on network plane A Feb 12 10:41:19 SC-2 kernel: [71595.365051] tipc: Lost contact with <1.1.3> Feb 12 10:41:19 SC-2 kernel: [71595.372092] tipc: Resetting link <1.1.2:eth3-1.1.4:eth0>, peer not responding Feb 12 10:41:19 SC-2 kernel: [71595.372100] tipc: Lost link <1.1.2:eth3-1.1.4:eth0> on network plane A Feb 12 10:41:19 SC-2 kernel: [71595.372105] tipc: Lost contact with <1.1.4> Feb 12 10:41:19 SC-2 osafimmd[29890]: ER Standby IMMD recieved reset message. All IMMNDs will restart. Feb 12 10:41:19 SC-2 osafimmd[29890]: ER IMM RELOAD with NO persistent back end => ensure cluster restart by IMMD exit at both SCs, exiting Feb 12 10:41:20 SC-2 osafimmnd[30074]: WA SC Absence IS allowed:900 IMMD service is DOWN Feb 12 10:41:20 SC-2 osafimmnd[30074]: WA IMMND was being synced or loaded (6), has to restart Feb 12 10:41:20 SC-2 opensafd[29839]: ER Could Not RESPAWN IMMND Feb 12 10:41:20 SC-2 opensafd[29839]: ER Failed DESC:IMMND Feb 12 10:41:20 SC-2 opensafd[29839]: ER Trying To RESPAWN /usr/lib64/opensaf/clc-cli/osaf-immnd attempt #2 Feb 12 10:41:20 SC-2 opensafd[29839]: ER Sending SIGABRT to IMMND, pid=30074, (origin parent pid=30068) Feb 12 10:41:35 SC-2 osafimmnd[30187]: Started ============================================== On 2/3/2016 5:05 PM, Hung Nguyen wrote: Hi Zoran, Please find my comments inline. BR, Hung Nguyen - DEK Technologies -------------------------------------------------------------------------------- From: Zoran Milinkovic zoran.milinko...@ericsson.com<mailto:zoran.milinko...@ericsson.com> Sent: Tuesday, December 22, 2015 9:14PM To: Neelakanta Reddy reddy.neelaka...@oracle.com<mailto:reddy.neelaka...@oracle.com> Cc: Opensaf-devel opensaf-devel@lists.sourceforge.net<mailto:opensaf-devel@lists.sourceforge.net> Subject: [devel] [PATCH 1 of 5] imm: add common library support for cloud resilience feature [#1625] osaf/libs/common/immsv/immpbe_dump.cc | 378 +++++++++++++++++++++++++- osaf/libs/common/immsv/immsv_evt.c | 43 ++ osaf/libs/common/immsv/include/immpbe_dump.hh | 2 +- osaf/libs/common/immsv/include/immsv_api.h | 1 + osaf/libs/common/immsv/include/immsv_evt.h | 17 +- 5 files changed, 430 insertions(+), 11 deletions(-) The patch contains IMM common library code for cloud resilience feature diff --git a/osaf/libs/common/immsv/immpbe_dump.cc b/osaf/libs/common/immsv/immpbe_dump.cc --- a/osaf/libs/common/immsv/immpbe_dump.cc +++ b/osaf/libs/common/immsv/immpbe_dump.cc @@ -1185,6 +1185,359 @@ bailout: exit(1); } +static bool verifyAttributeValuePBE(const char *attrVal, + SaImmAttrValueT value, SaImmValueTypeT attrType) { + + if(!attrVal && !value) { + return true; + } + if(!attrVal || !value) { + return false; + } + + switch(attrType) { + case SA_IMM_ATTR_SADOUBLET: { + char val[50]; + snprintf(val, 50, "%.15g", *(double *)value); + return strtod(attrVal, NULL) == strtod(val, NULL); + } + + case SA_IMM_ATTR_SAFLOATT: + return strtof(attrVal, NULL) == *(float *)value; + + case SA_IMM_ATTR_SAUINT32T: + return strtoul(attrVal, NULL, 10) == *(SaUint32T *)value; + + case SA_IMM_ATTR_SAINT32T: + return strtol(attrVal, NULL, 10) == *(SaInt32T *)value; + + case SA_IMM_ATTR_SAUINT64T: + return strtoull(attrVal, NULL, 10) == *(SaUint64T *)value; + + case SA_IMM_ATTR_SATIMET: + case SA_IMM_ATTR_SAINT64T: + return strtoll(attrVal, NULL, 10) == *(SaInt64T *)value; + + case SA_IMM_ATTR_SASTRINGT: + return strcmp(attrVal, *(SaStringT *)value) == 0; + + case SA_IMM_ATTR_SANAMET: + return strcmp(attrVal, osaf_extended_name_borrow((SaNameT *)value)) == 0; + + case SA_IMM_ATTR_SAANYT: { + std::ostringstream ost; + SaAnyT *any = (SaAnyT *)value; + + for (unsigned int i = 0; i < any->bufferSize; i++) { + ost << std::hex + << (((int)any->bufferAddr[i] < 0x10)? "0":"") + << (int)any->bufferAddr[i]; + } + + return strcmp(ost.str().c_str(), attrVal) == 0; + } + + default: + LOG_ER("UNKNOWN ATTRIBUTE TYPE: %d", attrType); + abort(); + } + + return false; +} + +static bool verifyObjectAttributePBE(const char *attrVal, SaImmAttrValuesT_2 *attr) { + unsigned int i; + + if(!attrVal && attr->attrValuesNumber == 0) { + return true; + } + + for(i=0; i<attr->attrValuesNumber; i++) { + if(verifyAttributeValuePBE(attrVal, attr->attrValues[i], attr->attrValueType)) { + return true; + } + } + + return false; +} + +static bool verifyClassObjectsPBE(std::string classNameString, + SaImmHandleT immHandle, + void* db_handle) +{ + sqlite3* dbHandle = (sqlite3 *) db_handle; + SaImmSearchHandleT searchHandle = 0; + SaImmSearchParametersT_2 searchParam; + const char *className = classNameString.c_str(); + SaAisErrorT err; + std::string sql; + char **result = NULL; + int nrow; + int ncol; + char *errmsg; + int n, c, r, i; + int attrPos; + int objectCounter = 0; + std::map<std::string, int> rdnMap; // <RDN, row in the result> + std::map<std::string, int>::iterator rmi; + SaStringT objectName; + SaImmAttrValuesT_2 **attributes; + SaImmAttrDefinitionT_2 **attrDefinitions = NULL; + SaImmClassCategoryT classCategory; + int rdnAttrDefIndex = -1; + bool ok = true; + int rc; + + TRACE_ENTER(); + + err = saImmOmClassDescriptionGet_2(immHandle, (SaImmClassNameT)className, &classCategory, &attrDefinitions); + if(err != SA_AIS_OK) { + LOG_ER("Cannot fetch class description for class '%s': %d", className, err); + TRACE_LEAVE(); + return false; + } + + // Skip runtime classes + if(classCategory == SA_IMM_CLASS_RUNTIME) { + saImmOmClassDescriptionMemoryFree_2(immHandle, attrDefinitions); + TRACE_LEAVE(); + return true; + } + + for(i=0; attrDefinitions[i]; i++) { + if(attrDefinitions[i]->attrFlags & SA_IMM_ATTR_RDN) { + rdnAttrDefIndex = i; + } + } + + if(attrDefinitions[i]) { + // This should never happen + LOG_ER("Cannot find RDN attribute in search result for class '%s'", className); + abort(); + } + + searchParam.searchOneAttr.attrName = (SaImmAttrNameT)"SaImmAttrClassName"; + searchParam.searchOneAttr.attrValue = &className; + searchParam.searchOneAttr.attrValueType = SA_IMM_ATTR_SASTRINGT; + + err = saImmOmSearchInitialize_2(immHandle, NULL, SA_IMM_SUBTREE, + SA_IMM_SEARCH_ONE_ATTR | SA_IMM_SEARCH_GET_ALL_ATTR | SA_IMM_SEARCH_PERSISTENT_ATTRS, + &searchParam, NULL, &searchHandle); + if(err != SA_AIS_OK) { + LOG_ER("Cannot fetch objects of class '%s' from IMM: %d", className, err); + goto bailout; + } + + sql = "SELECT o.dn, c.* FROM objects o, "; + sql.append(className); + sql.append(" c WHERE o.obj_id = c.obj_id"); + rc = sqlite3_get_table(dbHandle, sql.c_str(), &result, &nrow, &ncol, &errmsg); + if(rc) { + LOG_ER("SQL statement ('%s') failed. Reason: %s", sql.c_str(), errmsg); + goto bailout; + } + + // Map object DNs for fast search + for(n=1; n<=nrow; n++) { + rdnMap[result[n * ncol]] = n; + } + + // Iterate through IMM search result and check attributes + while((err = saImmOmSearchNext_o3(searchHandle, &objectName, &attributes)) == SA_AIS_OK) { + // Increase object counter + objectCounter++; + + // Skip checking IMM object + if(!strcmp(objectName, "opensafImm=opensafImm,safApp=safImmService") || + !strcmp(objectName, "safRdn=immManagement,safApp=safImmService")) { + continue; + } + + // Find object index in the fetched result + rmi = rdnMap.find(objectName); + if(rmi == rdnMap.end()) { + LOG_ER("IMM object '%s' cannot be found in PBE database", objectName); + goto bailout; + } + + // Check each attribute + for(i=0; attrDefinitions[i]; i++) { + // Don't check RDN attribute + if(i == rdnAttrDefIndex) { + continue; + } + + /* Skip persistent attributes. + * TODO: Should persistent runtime data be checked ??? + */ + if(attrDefinitions[i]->attrFlags & SA_IMM_ATTR_RUNTIME) { + continue; + } + + // Skip default attributes + if(!strcmp(attrDefinitions[i]->attrName, "SaImmAttrImplementerName") || + !strcmp(attrDefinitions[i]->attrName, "SaImmAttrClassName") || + !strcmp(attrDefinitions[i]->attrName, "SaImmAttrAdminOwnerName")) { + continue; + } + + // Find attribute from the search result + for(attrPos=0; attributes[attrPos]; attrPos++) { + if(!strcmp(attributes[attrPos]->attrName, attrDefinitions[i]->attrName)) { + break; + } + } + if(!attributes[attrPos]) { + LOG_ER("Cannot find attribute '%s' in object '%s' in the search result", + attrDefinitions[i]->attrName, objectName); + goto bailout; + } + + if(attrDefinitions[i]->attrFlags & SA_IMM_ATTR_MULTI_VALUE) { + // Check multi value attribute + char **res; + int nr, nc; // number of rows and columns + + // Create SQL statements to fetch multi values + switch(attrDefinitions[i]->attrValueType) { + case SA_IMM_ATTR_SADOUBLET: + case SA_IMM_ATTR_SAFLOATT: + sql = "SELECT real_val FROM objects_real_multi WHERE obj_id = "; + sql.append(result[rmi->second * ncol + 1]); + sql.append(" AND attr_name = '"); + sql.append(attrDefinitions[i]->attrName); + sql.append("'"); + break; + + case SA_IMM_ATTR_SAINT32T: + case SA_IMM_ATTR_SAINT64T: + case SA_IMM_ATTR_SAUINT32T: + case SA_IMM_ATTR_SAUINT64T: + case SA_IMM_ATTR_SATIMET: + sql = "SELECT int_val FROM objects_int_multi WHERE obj_id = "; + sql.append(result[rmi->second * ncol + 1]); + sql.append(" AND attr_name = '"); + sql.append(attrDefinitions[i]->attrName); + sql.append("'"); + break; + + case SA_IMM_ATTR_SASTRINGT: + case SA_IMM_ATTR_SANAMET: + sql = "SELECT text_val FROM objects_text_multi WHERE obj_id = "; + sql.append(result[rmi->second * ncol + 1]); + sql.append(" AND attr_name = '"); + sql.append(attrDefinitions[i]->attrName); + sql.append("'"); + break; + + case SA_IMM_ATTR_SAANYT: + sql = "SELECT real_val FROM objects_real_multi WHERE obj_id = "; + sql.append(result[rmi->second * ncol + 1]); + sql.append(" AND attr_name = '"); + sql.append(attrDefinitions[i]->attrName); + sql.append("'"); + break; + + default: + LOG_ER("UNKOWN attribute type: %d", attrDefinitions[i]->attrValueType); + abort(); + } + + // Execute SQL + if(sqlite3_get_table(dbHandle, sql.c_str(), &res, &nr, &nc, &errmsg) != SQLITE_OK) { + LOG_ER("Cannot fetch values for attribute '%s' in class '%s'. Reason: %s", + attrDefinitions[i]->attrName, className, errmsg); + goto bailout; + } + + // First multi value check. Check number of values + if(nr != (int)attributes[attrPos]->attrValuesNumber) { + LOG_ER("Mismatch in number of multi value attributes in attribute '%s' in object '%s'", + attributes[attrPos]->attrName, objectName); + ok = false; + goto finish; + } + + for(r=1; r<=nr; r++) { + // First loop. res vs. attributes + if(!verifyObjectAttributePBE(res[r], attributes[attrPos])) { + LOG_ER("Cannot match value for multi value attribute '%s' in object '%s'", + attributes[attrPos]->attrName, objectName); + // Values do not match + ok = false; + goto finish; + } + // Second loop. attributes vs. res + for(n=0; n<nr; n++) { + if(verifyAttributeValuePBE(res[r], attributes[attrPos]->attrValues[n], attributes[attrPos]->attrValueType)) { + // Found value + break; + } + } + if(n == nr) { + LOG_ER("Cannot match value for multi value attribute '%s' in object '%s'.", + attributes[attrPos]->attrName, objectName); + // Values didn't match in the second loop + ok = false; + goto finish; + } + } + +finish: + sqlite3_free_table(res); + if(!ok) { + goto bailout; + } + } else { // Check non-multi value attribute + // Find column in 'result' + for(c=2; c<ncol; c++) { + if(!strcmp(attrDefinitions[i]->attrName, result[c])) { + break; + } + } + + // Verify fetched value against value from the search result + if(!verifyObjectAttributePBE(result[rmi->second * ncol + c], attributes[attrPos])) { + LOG_ER("Cannot match attribute value for attribute '%s' in object '%s'", + attributes[attrPos]->attrName, objectName); + goto bailout; + } + } + } + } + + saImmOmSearchFinalize(searchHandle); + sqlite3_free_table(result); + saImmOmClassDescriptionMemoryFree_2(immHandle, attrDefinitions); + + if(objectCounter != nrow) { + LOG_ER("Mismatch in number of objects between IMM database and PBE database"); + TRACE_LEAVE(); + return false; + } + + TRACE_LEAVE(); + + return true; + +bailout: + if(result) { + sqlite3_free_table(result); + } + + if(searchHandle) { + saImmOmSearchFinalize(searchHandle); + } + + if(attrDefinitions) { + saImmOmClassDescriptionMemoryFree_2(immHandle, attrDefinitions); + } + + TRACE_LEAVE(); + + return false; +} + static ClassInfo* verifyClassPBE(std::string classNameString, SaImmHandleT immHandle, void* db_handle) @@ -1199,11 +1552,9 @@ static ClassInfo* verifyClassPBE(std::st SaImmAttrDefinitionT_2 **attrDefinitions; SaAisErrorT errorCode; /* - This function does very little verification right now. It should do more. - Reason is we are re-attaching to a DB file after a gap during which we do not know - what has happened to that file. We should at least verify that the number of classes - and number of objects match. Each object should really have a checksum. - Verification of objects should be a separate function verifyObjectPBE. + We are re-attaching to a DB file after a gap during which we do not know + what has happened to that file. We should at least verify that the number + of classes and number of objects match. */ TRACE_ENTER(); @@ -2509,7 +2860,7 @@ bool dumpClassesToPbe(SaImmHandleT immHa return false; } -int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, void* db_handle) +int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, void* db_handle, bool pbeVerifyObjects) { /* Function used only when re-connecting to an already existing DB file. */ std::list<std::string> classNameList; @@ -2524,6 +2875,8 @@ int verifyPbeState(SaImmHandleT immHandl int nrows=0; int ncols=0; bool badfile=false; + bool verified; + TRACE_ENTER(); classNameList = getClassNames(immHandle); @@ -2542,11 +2895,20 @@ int verifyPbeState(SaImmHandleT immHandl ClassInfo* cl_info = verifyClassPBE((*it), immHandle, dbHandle); if(cl_info) { (*classIdMap)[(*it)] = cl_info; - it++; } else { badfile=true; goto bailout; } + + if(pbeVerifyObjects) { + verified = verifyClassObjectsPBE((*it), immHandle, dbHandle); + if(!verified) { + badfile=true; + goto bailout; + } + } + + it++; } rc = sqlite3_get_table(dbHandle, sqlQ.c_str(), &result, &nrows, &ncols, &qErr); @@ -3816,7 +4178,7 @@ SaAisErrorT getCcbOutcomeFromPbe(void* d return SA_AIS_ERR_LIBRARY; } -int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, void* db_handle) +int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, void* db_handle, bool pbeVerifyObjects) { abort(); return 0; diff --git a/osaf/libs/common/immsv/immsv_evt.c b/osaf/libs/common/immsv/immsv_evt.c --- a/osaf/libs/common/immsv/immsv_evt.c +++ b/osaf/libs/common/immsv/immsv_evt.c @@ -2908,6 +2908,27 @@ static uint32_t immsv_evt_enc_toplevel(I ncs_enc_claim_space(o_ub, 4); /* immdevt->info.ctrl_msg.pbeFile.buf encoded by sublevel */ } + + if((immdevt->info.ctrl_msg.refresh == 2) && + (immdevt->type == IMMD_EVT_ND2D_INTRO)) { /* Intro after IMMD restart. */ + + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 8); + ncs_encode_64bit(&p8, immdevt->info.ctrl_msg.fevs_count); + ncs_enc_claim_space(o_ub, 8); + + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); + ncs_encode_32bit(&p8, immdevt->info.ctrl_msg.admo_id_count); + ncs_enc_claim_space(o_ub, 4); + + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); + ncs_encode_32bit(&p8, immdevt->info.ctrl_msg.ccb_id_count); + ncs_enc_claim_space(o_ub, 4); + + IMMSV_RSRV_SPACE_ASSERT(p8, o_ub, 4); + ncs_encode_32bit(&p8, immdevt->info.ctrl_msg.impl_count); + ncs_enc_claim_space(o_ub, 4); + } + break; case IMMD_EVT_ND2D_SYNC_FEVS_BASE: @@ -4283,6 +4304,28 @@ static uint32_t immsv_evt_dec_toplevel(N ncs_dec_skip_space(i_ub, 4); /* immdevt->info.ctrl_msg.pbeFile.buf decoded by sublevel */ } + + + if((immdevt->info.ctrl_msg.refresh == 2) && + (immdevt->type == IMMD_EVT_ND2D_INTRO)) { /* Intro after IMMD restart. */ + + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 8); + immdevt->info.ctrl_msg.fevs_count = ncs_decode_64bit(&p8); + ncs_dec_skip_space(i_ub, 8); + + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); + immdevt->info.ctrl_msg.admo_id_count = ncs_decode_32bit(&p8); + ncs_dec_skip_space(i_ub, 4); + + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); + immdevt->info.ctrl_msg.ccb_id_count = ncs_decode_32bit(&p8); + ncs_dec_skip_space(i_ub, 4); + + IMMSV_FLTN_SPACE_ASSERT(p8, local_data, i_ub, 4); + immdevt->info.ctrl_msg.impl_count = ncs_decode_32bit(&p8); + ncs_dec_skip_space(i_ub, 4); + } + break; case IMMD_EVT_ND2D_SYNC_FEVS_BASE: diff --git a/osaf/libs/common/immsv/include/immpbe_dump.hh b/osaf/libs/common/immsv/include/immpbe_dump.hh --- a/osaf/libs/common/immsv/include/immpbe_dump.hh +++ b/osaf/libs/common/immsv/include/immpbe_dump.hh @@ -81,7 +81,7 @@ void deleteClassToPBE(std::string classN ClassInfo* theClass); int verifyPbeState(SaImmHandleT immHandle, ClassMap *classIdMap, - void* db_handle); + void* db_handle, bool pbeVerifyObjects); int dumpObjectsToPbe(SaImmHandleT immHandle, ClassMap* classIdMap, void* db_handle); diff --git a/osaf/libs/common/immsv/include/immsv_api.h b/osaf/libs/common/immsv/include/immsv_api.h --- a/osaf/libs/common/immsv/include/immsv_api.h +++ b/osaf/libs/common/immsv/include/immsv_api.h @@ -61,6 +61,7 @@ extern "C" { #define OPENSAF_IMM_LONG_DNS_ALLOWED "longDnsAllowed" #define OPENSAF_IMM_ACCESS_CONTROL_MODE "accessControlMode" #define OPENSAF_IMM_AUTHORIZED_GROUP "authorizedGroup" +#define OPENSAF_IMM_SC_ABSENCE_ALLOWED "scAbsenceAllowed" typedef enum { ACCESS_CONTROL_DISABLED = 0, diff --git a/osaf/libs/common/immsv/include/immsv_evt.h b/osaf/libs/common/immsv/include/immsv_evt.h --- a/osaf/libs/common/immsv/include/immsv_evt.h +++ b/osaf/libs/common/immsv/include/immsv_evt.h @@ -412,7 +412,9 @@ typedef struct immsv_d2nd_control { IMMSV_OCTET_STRING dir; IMMSV_OCTET_STRING xmlFile; - IMMSV_OCTET_STRING pbeFile; + IMMSV_OCTET_STRING pbeFile; + + } IMMSV_D2ND_CONTROL; /**************************************************************************** @@ -422,7 +424,7 @@ typedef struct immsv_d2nd_control { typedef struct immsv_nd2d_control { SaUint32T ndExecPid; SaUint32T epoch; - uint8_t refresh; //TRUE=> This is a refresh of epoch. + uint8_t refresh; /* refresh>0 => Refresh of epoch; refresh==2 => IMMD restart intro.*/ uint8_t pbeEnabled;/* OpenSaf4.4: 2:not-enabled-not-configured can be convertred to 0 in immd. 3:not-enabled-configured @@ -435,6 +437,17 @@ typedef struct immsv_nd2d_control { IMMSV_OCTET_STRING dir; IMMSV_OCTET_STRING xmlFile; IMMSV_OCTET_STRING pbeFile; + + /* OpenSaf4.6: At restart of IMMD sv-id, IMMND gives feeedback on: [Hung] Not OpenSaf4.6 :) + - fevs_count + - admo_id_count + - ccb_id_count + - impl_count + */ + SaUint64T fevs_count; //Max received at IMMND for Fevs Count + SaUint32T admo_id_count;//Max received at IMMND for AdminOwner ID + SaUint32T ccb_id_count; //Max received at IMMND for CCB ID + SaUint32T impl_count; //Max received at IMMND for Implementer ID } IMMSV_ND2D_CONTROL; typedef struct immsv_nd2d_2_pbe { ------------------------------------------------------------------------------ _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net<mailto:Opensaf-devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/opensaf-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net<mailto:Opensaf-devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/opensaf-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
