Check the SO openstack adapter for errors. The error message seems to indicate the source of the error is OPENSTACK.
"extSystemErrorSource" : "OPENSTACK", but I'm not totally familiar with this error message so we might need an SO developer. Brian From: Narayanasetty Sowmya <[email protected]> Sent: Friday, December 11, 2020 1:37 PM To: FREEMAN, BRIAN D <[email protected]>; [email protected] Cc: Velugubantla Praveen <[email protected]> Subject: Re: [onap-discuss] Not able to deploy a VNF through GR-API in guilin Hi Brian, Thanks for your inputs. We did redeploy of sdnc and all healthchecks are passing and we are no more facing that issue but still we are facing an issue while deploying our sample VNF. The error is as follows: "STATUS: Error Source: OPENSTACK, Error Message: Exception occured during vnf adapter: 404 NOT FOUND: . FLOW STATUS: CreateNetworkBB has failed. TASK INFORMATION: Task name is null." Attaching bpmn trace for reference( where you can find msg like Exception occured during vnf adapter: 404 NOT FOUND: .") And we also seen that one of the SO component pod is not present in this release when compared to frankfurt i.e., so-ve-vnfm-adapter-55dcbcf479-5mhkm. Is it removed now? Thanks and Regards, Sowmya ________________________________ From: FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>> Sent: 11 December 2020 20:10 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>; FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>>; Narayanasetty Sowmya <[email protected]<mailto:[email protected]>> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: RE: [onap-discuss] Not able to deploy a VNF through GR-API in guilin Remembered that I had some postman queries for cluster state. In my environment peer voting status is healthy with all peer's reporting true. sdnc ssl nodeport is 30267 https://{{sdnc_port}}/jolokia/read/org.opendaylight.controller:type=DistributedOperationalDatastore,Category=ShardManager,name=shard-manager-operational<https://urldefense.com/v3/__https:/*7b*7bsdnc_port*7d*7d/jolokia/read/org.opendaylight.controller:type=DistributedOperationalDatastore,Category=ShardManager,name=shard-manager-operational__;JSUlJQ!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7KaW57zU$> { "request": { "mbean": "org.opendaylight.controller:Category=ShardManager,name=shard-manager-operational,type=DistributedOperationalDatastore", "type": "read" }, "value": { "LocalShards": [ "member-1-shard-default-operational", "member-1-shard-prefix-configuration-shard-operational", "member-1-shard-topology-operational", "member-1-shard-entity-ownership-operational", "member-1-shard-inventory-operational", "member-1-shard-toaster-operational" ], "SyncStatus": true, "MemberName": "member-1" }, "timestamp": 1607697354, "status": 200 } https://{{sdnc_port}}/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-inventory-config,type=DistributedConfigDatastore<https://urldefense.com/v3/__https:/*7b*7bsdnc_port*7d*7d/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-inventory-config,type=DistributedConfigDatastore__;JSUlJQ!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7xh-w5cw$> { "request": { "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-inventory-config,type=DistributedConfigDatastore", "type": "read" }, "value": { "ReadWriteTransactionCount": 0, "SnapshotIndex": 1, "InMemoryJournalLogSize": 1, "ReplicatedToAllIndex": 1, "Leader": "member-1-shard-inventory-config", "LastIndex": 2, "RaftState": "Leader", "LastApplied": 2, "LastCommittedTransactionTime": "1970-01-01 00:00:00.000", "LastLogIndex": 2, "LastLeadershipChangeTime": "2020-11-19 17:13:07.706", "PeerAddresses": "member-2-shard-inventory-config: akka.tcp://opendaylight-cluster-d...@dev-sdnc-1.sdnc-cluster.onap.svc.cluster.local:2550/user/shardmanager-config/member-2-shard-inventory-config, member-3-shard-inventory-config: akka.tcp://opendaylight-cluster-d...@dev-sdnc-2.sdnc-cluster.onap.svc.cluster.local:2550/user/shardmanager-config/member-3-shard-inventory-config", "FollowerInitialSyncStatus": false, "FollowerInfo": [ { "timeSinceLastActivity": "00:00:00.057", "active": true, "matchIndex": 2, "voting": true, "id": "member-2-shard-inventory-config", "nextIndex": 3 }, { "timeSinceLastActivity": "00:00:00.065", "active": true, "matchIndex": 2, "voting": true, "id": "member-3-shard-inventory-config", "nextIndex": 3 } ], "FailedReadTransactionsCount": 0, "Voting": true, "StatRetrievalTime": "4.595 ms", "CurrentTerm": 12, "LastTerm": 12, "FailedTransactionsCount": 0, "PendingTxCommitQueueSize": 0, "VotedFor": "member-1-shard-inventory-config", "SnapshotCaptureInitiated": false, "CommittedTransactionsCount": 0, "TxCohortCacheSize": 0, "PeerVotingStates": "member-2-shard-inventory-config: true, member-3-shard-inventory-config: true", "LastLogTerm": 12, "StatRetrievalError": null, "CommitIndex": 2, "SnapshotTerm": 12, "AbortTransactionsCount": 0, "ReadOnlyTransactionCount": 0, "ShardName": "member-1-shard-inventory-config", "LeadershipChangeCount": 1, "InMemoryJournalDataSize": 33 }, "timestamp": 1607697502, "status": 200 } From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of Brian Freeman Sent: Friday, December 11, 2020 9:13 AM To: [email protected]<mailto:[email protected]>; FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>>; Narayanasetty Sowmya <[email protected]<mailto:[email protected]>> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: Re: [onap-discuss] Not able to deploy a VNF through GR-API in guilin ***Security Advisory: This Message Originated Outside of AT&T *** Reference http://cso.att.com/EmailSecurity/IDSP.html<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fcso.att.com*2FEmailSecurity*2FIDSP.html&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685258517*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=Yc*2FraLLLr2sTqzvQIJz4gSjdlj9JbjijkOBYlUPLgd0*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJQ!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7O1UeLN8$> for more information. Correcting some misleading typo's Brian From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of Brian Freeman Sent: Friday, December 11, 2020 9:06 AM To: Narayanasetty Sowmya <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: Re: [onap-discuss] Not able to deploy a VNF through GR-API in guilin ***Security Advisory: This Message Originated Outside of AT&T *** Reference http://cso.att.com/EmailSecurity/IDSP.html<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fcso.att.com*2FEmailSecurity*2FIDSP.html&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685258517*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=Yc*2FraLLLr2sTqzvQIJz4gSjdlj9JbjijkOBYlUPLgd0*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJQ!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7O1UeLN8$> for more information. If you look inside the guilin-sdnc-2 you will likely see opendaylight errors in the karaf.log. I would delete the pods that have shard errors in karaf.log and make sure healthcheck for sdnc passes. There are methods to see cluster health inside opendaylight but I'm not familiar enough with the command from the opendaylight documentation. brian From: Narayanasetty Sowmya <[email protected]<mailto:[email protected]>> Sent: Friday, December 11, 2020 7:39 AM To: FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: Re: Not able to deploy a VNF through GR-API in guilin Hi BRIAN, Thanks for your response, Here is the state of SDNC & SDNR pods in our environment, root@guilin-nfs:~/oom/kubernetes/robot# kubectl get pods | grep sdnc guilin-sdnc-0 2/2 Running 0 9d guilin-sdnc-1 2/2 Running 0 9d guilin-sdnc-2 2/2 Running 0 9d guilin-sdnc-ansible-server-58c4fdc4c9-mg4qv 1/1 Running 0 9d guilin-sdnc-db-0 1/1 Running 0 9d guilin-sdnc-dbinit-job-6pkjq 0/1 Completed 0 9d guilin-sdnc-dgbuilder-7d69754bfc-ct2vp 1/1 Running 0 9d guilin-sdnc-dmaap-listener-65687bf5f-kcf62 1/1 Running 0 9d guilin-sdnc-sdnrdb-init-job-jstkg 0/1 Completed 0 9d guilin-sdnc-sdnrdb-init-job-stcwp 0/1 Init:Error 0 9d guilin-sdnc-ueb-listener-5d69799658-8bd4p 1/1 Running 0 9d guilin-sdnc-web-7fcd558c58-zwkzs 1/1 Running 0 9d guilin-so-sdnc-adapter-6fb55759c8-7m7qr 2/2 Running 0 9d root@guilin-nfs:~/oom/kubernetes/robot# ****************************************************************************************************************** root@guilin-nfs:~/oom/kubernetes/robot# kubectl get pods | grep sdnr guilin-sdnc-sdnrdb-init-job-ccf4r 0/1 Init:Error 0 9d guilin-sdnc-sdnrdb-init-job-jstkg 0/1 Completed 0 9d guilin-sdnc-sdnrdb-init-job-stcwp 0/1 Init:Error 0 9d guilin-sdnrdb-coordinating-only-6964b89b5b-vph8f 2/2 Running 0 9d guilin-sdnrdb-master-0 1/1 Running 0 9d guilin-sdnrdb-master-1 1/1 Running 0 9d guilin-sdnrdb-master-2 1/1 Running 0 9d As seems all the containers are in good state, Is the deletion of that pod will solve the problem ? Regards, Sowmya ________________________________ From: FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>> Sent: 11 December 2020 17:54 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>; Narayanasetty Sowmya <[email protected]<mailto:[email protected]>> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: RE: Not able to deploy a VNF through GR-API in guilin Sounds like one of your sdnc containers is down (perhaps dev-sdnc-2). kubectl -n onap get pod | grep sdnc Sdnc failing healthcheck means you won't be able to deploy a VNF anyway. I'd restart the failed sdnc container : kubectl -n onap delete pod dev-sdnc-0 since it looks like the containers are either network isolated or one opendaylight container is not responding to the AKKA cluster health check messages. brian From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of Narayanasetty Sowmya Sent: Friday, December 11, 2020 3:39 AM To: [email protected]<mailto:[email protected]> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: [onap-discuss] Not able to deploy a VNF through GR-API in guilin Hi Team, I am trying to deploy a vnf using gr-api by using the python scripts. But I am facing an error as follows: "response-message": "TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-config/member-3-shard-default-config#-1175421278)]] after [120000 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.]]}" And in my Guilin setup, HealthCheck of enhanced sdnc is failing. PFA log of the so-sdnc adapter. Any inputs on this would help me resolve my issue. Thanks and Regards, Sowmya L&T Technology Services Ltd www.LTTS.com<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Find01.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__http*3A*2Fwww.LTTS.com__*3B!!BhdT!1Mg16-t-3L-1lkjU_Cg3R_Da7gUg-BKFNNqelZht65qxoaMiQxIO8xT5Zgw7BF4*24*26data*3D04*7C01*7Cnarayanasetty.sowmya*40ltts.com*7C16586f7aea364dc7395e08d89dcfc8a3*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432863038285385*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26sdata*3DFFIqW*2FHV4bKntUPor*2B7bng5lzuc5RhylKoynFMVWizg*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!BhdT!y-gQboiFoUwOucdNF919Xds2yFAv0SUtiyW0_I_iWHVvYgadshYa8_OtELUEGqY*24&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685268517*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=shqIzVVB*2FMw8O05DCJ6Nk9RnsA19JnO9qr3BZF6fXQE*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKiUlKioqKioqKioqKioqJSUqKiolJSUlJSUlJSUlJSUlJSUlJSU!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7BAnb5Do$> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2Find01.safelinks.protection.outlook.com*2F*3Furl*3Dhttps*3A*2F*2Furldefense.com*2Fv3*2F__http*3A*2Fwww.Ltts.com__*3B!!BhdT!1Mg16-t-3L-1lkjU_Cg3R_Da7gUg-BKFNNqelZht65qxoaMiQxIO8xT5YmzWFqo*24*26data*3D04*7C01*7Cnarayanasetty.sowmya*40ltts.com*7C16586f7aea364dc7395e08d89dcfc8a3*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432863038295378*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000*26sdata*3D1FMAps7KOkjh*2BboeByJiAZM3MDRrySR0g7uwfrVUD8g*3D*26reserved*3D0__*3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!BhdT!y-gQboiFoUwOucdNF919Xds2yFAv0SUtiyW0_I_iWHVvYgadshYa8_OtCSmEzOg*24&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685278515*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=RGtGVsTfq2JyagXvactDWb0aqWxA8Vu3ETggbGQtNNo*3D&reserved=0__;JSUlJSUlJSUlJSoqKioqKioqKiUlKioqKioqKioqKioqJSUqKiUlJSUlJSUlJSUlJSUlJSUl!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7Mo-aEWI$>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. L&T Technology Services Ltd www.LTTS.com<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__http*3A*2Fwww.LTTS.com__*3B!!BhdT!y-gQboiFoUwOucdNF919Xds2yFAv0SUtiyW0_I_iWHVvYgadshYa8_Ot5LFXvFM*24&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685278515*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=lg2qykNIxuiGOzLD5OXEXBql0t*2BSs04bP0cUldRFg7c*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on76NyLxJk$> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://urldefense.com/v3/__https:/ind01.safelinks.protection.outlook.com/?url=https*3A*2F*2Furldefense.com*2Fv3*2F__http*3A*2Fwww.Ltts.com__*3B!!BhdT!y-gQboiFoUwOucdNF919Xds2yFAv0SUtiyW0_I_iWHVvYgadshYa8_Oty3HW4wc*24&data=04*7C01*7Cnarayanasetty.sowmya*40Ltts.com*7C8fb6690ce8b84cad324a08d89de2ca40*7C311b33788e8a4b5ea33fe80a3d8ba60a*7C0*7C0*7C637432944685288505*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C1000&sdata=TEM1b7By2Zz*2FYP*2F6Sdzbz82wmEGs*2F7AshrHxEyCSXwI*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7M5lokpM$>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. L&T Technology Services Ltd www.LTTS.com<https://urldefense.com/v3/__http:/www.LTTS.com__;!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7S7-ZsAs$> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://urldefense.com/v3/__http:/www.Ltts.com__;!!BhdT!0cIKeHMRYi6D6Re4gzDRwhL5Nzv3V0pJ_2iGx5X2g3aWMAsQBPLO-on7DpLmuVc$>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22626): https://lists.onap.org/g/onap-discuss/message/22626 Mute This Topic: https://lists.onap.org/mt/78875217/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
