Hi Mir,
here is Marco. I’m working with Aniello.
Yes we excuteted the heatbridge command.
Here is the command we have sent.
./demo-k8s.sh onap heatbridge
RegionOne_ONAP-NF_20190821T135252709Z_vlb_Expansion_001 50276f1b-97ed-4
59d-a6b0-9c1c689be64d vLB 10.0.101.19
We will follow the Platania’s direction and we will check the settings in the
CDT regarding heathcheck.
Do you solved your problem about the PUT in the AAI that returns a response
which is not well understood ?
------------------------------------------------------------------
TIM
Marco Angelo Ferrero
Mobile & Fixed Innovation
IP, Transport & Core Innovation
TIM S.p.A.
Via G. Reiss Romoli, 274 – 10148 TORINO
+39 011 2286739
+39 331 6001354
Da: Mir, Adil Kamal [mailto:[email protected]]
Inviato: martedì 3 settembre 2019 17:02
A: [email protected]; [email protected]; PLATANIA, MARCO
(MARCO); Ferrero Marco Angelo; HERNANDEZ-HERRERO, JORGE
Cc: D'Alessandro Alessandro Gerardo; Signorelli Marco
Oggetto: [EXT] RE: [onap-discuss] R: VF Module Scale Out Use Case - questions
about closed-loop chain.
Hi Aniello, did you run the heatbridge? – that’s what puts the OAM IP for the
vlb in AAI. You can run it using robot or just manually try putting the OAM IP
in your AAI and see if that solves the issue.
From: <[email protected]> on behalf of Aniello Paolo Malinconico
<[email protected]>
Reply-To: "[email protected]" <[email protected]>,
"[email protected]" <[email protected]>
Date: Tuesday, September 3, 2019 at 10:05 AM
To: "PLATANIA, MARCO (MARCO)" <[email protected]>, Ferrero Marco Angelo
<[email protected]>, "[email protected]"
<[email protected]>, "HERNANDEZ-HERRERO, JORGE" <[email protected]>
Cc: D'Alessandro Alessandro Gerardo <[email protected]>,
Signorelli Marco <[email protected]>
Subject: [EXT][onap-discuss] R: VF Module Scale Out Use Case - questions about
closed-loop chain.
Hi all,
to unlock the “clamp lock”, you need to delete the entry into
infra_active_request table for the requestdb database.
Once deleted, it should continue to work.
In fact, now the SO starts the bpmn for the scaleout action, but I have still
some problems with the APPC.
In few works:
When the controlloop is activated, it sends a scaleout request to the SO which
starts the bpmn "WorkflowBB".
This workflow is composed by 5 or 6 Building Blocks (Healthcheck, Create
VFmodule, Assign, Activate, Healthcheck).
However, the first BB is an healthcheck operation, managed by the APPC
controller.
So the APPC makes some GET queries against the AAI to retrieve some
information, for example the oam vlb ip for the healthcheck.
For example (from appc karaf log):
2019-09-03T13:37:54,109 | INFO | appc-dispatcher-3 | AAIService
| 460 - org.onap.ccsdk.sli.adaptors.aai-service-provider - 0.4.1 |
Request URL :
https://aai.onap:8443/aai/v14/network/vnfcs/vnfc/RegionOne_ONAP-NF_20
190821T135252709Z_vlb_001
2019-09-03T13:37:54,137 | INFO | appc-dispatcher-3 | AAIService
| 460 - org.onap.ccsdk.sli.adaptors.aai-service-provider - 0.4.1 |
HttpURLConnection result: 200 : OK
2019-09-03T13:37:54,137 | INFO | appc-dispatcher-3 | metric
| 205 - org.onap.ccsdk.sli.core.sli-common - 0.4.1 |
2019-09-03T13:37:54,143 | INFO | appc-dispatcher-3 | AAIService
| 460 - org.onap.ccsdk.sli.adaptors.aai-service-provider - 0.4.1 |
Response code : 200, OK
2019-09-03T13:37:54,144 | INFO | appc-dispatcher-3 | AAIService
| 460 - org.onap.ccsdk.sli.adaptors.aai-service-provider - 0.4.1 |
Response data :
{"vnfc-name":"RegionOne_ONAP-NF_20190821T135252709Z_vlb_001","nfc-n
aming-code":"vLB","nfc-function":"vLB","in-maint":false,"is-closed-loop-disabled":false,"resource-version":"1566468976695","relationship-list":{"relationship":[{"related-to":"vserver","relationship-label":"tosca.relationships.HostedOn","
related-link":"/aai/v14/cloud-infrastructure/cloud-regions/cloud-region/CloudOwner/RegionOne/tenants/tenant/a9d84b4e2c1e47bfaa4d26d2d894c9c2/vservers/vserver/ccadfb9d-0386-43d5-b007-887ec9ce3fcc","relationship-data":[{"relationship-key":
"cloud-region.cloud-owner","relationship-value":"CloudOwner"},{"relationship-key":"cloud-region.cloud-region-id","relationship-value":"RegionOne"},{"relationship-key":"tenant.tenant-id","relationship-value":"a9d84b4e2c1e47bfaa4d26d2d894c
9c2"},{"relationship-key":"vserver.vserver-id","relationship-value":"ccadfb9d-0386-43d5-b007-887ec9ce3fcc"}],"related-to-property":[{"property-key":"vserver.vserver-name","property-value":"RegionOne_ONAP-NF_20190821T135252709Z_vlb_001"}]
}]}}
2019-09-03T13:37:54,156 | INFO | appc-dispatcher-3 | AaiService
| 493 - org.onap.appc.aai.client - 1.5.2 | AAIResponse: SUCCESS
2019-09-03T13:37:54,157 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | SORTED VSERVERS
[{vnfc-count=1, vnfc-name=RegionOne_ONAP-NF_20190821T135252709Z_vlb_001,
cloud-region-id=Re
gionOne,
vserver-selflink=http://163.162.95.137:8774/v2.1/a9d84b4e2c1e47bfaa4d26d2d894c9c2/servers/ccadfb9d-0386-43d5-b007-887ec9ce3fcc,
tenant-id=a9d84b4e2c1e47bfaa4d26d2d894c9c2, group-notation=null,
cloud-owner=CloudOwner, vserver-nam
e=RegionOne_ONAP-NF_20190821T135252709Z_vlb_001,
vnfc-ipaddress-v4-oam-vip=null,
vf-module-id=6e3b7f98-f87f-4aa7-b2eb-f1a8b0ed85d9, vnfc-type=vLB,
vnfc-function-code=vLB, vserver-id=ccadfb9d-0386-43d5-b007-887ec9ce3fcc}]
2019-09-03T13:37:54,159 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Final Context
2019-09-03T13:37:54,160 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vnfc-count Value = 1
2019-09-03T13:37:54,162 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vnfc-name Value =
RegionOne_ONAP-NF_20190821T135252709Z_vlb_001
2019-09-03T13:37:54,167 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].cloud-region-id Value = RegionOne
2019-09-03T13:37:54,168 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vserver-selflink Value =
http://163.162.95.137:8774/v2.1/a9d84b4
e2c1e47bfaa4d26d2d894c9c2/servers/ccadfb9d-0386-43d5-b007-887ec9ce3fcc
2019-09-03T13:37:54,170 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].tenant-id Value = a9d84b4e2c1e47bfaa4d26d2d894c9c2
2019-09-03T13:37:54,172 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].group-notation Value = null
2019-09-03T13:37:54,174 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].cloud-owner Value = CloudOwner
2019-09-03T13:37:54,175 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vserver-name Value =
RegionOne_ONAP-NF_20190821T135252709Z_vlb_0
01
2019-09-03T13:37:54,175 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vnfc-ipaddress-v4-oam-vip Value = null
2019-09-03T13:37:54,176 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vf-module-id Value = 6e3b7f98-f87f-4aa7-b2eb-f1a8b0ed85d9
2019-09-03T13:37:54,176 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vnfc-type Value = vLB
2019-09-03T13:37:54,176 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vnfc-function-code Value = vLB
2019-09-03T13:37:54,176 | INFO | appc-dispatcher-3 | AAIResourceNode
| 493 - org.onap.appc.aai.client - 1.5.2 | Populating Context Key =
tmp.vnfInfo.vm[0].vserver-id Value = ccadfb9d-0386-43d5-b007-887ec9ce3fcc
As you can see, some information are missing from the AAI response (i.e.
oam-vlb ip is missed).
Once retrieve the oam vlb ip, the APPC tries to make an healthcheck operation
versus the vLB VM by making:
http://null:8183/restconf/operational/health-vnf-onap-plugin:health-vnf-onap-plugin-state/health-check
Of course, this this request fails because the ip is missing.
And so the APPC fails returning to the SO bpmn the following error message:
2019-09-03T13:37:54,471 | ERROR | appc-dispatcher-3 | RestServiceNode
| 486 - org.onap.appc.flow.controller - 1.5.2 | Error Message : Error
While Sending Rest Request
java.lang.Exception: Error While Sending Rest Request
at
org.onap.appc.flow.controller.executorImpl.RestExecutor.execute(RestExecutor.java:81)
~[?:?]
at
org.onap.appc.flow.controller.node.RestServiceNode.sendRequest(RestServiceNode.java:86)
~[?:?]
Can anyone help me? Any workaround to solve this problem?
Thanks,
Aniello Paolo Malinconico
Aniello Paolo Malinconico
Net Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
[email protected]<mailto:[email protected]>
www.reply.it
[Net Reply]
Da: PLATANIA, MARCO (MARCO) <[email protected]>
Inviato: lunedì 2 settembre 2019 18:07
A: Ferrero Marco Angelo <[email protected]>;
[email protected]; HERNANDEZ-HERRERO, JORGE <[email protected]>
Cc: Malinconico Aniello Paolo <[email protected]>; D'Alessandro Alessandro
Gerardo <[email protected]>; Signorelli Marco
<[email protected]>
Oggetto: Re: VF Module Scale Out Use Case - questions about closed-loop chain.
Policy doesn’t execute closed loop for each received event. For a given event,
Policy runs one closed loop every X minutes, this is why the VNF is locked. At
the expiration of the timeout (lock), the closed loop is executed again. I
can’t remember where locks are kept, Jorge can help with that. Guard policies
are pushed by the user from CLAMP typically (unless you call Policy API
directly), so they do what the user tells them to do. Note however that for
scale out, Policy doesn’t call APPC, it calls SO (which later on calls APPC for
VNF health check and reconfiguration). I’m not sure whether your configuration
reflects this. You shouldn’t expect Policy to publish in APPC-LCM-READ.
Marco
From: Ferrero Marco Angelo
<[email protected]<mailto:[email protected]>>
Date: Monday, September 2, 2019 at 11:13 AM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>, "PLATANIA,
MARCO (MARCO)" <[email protected]<mailto:[email protected]>>,
"PLATANIA, MARCO (MARCO)"
<[email protected]<mailto:[email protected]>>,
"HERNANDEZ-HERRERO, JORGE" <[email protected]<mailto:[email protected]>>
Cc: "Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>
([email protected]<mailto:[email protected]>)"
<[email protected]<mailto:[email protected]>>, D'Alessandro
Alessandro Gerardo
<[email protected]<mailto:[email protected]>>,
Signorelli Marco
<[email protected]<mailto:[email protected]>>
Subject: VF Module Scale Out Use Case - questions about closed-loop chain.
Hi everybody,
We are testing the VF Module Scale Out use case.
Our vLB correctly sends data to the VES Collector and it correctly publishes
data on DMaaP topic unauthenticated.VES_MEASUREMENT_OUTPUT.
TCA correctly read data from this topic and, as the threshold is violated,
correctly publishes data on DMAAP topic unauthenticated.DCAE_CL_OUTPUT
Policy Drools read data from this topic and applies frequency guard but it does
not propagate the event even if it seems send evets to DMaap topic POLICY-CL-MGT
Who consumes messages from this topic (POLICY-CL-MGT) and who writes to topic
APPC-LCM-READ in order to propagate the commands to the APPC ?
Accessing to network.log file of policy drools we have found the following
messages :
[2019-08-31T00:37:54.089+00:00|Session
org.onap.policy.drools-applications.controlloop.common:controller-usecases:1.4.2:usecases][OUT|
{
"AAI": {
"vserver.prov-status": "ACTIVE",
"vserver.resource-version": "1566468980498",
"vserver.is-closed-loop-disabled": "false",
"vserver.vserver-name2": "RegionOne_ONAP-NF_20190821T135252709Z_vlb_001",
"vserver.vserver-id": "ccadfb9d-0386-43d5-b007-887ec9ce3fcc",
"vserver.vserver-selflink":
"http://163.162.95.137:8774/v2.1/a9d84b4e2c1e47bfaa4d26d2d894c9c2/servers/ccadfb9d-0386-43d5-b007-887e<https://urldefense.proofpoint.com/v2/url?u=http-3A__163.162.95.137-3A8774_v2.1_a9d84b4e2c1e47bfaa4d26d2d894c9c2_servers_ccadfb9d-2D0386-2D43d5-2Db007-2D887e&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=9F0hi2lvf516P4W9z1pFsP2SQyQELSaXXduf8XCZUR0&s=a66daKjKvJGjrCVBjQAagtnJGP6GkDySc0AOFhJkPRU&e=>
"vserver.in-maint": "false",
"vserver.vserver-name": "RegionOne_ONAP-NF_20190821T135252709Z_vlb_001"
},
"closedLoopAlarmStart": 1567211854626801,
"closedLoopControlName":
"LOOP_vLoadBalancer_20190821_v1_0_vLoadBalancer_201908210_k8s-tca-clamp-policy",
"version": "1.0.2",
"requestId": "2d530ae5-cdc5-41a3-83fe-0166431d26e7",
"closedLoopEventClient": "DCAE_INSTANCE_ID.dcae-tca",
"targetType": "VM",
"target": "vserver.vserver-name",
"from": "policy:usecases",
"policyScope": "onap.policies.controlloop.Operational:1.0.0",
"policyName":
"OPERATIONAL_vLoadBalancer_20190821_v1_0_vLoadBalancer_201908210_k8s-tca-clamp-policy.EVENT.MANAGER",
"policyVersion": "1.0.0",
"notification": "REJECTED",
"message": "The target RegionOne_ONAP-NF_20190821T135252709Z_vlb_001 is
already locked",
"notificationTime": "2019-08-31 00:37:54.089000+00:00",
"history": []
}
[2019-08-31T00:38:08.124+00:00|qtp1534754611-39]10.42.10.195 - -
[31/Aug/2019:00:38:08 +0000] "GET /healthcheck HTTP/1.1" 200 102
root@cp-1:~/oom/kubernetes# 00|qtp1534754611-35]10.42.10.195 - -
[31/Aug/2019:00:38:23 +0000] "GET /healthcheck HTTP/1.1" 200 102
It seems that we vserver entry in the AAI (vserver relative to the vLB) is
locked ?
Why it is locked ?
Who locks it ?
Where are kept the locks ?
I have also accessed the policy db
root@dev-policy-policydb-868884d57f-g4h9g:/# mysql -uroot -psecret
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 776820
Server version: 10.2.14-MariaDB-10.2.14+maria~jessie mariadb.org binary
distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+---------------------+
| Database |
+---------------------+
| information_schema |
| log |
| migration |
| mysql |
| onap_sdk |
| operationshistory |
| operationshistory10 |
| performance_schema |
| policyadmin |
| pooling |
| support |
+---------------------+
11 rows in set (0.02 sec)
And I have checked the operationhistory schema :
I made the resume query
MariaDB [operationshistory]> select count(*),message from operationshistory
group by message;
+----------+---------------------------+
| count(*) | message |
+----------+---------------------------+
| 1474 | 999 Failed |
| 11411 | Operation denied by Guard |
+----------+---------------------------+
2 rows in set (0.03 sec)
And I always got ‘999 Failed’ every time the guard let it me execute the policy.
Who send this code ?
Where I can get the log of the policy executor that generate this message ?
Thank you in advance for your precious help,
Marco Ferrero
------------------------------------------------------------------
TIM
Marco Angelo Ferrero
Mobile & Fixed Innovation
IP, Transport & Core Innovation
TIM S.p.A.
Via G. Reiss Romoli, 274 – 10148 TORINO
+39 011 2286739
+39 331 6001354
[https://img.tim.it:443/sdr/banner-mail-dipe.jpg]<https://urldefense.proofpoint.com/v2/url?u=https-3A__on.tim.it_banner-2Dmail-2Ddip&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=9F0hi2lvf516P4W9z1pFsP2SQyQELSaXXduf8XCZUR0&s=4U5QxZWZQCB1NrAZBIB5Jfnzl7eOlV1Oc2DkNq1QtUs&e=>
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone
indicate. La diffusione, copia o qualsiasi altra azione derivante dalla
conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate
ricevuto questo documento per errore siete cortesemente pregati di darne
immediata comunicazione al mittente e di provvedere alla sua distruzione,
Grazie.
This e-mail and any attachments is confidential and may contain privileged
information intended for the addressee(s) only. Dissemination, copying,
printing or use by anybody else is unauthorised. If you are not the intended
recipient, please delete this message and any attachments and advise the sender
by return e-mail, Thanks.
Rispetta l'ambiente. Non stampare questa mail se non è necessario.
________________________________
External Email: Please use caution when opening links and attachments /
Courriel externe: Soyez prudent avec les liens et documents joints
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18802): https://lists.onap.org/g/onap-discuss/message/18802
Mute This Topic: https://lists.onap.org/mt/33128103/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
