Hi all, I have changed my ves agent by using the 5.x library version and finally my clamp deployment seems to work . In fact now the workflow is the following:
1. Ves agent sends messages to VESCollector
2. VesCollector validates them and publishes into MR
unauthenticated.VES_MEASUREMENT_OUTPUT … example:
{\"event\":{\"commonEventHeader\":{\"startEpochMicrosec\":1565351475324518,\"sourceId\":\"de305d54-75b4-431b-adb2-eb6b9e546014\",\"eventId\":\"mvfs000000004\",\"nfcNamingCode\":\"AFX\",\"reportingEntityId\":\"cc305d54-75b4-431b-adb2-eb6b9e541234\",\"internalHeaderFields\":{\"collectorTimeStamp\":\"Fri,
08 09 2019 11:53:50
UTC\"},\"eventType\":\"applicationVnf\",\"priority\":\"Normal\",\"version\":3,\"reportingEntityName\":\"ibcx0001vm002oam001\",\"sequence\":0,\"domain\":\"measurementsForVfScaling\",\"lastEpochMicrosec\":1565351495566922,\"eventName\":\"vLoadBalancer\",\"sourceName\":\"scfx0001vm002cap001\",\"nfNamingCode\":\"AFX\"},\"measurementsForVfScalingFields\":{\"cpuUsageArray\":[{\"percentUsage\":0,\"cpuIdentifier\":\"Cpu1\",\"cpuIdle\":0,\"cpuUsageSystem\":0,\"cpuUsageUser\":0}],\"measurementInterval\":20,\"requestRate\":0,\"measurementsForVfScalingVersion\":2,\"vNicPerformanceArray\":[{\"transmittedOctetsDelta\":1415,\"receivedTotalPacketsDelta\":3,\"vNicIdentifier\":\"enp0s3\",\"valuesAreSuspect\":\"true\",\"transmittedTotalPacketsDelta\":3,\"receivedOctetsDelta\":287}]}}}"
3. CLAMP deployment read the messages and if the receivedTotalPacketsDelta
> threshold , publishes the message into MR unauthenticated.DCAE_CL_OUTPUT ….
"{\"closedLoopEventClient\":\"DCAE_INSTANCE_ID.dcae-tca\",\"policyVersion\":\"1.0.2\",\"policyName\":\"vlbpolicy\",\"policyScope\":\"vlbclosedloop\",\"target_type\":\"VM\",\"AAI\":{\"vserver.vserver-name\":\"scfx0001vm002cap001\"},\"closedLoopAlarmStart\":1565351495566922,\"closedLoopEventStatus\":\"ONSET\",\"closedLoopControlName\":\"LOOP_vLoadBalancer_v1_0_vLoadBalancer0_k8s-tca-clamp-policy-05162019\",\"version\":\"1.0.2\",\"target\":\"vserver.vserver-name\",\"requestID\":\"8272300a-7239-46b5-ab28-0bd9b9c826cc\",\"from\":\"DCAE\"}"]
Now, I want to understand all steps from this point to the end of the clamp
procedure.
Which onap component is listening on the DCAE_CL_OUTPUT topic and sends the
command to activate the policy (vlbpolicy in my case) retro-action associated
to the clamp deployment?
Thanks,
Aniello Paolo Malinconico
Aniello Paolo Malinconico
Net Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
[email protected]<mailto:[email protected]>
www.reply.it<http://www.reply.it> <http://>
[Net Reply]
Da: VENKATESH KUMAR, VIJAY <[email protected]>
Inviato: giovedì 8 agosto 2019 23:26
A: Malinconico Aniello Paolo <[email protected]>; PLATANIA, MARCO
<[email protected]>; [email protected]
Cc: MALAKOV, YURIY <[email protected]>; D'Alessandro Alessandro Gerardo
<[email protected]>
Oggetto: RE: Clamp Clarifications
HI,
The VES 7.x field structure has other differences beside versions hence there
may be more fields to tweak.
Following is 5.4 schema definition - you may want to change your agent/source
to be complaint to this schema.
https://git.onap.org/dcaegen2/collectors/ves/tree/etc/CommonEventFormat_28.4.1.json
An example 5.4 complaint event is below for reference.
{"commonEventHeader":{"startEpochMicrosec":1565298941556623,"eventId":"mvfs00000001","sequence":0,"domain":"measurementsForVfScaling","lastEpochMicrosec":1565298951556623,"eventName":"vFirewallBroadcastPackets","reportingEntityId":"No
UUID
available","sourceName":"vofwl01fwl9a3a","priority":"Normal","version":3,"reportingEntityName":"vofwl01fwl9a3a"},"measurementsForVfScalingFields":{"measurementInterval":10,"measurementsForVfScalingVersion":2,"vNicPerformanceArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":503,"vNicIdentifier":"eth1","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":21586}]}}
Thanks,
Vijay
From: Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>
Sent: Thursday, August 08, 2019 11:25 AM
To: VENKATESH KUMAR, VIJAY <[email protected]<mailto:[email protected]>>; PLATANIA,
MARCO <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>
Cc: MALAKOV, YURIY <[email protected]<mailto:[email protected]>>; D'Alessandro
Alessandro Gerardo
<[email protected]<mailto:[email protected]>>
Subject: R: Clamp Clarifications
Hi all, thanks for replying.
How can I change the VESsource to 5.4?
In my ves agent message there are 2 fields about the version :
“version”: “4.1”,
“vesEventListenerVersion”: “7.1”
Should I change the last one from 7.1 to 5.4?
Thanks a lot
Aniello Paolo Malinconico
Net Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
[email protected]<mailto:[email protected]>
www.reply.it<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.reply.it&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=_PKTq53Zmjn0WmTnCkdUL5ztduOJW-0GC6jF_wzGm4A&s=7PY8gpHvBz8LvN-V24KWHtVqM0KEXvUDqIf2T7E8mjs&e=>
[Net Reply]
Da: VENKATESH KUMAR, VIJAY <[email protected]<mailto:[email protected]>>
Inviato: giovedì 8 agosto 2019 16:40
A: PLATANIA, MARCO
<[email protected]<mailto:[email protected]>>; Malinconico
Aniello Paolo <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>
Cc: MALAKOV, YURIY <[email protected]<mailto:[email protected]>>; D'Alessandro
Alessandro Gerardo
<[email protected]<mailto:[email protected]>>
Oggetto: RE: Clamp Clarifications
Hi Aniello/Marco, It looks like the errors from TCA pod are due to processing
VES7.x events; TCA support only 5.4 VES.
{"event":{"commonEventHeader":{"startEpochMicrosec":1565191901690446,"sourceId":"de305d54-75b4-431b-adb2-eb6b9e546014","eventId":"mvfs000000452","nfcNamingCode":"AFX","timeZoneOffset":"UTC-05:30","reportingEntityId":"cc305d54-75b4-431b-adb2-eb6b9e541234","internalHeaderFields":{"collectorTimeStamp":"Wed,
08 07 2019 03:33:53
UTC"},"eventType":"applicationVnf","priority":"Normal","version":"4.0.1","nfVendorName":"AT&T","reportingEntityName":"ibcx0001vm002oam001","sequence":0,"domain":"measurement","lastEpochMicrosec":1565191922200736,"eventName":"vLoadBalancer","vesEventListenerVersion":"7.0.1","sourceName":"scfx0001vm002cap001","nfNamingCode":"AFX"},"measurementFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"Cpu1","cpuIdle":0,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":20,"measurementFieldsVersion":"4.0","requestRate":0,"nicPerformanceArray":[{"transmittedOctetsDelta":7680,"receivedTotalPacketsDelta":96,"nicIdentifier":"lo","valuesAreSuspect":"true","transmittedTotalPacketsDelta":96,"receivedOctetsDelta":7680},{"transmittedOctetsDelta":4584,"receivedTotalPacketsDelta":48,"nicIdentifier":"enp0s3","valuesAreSuspect":"true","transmittedTotalPacketsDelta":48,"receivedOctetsDelta":4584},{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":0,"nicIdentifier":"docker0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":0}]}}}
If you can switch VESsource to 5.4, I believe CL flow through DCAE/TCA should
be okay. Btw, we do also have option to enable transformation on VESCollector
pod to change 7x input to 5x o/p – but need to dig into documentation. Let me
know if that something you would like to try.
Thanks,
Vijay
From: PLATANIA, MARCO
<[email protected]<mailto:[email protected]>>
Sent: Thursday, August 08, 2019 8:36 AM
To: Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>; VENKATESH
KUMAR, VIJAY <[email protected]<mailto:[email protected]>>
Cc: MALAKOV, YURIY <[email protected]<mailto:[email protected]>>; D'Alessandro
Alessandro Gerardo
<[email protected]<mailto:[email protected]>>
Subject: Re: Clamp Clarifications
+ Vijay.
I believe this is a DCAE thing, you may want to post the Consul configuration
here.
Regarding the vServer name, heatbridge writes it to AAI when running that
script after VNF instantiation. You can look at the documentation for
instructions about how to run heatbridge.
Marco
From: Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>
Date: Thursday, August 8, 2019 at 8:14 AM
To: "PLATANIA, MARCO (MARCO)"
<[email protected]<mailto:[email protected]>>,
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Cc: "MALAKOV, YURIY" <[email protected]<mailto:[email protected]>>, D'Alessandro
Alessandro Gerardo
<[email protected]<mailto:[email protected]>>
Subject: R: Clamp Clarifications
Hi Marco,
My VES agent is similar to the VESreporting_vFW .
I have no changed the sourceName, in fact it is scfx0001vm002cap001 .
Is this value custom (I think yes, the only constraint is that the name into
sourceName variable must exists into AAI, right?)
How can I add this information to the AAI?
However, I think the closed loop (at least the sub/pub phase) should continue
to work but fails in my case….
Can you have a look to the attached log file?
The log file refers to the POD
dep-s8583ff5e422342558ad8c7b924e334f9-dcaegen2-analytics-thlt8b that is the pod
carried out from the CLAMP deployment. It reports some errors about the version
value 4.0.1 that is into a InvalidFormat.
The second attached file is my tca.yaml configuration.
Thanks a lot,
Aniello Paolo Malinconico
Aniello Paolo Malinconico
Net Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
[email protected]<mailto:[email protected]>
www.reply.it<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.reply.it&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=Fn_8J8QAB75GTKbcqt3DQuDiFB0JbFdJfLPXkYWEjVI&s=450nMUMvXydiJfYRIQ8QZ-xZ8ufuBvBEzIJrDmhXUd8&e=>
[Net Reply]
Da: PLATANIA, MARCO (MARCO)
<[email protected]<mailto:[email protected]>>
Inviato: mercoledì 7 agosto 2019 19:42
A: Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>
Cc: MALAKOV, YURIY <[email protected]<mailto:[email protected]>>
Oggetto: Re: Clamp Clarifications
The procedure seems correct, but considering that you aren’t using a real VNF,
something may be lost. For example, is your VES client reporting the source
name? This would be the name of the vServer in which the VES agent runs when
deployed in a real VNF. If so, is this vServer name in AAI? If not, DCAE fails
the enrichment, although the closed loop should still go on.
Topics are created when needed, so if unauthenticated.DCAE_CL_OUTPUT isn’t
there is because no message has been published in that topic yet. Can you post
the TCA configuration from Consul?
Marco
From: Malinconico Aniello Paolo
<[email protected]<mailto:[email protected]>>
Date: Wednesday, August 7, 2019 at 9:56 AM
To: "PLATANIA, MARCO (MARCO)"
<[email protected]<mailto:[email protected]>>,
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Cc: "MALAKOV, YURIY" <[email protected]<mailto:[email protected]>>
Subject: Clamp Clarifications
Hi Marco,
first of all thanks for your help to understand and improve the ONAP framework.
I am having some doubts about the CLAMP working cycle. I explain better below:
I am following the VF Scale Out Use Case here :
https://docs.onap.org/en/latest/submodules/integration.git/docs/docs_scaleout.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.onap.org_en_latest_submodules_integration.git_docs_docs-5Fscaleout.html&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=pW6gGbKyrKXrSi0L644R1zJpiwlM7fcTQy2EmwLx4ho&e=>
The deployment part, after some troubles, now works fine: all VM have been
installed and configured correctly .
Now, I am focusing my attention on the CLAMP deploymrnts part.
In particulare I have created a clamp deployment as you done here
https://wiki.onap.org/display/DW/2019-07-31+Control+Loop+Sub+Committee+Weekly+Meeting?preview=/68539751/68539752/zoom_0.mp4<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_display_DW_2019-2D07-2D31-2BControl-2BLoop-2BSub-2BCommittee-2BWeekly-2BMeeting-3Fpreview-3D_68539751_68539752_zoom-5F0.mp4&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=rXRwLIvwdLMRl3nSrgrVMNlbZFOJmf8MJJdt807gJeU&e=>
My clamp deployment is very similar to yours. The event name associated to my
clamp deployment is vLoadBalancer and the policy works if the threshold on
receiveidTotalPacketDelta > 50 .
After deploying it, a new pod has came up
dep-s8583ff5e422342558ad8c7b924e334f9-dcaegen2-analytics-thlt8b .
In my understanding this clamp deployment listens on the
unauthenticated.VES_MEASUREMENT_OUTPUT at eventName vLoadBalancer and if the
policy is accepted (receiveidTotalPacketDelta > 50), it publishes the event
(that will perform the closed loop i.e. command to run the scale out operation
via SO) into the unauthenticated.DCAE_CL_OUTPUT. Right?
So, to test it (only the publishing/subscribing events via clamp deployments)
without deploying the VMs, I have written a simple VES Agent similar to the
vFirewall’s one that sends the messages with eventName vLoadBalancer to the VES
Collector that validates them and publishes them into the Message Router at
unauthenticated.VES_MEASUREMENT_OUTPUT topic. If the validation fails the
message is dropped.
I am sure my ves agent works because if I perform the GET request
http://{{k8s}}:30227/events/unauthenticated.VES_MEASUREMENT_OUTPUT/n3ll1x93/1<https://urldefense.proofpoint.com/v2/url?u=http-3A__-257b-257bk8s-257d-257d-3A30227_events_unauthenticated.VES-5FMEASUREMENT-5FOUTPUT_n3ll1x93_1&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=eSMzsDMu64_B-5RN4W_emOxdIztkFnDmOcBQZJbJYtM&e=>
, there are multiple messages sent by my ves agent with the parameter
receiveidTotalPacketDelta that has a value greater than 50, so the policy
should be triggered.
To see if the clamp deployment works or not, I perform the GET request from
http://{{k8s}}:30227/events/unauthenticated.DCAE_CL_OUTPUT/OpenDCAE-clamp/c13<https://urldefense.proofpoint.com/v2/url?u=http-3A__-257b-257bk8s-257d-257d-3A30227_events_unauthenticated.DCAE-5FCL-5FOUTPUT_OpenDCAE-2Dclamp_c13&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=eGz5gHYn0Wlr-DoMC4_2JSzRnsn1kdbBc4dINaIWv28&e=>
, buti t returns the following messages:
It returns
{"mrstatus":3001,"helpURL":"http://onap.readthedocs.io<https://urldefense.proofpoint.com/v2/url?u=http-3A__onap.readthedocs.io_&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=Nb1DTdHK_s681aWg37i58olcby-tv2ikrB2Nde9avl8&e=>","message":"No
such topic exists.-[unauthenticated.DCAE_CL_OUTPUT]","status":404}.
In fact if I check all existing topics
http://{{k8s}}:30227/topics/<https://urldefense.proofpoint.com/v2/url?u=http-3A__-257b-257bk8s-257d-257d-3A30227_topics_&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=Fd8Cy9eyxHa82_fbLjXKeQfto1DeuSxCSLrokEfTL3c&e=>
, they are :
{
"topics": [
"org.onap.dmaap.mr.PNF_REGISTRATION",
"unauthenticated.SEC_HEARTBEAT_OUTPUT",
"AAI-EVENT",
"SDC-DISTR-STATUS-TOPIC-AUTO",
"SDC-DISTR-NOTIF-TOPIC-AUTO",
"org.onap.dmaap.mr.PNF_READY",
"POLICY-PDP-PAP",
"POA-AUDIT-RESULT",
"org.onap.dmaap.mr.PM_MAPPER",
"POA-AUDIT-INIT",
"unauthenticated.VES_MEASUREMENT_OUTPUT",
"org.onap.dmaap.mr.mirrormakeragent",
"__consumer_offsets",
"POA-RULE-VALIDATION"
]
}
And there is any topic called unauthenticated.DCAE_CL_OUTPUT .
So should I create this topic unauthenticated.DCAE_CL_OUTPUT manually?
Or something is wrong during my clamp configuration? How can I test the Clamp
deployment correctly?
Thanks a lot,
Aniello Paolo Malinconico
Aniello Paolo Malinconico
Net Reply
Via Cardinal Massaia, 83
10147 - Torino - ITALY
phone: +39 011 29100
[email protected]<mailto:[email protected]>
www.reply.it<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.reply.it&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=KgFIQiUJzSC0gUhJaQxg8eC3w16GC3sKgWIcs4iIee0&m=4UKmMXMQKyhOcMT4YhMFPpMoBW7UbBAToQJYjUMu37I&s=drVT0Q1UpI8TUtLIZFuwJbwYwV5PyrjnRyQIPqN_kOI&e=>
[Net Reply]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18500): https://lists.onap.org/g/onap-discuss/message/18500
Mute This Topic: https://lists.onap.org/mt/32809787/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
net.png
Description: net.png
