Hi Mytnyk,
Have an openstack installation here where libvirt-bin is installed
But libvirt-dev is not.
Had problems compile and install on openstack controller with libvirt 3.6.0
Are there updated instructions for libvirt.
ii contrail-heat 3.2.3.0-38
all OpenStack virtual network service - Opencontrail
ii erlang-base-hipe 1:18.1-1~u14.04+mos3
amd64 Erlang/OTP HiPE enabled virtual machine and base applications
ii libvirt-bin 1.2.9.3-9~u14.04+mos10
amd64 programs for the libvirt library
Have compiled collectd with kafka plugin on separate vm with instructions
provided.
Thanks
Gokul
From: SULLIVAN, BRYAN L
Sent: Tuesday, November 21, 2017 10:37 AM
To: Mytnyk, VolodymyrX <[email protected]>; Tahhan, Maryam
<[email protected]>
Cc: Mcmahon, Tony B <[email protected]>; Power, Damien
<[email protected]>; GUPTA, ALOK <[email protected]>; SINGARAJU, GOKUL
<[email protected]>; '[email protected]'
<[email protected]>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Thanks for the additional info. The detailed questions I had were on how to
build the domain (VM) XML configuration file outside of an OpenStack
environment. While VES will support OpenStack environments and be tested with
them, for non-OpenStack environments (e.g. Kubernetes) is there any dependence
on the virt plugin for collectd? Our main focus going forward (beyond
supporting our current OpenStack-based platform, AIC) is on cloud-native stacks
that do not include OpenStack or deploy apps/VNFs in VMs.
For OpenStack environments, the info you provided helps, but I will need to
test it. The issue I was having was that though I followed all the steps except
for creating the domain (VM) XML configuration file, the virt plugin was not
installed and thus collectd failed to find it and exited. If you're saying that
just because OpenStack compute code will have been installed prior (for
OpenStack hosts), the virt plugin will be installed (or the "make install"
process for collectd will find the present OpenStack domain (VM) XML
configuration files, and complete the job of installing the virt plugin...),
then OK and I will verify that.
Thanks,
Bryan Sullivan | AT&T
From: Mytnyk, VolodymyrX [mailto:[email protected]]
Sent: Tuesday, November 21, 2017 7:22 AM
To: SULLIVAN, BRYAN L (BRYAN L)
<[email protected]<mailto:[email protected]>>;
Tahhan, Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>; GUPTA, ALOK
<[email protected]<mailto:[email protected]>>; SINGARAJU, GOKUL
<[email protected]<mailto:[email protected]>>; '[email protected]'
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Hi Bryan,
To make collectd virt plugin work on a host, the libvirt library needs to be
installed on the system/host (virt
plugin<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_feature.userguide.html-23virt-2Dplugin&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=2HmUI80GTO-P-98xNvo_veQ3wEk3m2oCFz4TLHkg7bY&e=>,
` Dependencies: `). If the collectd is running in the OpenStack environment
(e.g. on compute), the dependencies for libvirtd should be installed
automatically during OpenStack installation. It means no other dependencies
needs to be installed for collectd virt plugin. Only install
collectd<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_collectd.ves.userguide.html-23install-2Dcollectd&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=Vcp88USCgBSGIMg5tP3gBe0OehWksYdDNv_E-_3P9G0&e=>
steps needs to be performed to install collectd on a host for the VES
purposes. The following commands can be used on a host to make sure that
libvirt dependencies have been installed by OpenStack:
$ sudo dpkg -l | grep virt
...
ii libvirt-dev:amd64 2.5.0-3ubuntu5.5~cloud0
amd64 development files for the libvirt library
...
If the collectd is running in non OpenStack environment, the libvirt should be
installed manually (via `sudo apt-get install libvirt-dev`) or from sources
(see virt
plugin<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_feature.userguide.html-23virt-2Dplugin&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=2HmUI80GTO-P-98xNvo_veQ3wEk3m2oCFz4TLHkg7bY&e=>
`Install libvirt` & ` install the dependencies: ` for more details how to
build from sources). It depends on what features/extended statistics you want
to receive via libvirt (see Extended statistics
requirements<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_feature.userguide.html-23id9&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=NRfkRsR7WyxlPs9ysUo4zFJfMb0u7Wf041eSEMAA4XA&e=>
section for more details). The extended statistics (if libvirt version
supports them) are mapped into `additionMeasurenmets` filed of the VES
message. From the VES app perspective, it shouldn't be difference what
statistics are exposed by libvirt & virt plugin (e.g. what version of libvirt
is installed and used).
Regarding the `virsh tool`. This tool is used to create and manage VM manually
via libvirt and CLI. As I understand, you are running collectd in OpenStack
environment, so you don't need to use this method. The OpenStack environment
should be used instead to create/manage VMs (OpenStack uses libvirt to manage
VMs).
If you receive collectd errors anyway during running collectd, please send me
the collectd logs and config.log from the source directory (the config.log are
created automatically after running `configure ...` command in the collectd
source tree), so I will try to figure out the problem.
Thanks and Regards,
Volodymyr
From: SULLIVAN, BRYAN L (BRYAN L) [mailto:[email protected]]
Sent: Tuesday, November 21, 2017 1:52 AM
To: Mytnyk, VolodymyrX
<[email protected]<mailto:[email protected]>>; Tahhan,
Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>; GUPTA, ALOK
<[email protected]<mailto:[email protected]>>; SINGARAJU, GOKUL
<[email protected]<mailto:[email protected]>>; '[email protected]'
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
OK, trying to use the node.yaml and collectd virt plugin as noted on the wiki,
still having errors with collectd (collectd fails as the virt plugin is not
found). I think this is probably due to some of the steps in the guide not
being completed. Honestly
http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/feature.userguide.html#virt-plugin<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_feature.userguide.html-23virt-2Dplugin&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=2HmUI80GTO-P-98xNvo_veQ3wEk3m2oCFz4TLHkg7bY&e=>
could be a bit clearer... it says e.g. "Create domain (VM) XML configuration
file. For more information on domain XML format and examples, please see:
https://libvirt.org/formatdomain.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__libvirt.org_formatdomain.html&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=g5YvEljC_5mmiOUqyIrbxpX2-C6nM24Fn2IFR0rfYGY&e=>"
but that's a reference to a long guide to libvirt XML files... something more
specific (e.g. a working example and more clarifications as below) would be
really helpful to people that aren't libvirt SMEs, e.g.
* What's typical or workable for libvirt as used by OpenStack
* What should be the "DOMAIN_NAME" in "virsh start DOMAIN_NAME" (maybe the
name matters?)
* Re "Check list of available Performance monitoring events and their
settings:" what would we be checking for and what settings are important to
check? E.g. which events map to measurements in the VES data model or are
mappable via the node.yaml file by the ves_app (presumably we should not worry
for now about those that can't be mapped).
* Re "Clone and install the collectd virt plugin:" is the cloning and
build/install of the collectd repo at this point redundant with the earlier
build (see
http://docs.opnfv.org/en/latest/submodules/barometer/docs/release/userguide/collectd.ves.userguide.html#install-collectd<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opnfv.org_en_latest_submodules_barometer_docs_release_userguide_collectd.ves.userguide.html-23install-2Dcollectd&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=3QaIPLABF1J5bS7ZdYVUJE8lzTFK6CtloDvgmhuKd84&s=Vcp88USCgBSGIMg5tP3gBe0OehWksYdDNv_E-_3P9G0&e=>
... at that point the goal was to "Build collectd with Kafka support" - but do
we have to build twice to get the virt plugin setup?)
Thanks,
Bryan Sullivan | AT&T
From: Mytnyk, VolodymyrX [mailto:[email protected]]
Sent: Monday, November 20, 2017 9:05 AM
To: SULLIVAN, BRYAN L (BRYAN L)
<[email protected]<mailto:[email protected]>>;
Tahhan, Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>; GUPTA, ALOK
<[email protected]<mailto:[email protected]>>; SINGARAJU, GOKUL
<[email protected]<mailto:[email protected]>>; '[email protected]'
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Hi Bryan,
See my comments inline.
* "How do I (as noted in the patch) get ves_app to generate
messages for the guest, hypervisor, and node separately? Do I need to run 3
instances of the ves_app?"
You need to run only one instance of the ves_app (patched one)
somewhere where ves_app will have an access to the Kafka server (e.g. on the
host).
* "Should I use the yaml files in the repo or the one (merged) that
you provided Friday? I assume I should use the ones in the repo."
You have to use new node.yaml from gerrit review (provided on
Friday,
https://gerrit.opnfv.org/gerrit/#/c/47427/1/3rd_party/collectd-ves-app/ves_app/node.yaml<https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.opnfv.org_gerrit_-23_c_47427_1_3rd-5Fparty_collectd-2Dves-2Dapp_ves-5Fapp_node.yaml&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=tAQbjXoKuI6-8IbLIJ1IFnPHe8rikjRZnuL2wqaDfmY&s=xDaI7ASpSIt1nc0vNxBrztdn8jTSJ4aq3FoVAlfb9-Y&e=>
)
* Also, you have to use collectd configuration files provided on Friday.
* For collectd that is running on each guest machine VM, you have to use
collectd-guest.conf (please make sure IP address of Kafka server is correct)
* For collectd that is running on the host machine, you have to use
collectd-host.conf (please make sure IP address of Kafka server is correct)
* `csv` and `logfile` plugins are enabled in the configuration files and
used for debug purposes.
The usage example of the `node.yaml` configuration:
[cid:[email protected]]
Thanks and Regards,
Volodymyr
From: SULLIVAN, BRYAN L (BRYAN L) [mailto:[email protected]]
Sent: Monday, November 20, 2017 6:33 PM
To: Mytnyk, VolodymyrX
<[email protected]<mailto:[email protected]>>; Tahhan,
Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>; GUPTA, ALOK
<[email protected]<mailto:[email protected]>>; SINGARAJU, GOKUL
<[email protected]<mailto:[email protected]>>; '[email protected]'
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Bringing it back to the list, and cc'ing Alok and Gokul as some of this relates
to the overall VES design.
OK, ready to work on testing your patch. I can locally edit the ves-setup
script to pull the patch when I clone barometer in the VES Agent (ves_app)
container.
Quick questions first though:
* How do I (as noted in the patch) get ves_app to generate messages for the
guest, hypervisor, and node separately? Do I need to run 3 instances of the
ves_app?
* Should I use the yaml files in the repo or the one (merged) that you
provided Friday? I assume I should use the ones in the repo.
Thanks,
Bryan Sullivan | AT&T
From: SULLIVAN, BRYAN L (BRYAN L)
Sent: Monday, November 20, 2017 5:36 AM
To: 'Mytnyk, VolodymyrX'
<[email protected]<mailto:[email protected]>>; Tahhan,
Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
I'm using Barometer master and guest mode based upon the discussions we had
(not the merged yaml). Trying to keep it simple on my end so I can get the
basic single node deployment completed. That's done now as of
https://gerrit.opnfv.org/gerrit/#/c/47475/<https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.opnfv.org_gerrit_-23_c_47475_&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=tAQbjXoKuI6-8IbLIJ1IFnPHe8rikjRZnuL2wqaDfmY&s=aeV-Bg_SbIp5MRiNuJiTPctpxyqK3n9QNAUmcaAIj1I&e=>
so I can start looking at your recent changes. Deploying patches from other
repos gets a little complicated as I clone Barometer in the scripts, and have
to edit the script to include checkout of your patch. So I usually wait until
changes from other repos have merged to test. But I'll see what I can do in the
meantime.
Thanks,
Bryan Sullivan | AT&T
From: Mytnyk, VolodymyrX [mailto:[email protected]]
Sent: Monday, November 20, 2017 5:18 AM
To: SULLIVAN, BRYAN L (BRYAN L)
<[email protected]<mailto:[email protected]>>;
Tahhan, Maryam <[email protected]<mailto:[email protected]>>
Cc: Mcmahon, Tony B
<[email protected]<mailto:[email protected]>>; Power, Damien
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Hi Bryan,
Based on the logs you provided
AssertionError: Wrong SELECT condition, selected 4 metrics
looks like the latest changes of VES app is not used. E.g. the
https://gerrit.opnfv.org/gerrit/#/c/47427/<https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.opnfv.org_gerrit_-23_c_47427_&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=wvfRnqnK1lxmd6iX8-opA5DLkTYOTy2WAtC-81JI7qc&s=lsPEgr9kC5dQSvNnVsMfzKNuVv4E3ZmbaqThVx6NXYc&e=>
patch is not applied.
1) Are you sure that this patch is applied and node.yaml configuration is
used by VES application?
2) Are you using collectd configuration files ( guests/hosts) provided by
me on Friday?
Thanks and Regards,
Volodymyr
From: SULLIVAN, BRYAN L (BRYAN L) [mailto:[email protected]]
Sent: Monday, November 20, 2017 3:09 PM
To: Tahhan, Maryam <[email protected]<mailto:[email protected]>>
Cc: Mytnyk, VolodymyrX
<[email protected]<mailto:[email protected]>>; Mcmahon,
Tony B <[email protected]<mailto:[email protected]>>; Power,
Damien <[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
If there's a better way to report issues, e.g. thru the github mirror, let me
know. I do have a number of things to followup on from the testing.
Thanks,
Bryan Sullivan | AT&T
From: Tahhan, Maryam [mailto:[email protected]]
Sent: Monday, November 20, 2017 5:06 AM
To: SULLIVAN, BRYAN L (BRYAN L)
<[email protected]<mailto:[email protected]>>
Cc: Mytnyk, VolodymyrX
<[email protected]<mailto:[email protected]>>; Mcmahon,
Tony B <[email protected]<mailto:[email protected]>>; Power,
Damien <[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] [barometer] ves_app.py exception when
collecting from multiple collectd clients
Hey Bryan
Thanks for the bug report. I've pinged Volodymyr on this, not sure if he
follows the mailing list.
BR
Maryam
From:
[email protected]<mailto:[email protected]>
[mailto:[email protected]] On Behalf Of SULLIVAN,
BRYAN L (BRYAN L)
Sent: Monday, November 20, 2017 4:39 AM
To: '[email protected]'
<[email protected]<mailto:[email protected]>>
Subject: [opnfv-tech-discuss] [barometer] ves_app.py exception when collecting
from multiple collectd clients
Hi Barometer team,
The exception below came up when the ves_app began getting events from multiple
collectd clients (one on each compute node). This occurs when there is a shared
kafka server and a single ves_app instance which is getting the events from the
multiple collectd clients. The exceptions started happening as soon as I
activated the 2nd collectd client. The exception info "AssertionError: Wrong
SELECT condition, selected 4 metrics" changed from "2 metrics" to "4 metrics"
as I added the 2nd thru 4th collectd client, so I'm pretty sure the issue is
related to there being more than one collectd client.
Let me know if you need more info.
Exception in thread Thread-130:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 1073, in run
self.function(*self.args, **self.kwargs)
File "/opt/ves/barometer/3rd_party/collectd-ves-app/ves_app/normalizer.py",
line 561, in on_timer
self.process_measurements()
File "/opt/ves/barometer/3rd_party/collectd-ves-app/ves_app/normalizer.py",
line 583, in process_measurements
measurements = loader.get_data()
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 33,
in get_data
return self.construct_document(self.get_node())
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 48,
in construct_document
for dummy in generator:
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 398,
in construct_yaml_map
value = self.construct_mapping(node)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208,
in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133,
in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 88,
in construct_object
data = constructor(self, node)
File "/opt/ves/barometer/3rd_party/collectd-ves-app/ves_app/normalizer.py",
line 411, in from_yaml
items.append(loader.construct_mapping(item))
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208,
in construct_mapping
return BaseConstructor.construct_mapping(self, node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133,
in construct_mapping
value = self.construct_object(value_node, deep=deep)
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 88,
in construct_object
data = constructor(self, node)
File "/opt/ves/barometer/3rd_party/collectd-ves-app/ves_app/normalizer.py",
line 350, in from_yaml
'Wrong SELECT condition, selected {} metrics'.format(len(metrics))
AssertionError: Wrong SELECT condition, selected 4 metrics
Thanks,
Bryan Sullivan | AT&T
_______________________________________________
opnfv-tech-discuss mailing list
[email protected]
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
