Hi Greg,
First of all, let’s make sure that the notification is
generated by collectd. To do so, create a simple collectd python plugin to dump
notifications into /tmp/python-notifications.dump' file:
cat > /etc/collectd_notification_dump.py <<EOF
import collectd
def notify(n):
f = open('/tmp/python-notifications.dump', 'a')
f.write('host: {}\n'.format(n.host))
f.write('plugin: {}\n'.format(n.plugin))
f.write('plugin_instance: {}\n'.format(n.plugin_instance))
f.write('type: {}\n'.format(n.type))
f.write('type_instance: {}\n'.format(n.type_instance))
f.write('time: {}\n'.format(n.time))
f.write('severity: {}\n'.format(n.severity))
f.write('message: {}\n'.format(n.message))
f.write('\n')
f.close()
collectd.register_notification(notify)
<<EOF
Add the following configuration to the collectd configuration file:
LoadPlugin python
<Plugin python>
ModulePath "/etc"
LogTraces true
Interactive false
Import "collectd_notification_dump"
</Plugin>
Restart collectd. All collectd notifications will be dump in
/tmp/python-notifications.dump' file. E.g. if the collectd `threshold` plugin
generate the notification, it will appear in the dump file. If not, there may
be a problem with configuring the threshold plugin.
Thanks and Regards,
Volodymyr
From: Waines, Greg [mailto:[email protected]]
Sent: Wednesday, August 30, 2017 10:18 PM
To: OpenStack Development Mailing List (not for usage questions)
<[email protected]>
Cc: Tahhan, Maryam <[email protected]>
Subject: Re: [openstack-dev] [vitrage] Collectd - to - Vitrage setup issues
Hi Ifat,
thanks for the reply ... just got around to trying your suggestions.
This definitely helps ... I no longer get any errors on re-starting collectd or
vitrage-graph.
i.e. it appears to load the collectd and updated vitrage conf files correctly
now.
Now still don’t get any alarms in vitrage.
HOWEVER I suspect it may be my collectd setup now.
( WARNING I am NOT a collectd expert. ;) )
I suspect that the vitrage-collectd plugin only sends collectd NOTIFICATIONS or
THRESHOLD Events to vitrage.
i.e. it likely does NOT send just statistic/status samples to vitrage.
I can see that collectd sampling is happening ... I have logfile and csv and
rrd plugins running and samples are being captured in the specified directories
/ files.
I tried to set threshold for CPU based on an example I had found on web.
See attached collectd.conf file .
BUT really not sure if the threshold configuration in my collectd.conf is
correct or working ... is there a way to confirm this ? ( any collectd
experts out there ? )
OR
Is there an example collectd.conf that has notifications or thresholds
(whatever vitrage needs) setup for something basic like CPU ?
Greg.
From: "Afek, Ifat (Nokia - IL/Kfar Sava)"
<[email protected]<mailto:[email protected]>>
Reply-To:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Monday, August 28, 2017 at 9:42 AM
To:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: [openstack-dev] [vitrage] Collectd - to - Vitrage setup issues
Hi Greg,
I’m less familiar with the collectd configuration and the events that it sends.
Regarding the collectd_conf.yaml, it is definitely missing. You should add a
/etc/vitrage/collectd_conf.yaml file that looks like that:
collectd:
- collectd_host: <collectd resource name>
type: <openstack resource type, e.g. nova.host>
name: <openstack resource name, e.g. host1>
- collectd_host: …
This file maps a Collectd resource to the corresponding resource in OpenStack.
Only resources that are listed in this file will have their alarms imported to
Vitrage.
Next, you should add a reference to this file in /etc/vitrage/vitrage.conf:
[collectd]
config_file = /etc/vitrage/collectd_conf.yaml
Then you should restart vitrage-graph.
Let me know if it helped,
Ifat.
From: "Waines, Greg"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, 23 August 2017 at 21:19
I am trying to get collectd to report some alarms to vitrage in a devstack
setup,
I am using a devstack created on a late version of ocata.
And my devstack with vitrage appears to be working ok otherwise;
e.g. I can create VMs, and raise fake alarms using “vitrage event post
-type=compute.host.down ...” or with “aodh alarm create ...
resource_id=instance-uuid” ... and they get reported fine in vitrage.
UNFORTUNATELY not seeing anything in vitrage from collectd, and
don’t believe I’m seeing anything even from collectd,
for example from the syslog output plugin.
I’ve attached the following files: ( not sure if these get distributed on
mailing list )
· /etc/collectd/collectd.conf <-- do these look ok ?
· /etc/vitrage/vitrage.conf <-- do these look ok ?
· /var/log/syslog ... around the time when I updated
collectd.conf and vitrage.conf and restarted collectd and vitrage-graph
o QUESTIONS
• NOTE THE FOLLOWING ERRORS IN THE SYSLOG FILE ... where do I get the
collectd_conf.yaml file from ? Can’t see it in the devstack files for vitrage.
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.039 25962 ERROR vitrage.utils.file [-] File doesn't exist:
/etc/vitrage/collectd_conf.yaml.
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver [-] failed in init
'NoneType' object has no attribute '__getitem__' : TypeError: 'NoneType' object
has no attribute '__getitem__'
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver Traceback (most
recent call last):
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver File
"/opt/stack/vitrage/vitrage/datasources/collectd/driver.py", line 65, in
_configuration_mapping
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver
collectd_config_elements = collectd_config[COLLECTD_DATASOURCE]
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver TypeError:
'NoneType' object has no attribute '__getitem__'
· Aug 23 14:09:31 localhost vitrage-graph[25962]: 2017-08-23
14:09:31.040 25962 ERROR vitrage.datasources.collectd.driver
•
• IT DOESN”T SEEM LIKE collectd is actually getting any events anyways ...
shouldn’t I see some collectd events being reported in /var/log/syslog from
some of the monitoring plugins that are loaded ?
· gregs-air:collectd-info gregwaines$ fgrep "localhost collectd" syslog
· Aug 23 13:56:07 localhost collectd[23267]: supervised by systemd,
will signal readyness
· Aug 23 13:56:07 localhost collectd[23267]: Initialization complete,
entering read-loop.
· Aug 23 13:56:07 localhost collectd[23267]: rrdtool plugin: Adjusting
"RandomTimeout" to 0.000 seconds.
· Aug 23 14:09:05 localhost collectd[23267]: Exiting normally.
· Aug 23 14:09:05 localhost collectd[23267]: collectd: Stopping 5 read
threads.
· Aug 23 14:09:05 localhost collectd[23267]: rrdtool plugin: Shutting
down the queue thread.
· Aug 23 14:09:05 localhost collectd[23267]: collectd: Stopping 5 write
threads.
· Aug 23 14:09:07 localhost collectd[25824]: supervised by systemd,
will signal readyness
· Aug 23 14:09:07 localhost collectd[25824]: Initialization complete,
entering read-loop.
· Aug 23 14:09:07 localhost collectd[25824]: rrdtool plugin: Adjusting
"RandomTimeout" to 0.000 seconds.
·
· /etc/vitrage/templates/host_down_scenarios.yaml
· /etc/vitrage/templates/host_high_cpu_load_scenarios.yaml
o Am I suppose to have some templates that are specific to the collectd
events/alarms that are being reported to vitrage ?
Any other suggestions on things to look at in order to understand what’s wrong ?
Greg.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
