Hi Volodymyr,

Thinking about this again, I understand that Vitrage is not supposed to use the 
Collectd message as part of the alarm unique key. We currently have a bug with 
clearing Collectd alarms, as a result of the Vitrage ID refactoring that 
happened in Pike. Until we fix it, you can you this patch[1] for your demo.

[1] https://review.openstack.org/#/c/484300

Let me know if it helped,
Ifat.


From: "Afek, Ifat (Nokia - IL/Kfar Sava)" <ifat.a...@nokia.com>
Date: Sunday, 16 July 2017 at 12:28
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

According to the vitrage-collector.log, when the alarm is cleared it has a 
different message:

Raise alarm:
{'vitrage_datasource_action': 'update', 'resource_name': u'qvo818dd156-be', 
u'severity': u'WARNING', u'plugin': u'ovs_events', 'vitrage_entity_type': 
'collectd', u'id': u'd211725834f26fa268016d8b23adf7d7', 'vitrage_sample_date': 
'2017-07-14 07:31:21.405670+00:00', u'host': u'silpixa00399503', u'time': 
1500017481.363748, u'collectd_type': u'gauge', u'plugin_instance': 
u'qvo818dd156-be', u'type_instance': u'link_status', 'vitrage_event_type': 
u'collectd.alarm.warning', u'message': u'link state of "qvo818dd156-be" 
interface has been changed to "DOWN"', 'resource_type': u'neutron.port'}

Clear alarm:
{'vitrage_datasource_action': 'update', 'resource_name': u'qvo818dd156-be', 
u'severity': u'OK', u'plugin': u'ovs_events', 'vitrage_entity_type': 
'collectd', u'id': u'd211725834f26fa268016d8b23adf7d7', 'vitrage_sample_date': 
'2017-07-14 07:31:35.851112+00:00', u'host': u'silpixa00399503', u'time': 
1500017495.841522, u'collectd_type': u'gauge', u'plugin_instance': 
u'qvo818dd156-be', u'type_instance': u'link_status', 'vitrage_event_type': 
u'collectd.alarm.ok', u'message': u'link state of "qvo818dd156-be" interface 
has been changed to "UP"', 'resource_type': u'neutron.port'}

The ‘message’ is converted to the name of the alarm, which is considered part 
of its unique key. If the message is changed from “DOWN” to “UP”, we don’t 
recognize that it’s the same alarm.
Any idea how this can be solved? Can you modify the message so it will be the 
same in both cases? Or is there another field that can uniquely identify the 
alarm?

Thanks,
Ifat.


From: "Mytnyk, VolodymyrX" <volodymyrx.myt...@intel.com>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Date: Friday, 14 July 2017 at 10:56
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Ifat,

                Thank you for fixing the issue. The patch works and I’m able to 
map the alarm to port now. Also, as a workaround, I was able to fix/resolve the 
issue by creating the static datasource (attached static_port.yaml) and 
disabling the neutron port datasource in the vitrage.conf.

Another issue that I still observe is the deleting of the alarm from the graph 
when OK collectd notification is sent (port is becomes up). Currently, it is 
not removed from the entity graph. Is it an issue in the Vitrage too? Attaching 
all logs (collected using the fix provided by you).

The 3rd issue is the Vitrage-Mistral integration, but I will describe this as a 
separate mail thread.

Thanks and Regards,
Volodymyr

From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
Sent: Thursday, July 13, 2017 5:47 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Cc: Tahhan, Maryam <maryam.tah...@intel.com>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

I believe that this change[1] will fix your problem.

[1] https://review.openstack.org/#/c/482212/

Best Regards,
Ifat.

From: "Mytnyk, VolodymyrX" 
<volodymyrx.myt...@intel.com<mailto:volodymyrx.myt...@intel.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Tuesday, 11 July 2017 at 12:48
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Ifat,

Thank you for investigating the issue.

The port name is unique on the graph.  The ovs port name in collectd ovs_events 
plugin is identified by the ‘plugin_instance’ notification field.

Thanks and Regards,
Volodymyr

From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
Sent: Tuesday, July 11, 2017 12:00 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: Tahhan, Maryam <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

I’m working on this issue.
One question: is the port name, as defined by ‘plugin_instance’, supposed to be 
unique in the graph? If not, then how do you uniquely identify the port (in 
collectd)?

Thanks,
Ifat.

From: "Mytnyk, VolodymyrX" 
<volodymyrx.myt...@intel.com<mailto:volodymyrx.myt...@intel.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Friday, 7 July 2017 at 13:27
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Ifat,

I’ve tested the template file modified by you with enabled debug for the 
Vitrage graph. See all Vitrage logs in the attachments.

Thank you!

Best Regards,
Volodymyr

From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
Sent: Friday, July 7, 2017 12:42 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: Tahhan, Maryam <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

Can you please enable debug information in vitrage.conf, restart vitrage-graph, 
and send me the vitrage-graph.log file (in the time where the alarm is raised)? 
I’ll try to understand why the alarm is not connected to the port. The 
definitions in collectd_conf.yaml seem correct.

I did find some issues with the template file – in the alarm definition, you 
specified the name of the resource instead of the name/rawtext of the alarm. 
Also, the name of the port was missing in the port definition. See the attached 
template (which I haven’t checked, but I believe should work). In any case, 
this will not fix the problem with the alarm being connected to the resource; 
it is relevant only for the next phase after we fix the first problem.

Best Regards,
Ifat.

From: "Mytnyk, VolodymyrX" 
<volodymyrx.myt...@intel.com<mailto:volodymyrx.myt...@intel.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Friday, 7 July 2017 at 10:35
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Ifat,

                Sorry, I forgot to attach the topology dump. Attaching it now.

Also, I’ve checked the topology, and looks like there is no relationship 
between neutron port and the alarm for some reason.

Thanks and Regards,
Volodymyr

From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
Sent: Friday, July 7, 2017 12:15 AM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: Tahhan, Maryam <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

Seems like the problem is that the alarm does not get connected to the port. In 
your collectd_conf.yaml, you should write:

collectd:
- collectd_host: silpixa00399503/ovs_events/qvo818dd156-be   (collectd resource 
name)
   type: neutron.port
   name: qvo818dd156      (openstack neutron port name)

By doing this, you cause any Collectd alarm that is raised on the Collectd 
source named silpixa00399503/ovs_events/qvo818dd156-be to be connected in 
Vitrage to a resource of type neutron.port with name qvo818dd156.

Try to look in the output of ‘vitrage topology show’ (you did not attach it to 
the mail) and see the exact details of the port.

Let me know if it helped,
Ifat.

From: "Mytnyk, VolodymyrX" 
<volodymyrx.myt...@intel.com<mailto:volodymyrx.myt...@intel.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Thursday, 6 July 2017 at 23:59
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Ifat,

                Thank you for your help. See my response on your questions 
below:


·         You see the neutron port in your entity graph, and it is connected to 
the VM
V:  yes, it’s correct.


·         You see the alarm on the alarms view, and its resources is the 
neutron port
V: I see the alarm, sent by collectd , on the alarm page (or vitrage alarm 
list).  Not sure how to check the alarm resources?


·         You see the alarm in the entity graph, connected to the neutron port
V: The Alarm is show on the entity graph, but it’s NOT connected to the neutron 
port.


·         You are asking why the neutron port in the entity graph is green and 
not red?
V: The neutron port doesn’t change the color at all.

I’ve validated and tried the suggested template but unfortunately it doesn’t 
help. I still see the alarm on the graph but it is not connected to neutron 
port. Also, the neutron port desn’t change the state. Please note, that I’m 
using master branch for the Vitrage in my environment.
Also, attaching the topology dump with raised alarm.

Used configurations :
====================

collectd:
- collectd_host: silpixa00399503/ovs_events/qvo818dd156-be
   type: neutron.port
   name: qvo818dd156                  # What name we should use here in case of 
neutron.port?

metadata:
name: ovs_interface_down
description: ovs interface down
definitions:
entities:
  - entity:
     category: ALARM
     type: collectd
     name: qvo818dd156
     template_id: collectd_alarm
  - entity:
     category: RESOURCE
     type: neutron.port
     template_id: port
relationships:
  - relationship:
     source: collectd_alarm
     relationship_type: on
     target: port
     template_id: collectd_alarm_on_port
scenarios:
- scenario:
    condition: collectd_alarm_on_port
    actions:
     - action:
        action_type: set_state
        action_target:
         target: port
        properties:
         state: ERROR

Thanks and Regards,
Volodymyr


From: Afek, Ifat (Nokia - IL/Kfar Sava) [mailto:ifat.a...@nokia.com]
Sent: Thursday, July 6, 2017 10:01 PM
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: Tahhan, Maryam <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: Re: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi Volodymyr,

Let me see if I understood the problem correctly. Are my assumptions right?

·         You see the neutron port in your entity graph, and it is connected to 
the VM

·         You see the alarm on the alarms view, and its resources is the 
neutron port

·         You see the alarm in the entity graph, connected to the neutron port

·         You are asking why the neutron port in the entity graph is green and 
not red?

The template that you used as an example is not what you are asking for. This 
template states that if there is an alarm on the host (in your case, the alarm 
is on the neutron port), and the host contains an instance -> raise an alarm on 
the instance and set its state to suboptimal. The template that (I think) you 
need is more simple: if there is an alarm on a neutron port, set its state to 
error.

Below is the template that I think you should use. Note that I haven’t tested 
it locally (because I don’t have the same environment as yours), so there could 
be errors. As a first step, make sure you are using the correct collectd alarm 
name (use ‘name’ and not ‘rawtext’ which is Zabbix-specific), and then run 
‘vitrage template validate’ to verify its correctness.


metadata:
 name: ovs_interface_down
 description: ovs interface is down
definitions:
 entities:
  - entity:
     category: ALARM
     type: collectd
     name: <put the collectd alarm name here>
     template_id: collectd_alarm
  - entity:
     category: RESOURCE
     type: neutron.port
     template_id: port
 relationships:
  - relationship:
     source: collectd_alarm
     relationship_type: on
     target: port
     template_id : collectd_alarm_on_port
scenarios:
 - scenario:
    condition: collectd_alarm_on_port
     - action:
        action_type: set_state
        action_target:
         target: port
        properties:
         state: ERROR


Let me know if it helped,
Ifat.

From: "Mytnyk, VolodymyrX" 
<volodymyrx.myt...@intel.com<mailto:volodymyrx.myt...@intel.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Thursday, 6 July 2017 at 17:18
To: 
"openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Cc: "Tahhan, Maryam" <maryam.tah...@intel.com<mailto:maryam.tah...@intel.com>>
Subject: [openstack-dev] [Vitrage] Collectd notification isn't shown on the 
Vitrage Graph

Hi All,

                We are trying to configure the collectd and Vitrage service to 
raise an alarm in the Vitrage and show the alarm on the graph if a VM interface 
goes down. The collectd & collectd_vitrage plugin have been configured to send 
the notification if the OvS interface connected to VM  goes down (using OvS 
events plugin). In our case, when interface goes down, the collectd event is 
sent to Vitrage and we’re able to see it on the Alarm page. But we would like 
to show the alarm on the Entity Graph (highlighting the neutron port) but for 
some reason it doesn’t work. The 
https://github.com/openstack/vitrage/blob/master/tools/load_generator/templates/vm_0.yaml
 template has been used as a base for our scenario (changing `zabbix` 
datasource to `collectd` in the file). Our collectd datasource configuration 
(collectd_conf.yaml) looks like this (tried both variants):

collectd:
- collectd_host: silpixa00399503/ovs_events/qvo818dd156-be
   type: neutron.port
   name: qvo818dd156-be

collectd:
- collectd_host: silpixa00399503/ovs_events/qvo818dd156-be
   type: nove.host
   name: silpixa00399503

Our setup configuration looks like this:

|HOST| ---->  |VM| ---> |NEUTRON.PORT|---> |NEUTRON.NETWOR|

Any help on the issue would be much appreciated.

Thanks and Regards,
Volodymyr
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to