Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
On Sun, Jan 10 2016, AFEK, Ifat (Ifat) wrote: > I'm a bit confused. This thread started by liusheng asking to remove the > notifications to the bus[1], and I replied saying that we do want to use this > capability in Vitrage. What is the notifier plugin that you suggested? Is it > the same notifier that liusheng referred to, or something else? The thread is about removing oslo.messaging notifications on alarm update (creation, deletion, etc). What you want is to receive an oslo.messaging notification on alarm triggering. This does not exist yet, but should be a aodh-notifier plugin to be written – which should be pretty straightforward. -- Julien Danjou # Free Software hacker # https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
so one point i should mention is that the notification we propose on removing was originally attended for Ceilometer events. much in same way as disabling the aodh-notifier to allow Vitrage to consume alarm triggers, it isn't exactly safe for Vitrage to consume the existing messages as Ceilometer may also consume it. the idea with moving it to a discrete notifier (whether zaqar or plain oslo.messaging) is so the consumer of message is explicit. On 10/01/2016 1:30 AM, AFEK, Ifat (Ifat) wrote: Hi Julien, I'm a bit confused. This thread started by liusheng asking to remove the notifications to the bus[1], and I replied saying that we do want to use this capability in Vitrage. What is the notifier plugin that you suggested? Is it the same notifier that liusheng referred to, or something else? [1] https://review.openstack.org/#/c/246727/ Thanks, Ifat. -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: Friday, January 08, 2016 11:42 AM On Thu, Jan 07 2016, AFEK, Ifat (Ifat) wrote: We have two motivations: one is that we want to get notifications on every change, but we don't want to register our webhook to each and every alarm; and the other is that we already listen to the message bus for other openstack components, so we would like to handle aodh the same way. If we disable aodh-notifier, won't it have other impacts on the system? What if someone else used a webhook, and won't be notified because we disabled the notifier? We could have a notifier plugin that would send a notification using oslo.messaging I guess. That shouldn't be a big deal. -- Julien Danjou ;; Free Software hacker ;; https://julien.danjou.info __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi Ifat, I am OK to abandon the change and keep notifications about alarm changes if you really need. the problem is if we enabled the notifications and if there are many alarms periodically evaluated and updated by aodh-evaluator, there will emit many these notification messages and will be stored as events in ceilometer, that may will disperse the events we really want. how about sending these notifications message on a different topic? that can avoid consuming by ceilometer-notification-agent. thanks Liu sheng At 2016-01-10 14:30:34, "AFEK, Ifat (Ifat)"wrote: >Hi Julien, > >I'm a bit confused. This thread started by liusheng asking to remove the >notifications to the bus[1], and I replied saying that we do want to use this >capability in Vitrage. What is the notifier plugin that you suggested? Is it >the same notifier that liusheng referred to, or something else? > > >[1] https://review.openstack.org/#/c/246727/ > >Thanks, >Ifat. > >> -Original Message- >> From: Julien Danjou [mailto:jul...@danjou.info] >> Sent: Friday, January 08, 2016 11:42 AM >> >> On Thu, Jan 07 2016, AFEK, Ifat (Ifat) wrote: >> >> > We have two motivations: one is that we want to get notifications on >> > every change, but we don't want to register our webhook to each and >> > every alarm; and the other is that we already listen to the message >> > bus for other openstack components, so we would like to handle aodh >> the same way. >> > >> > If we disable aodh-notifier, won't it have other impacts on the >> > system? What if someone else used a webhook, and won't be notified >> > because we disabled the notifier? >> >> We could have a notifier plugin that would send a notification using >> oslo.messaging I guess. That shouldn't be a big deal. >> >> -- >> Julien Danjou >> ;; Free Software hacker >> ;; https://julien.danjou.info > >__ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi Pradeep, > From: Pradeep Kilambi [mailto:pkila...@redhat.com] > Sent: Thursday, January 07, 2016 9:15 PM > > > > alternatively, it was discussed that maybe adding a zaqar notifier > > > would be useful. that way, aodh-notifier would send alarm to zaqar and > > > you could configure it to requeue or maybe send a smtp. > > > > What are the advantages of using Zaqar over using the message bus? Is the > > solution you suggested already supported? > > This work is currently in progress, I'm hoping to get this support into > Mitaka. We want to do the integration with Aodh in the coming weeks, so I guess we won't use Zaqar. We will consider using it in Newton. Thanks, Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi Julien, I'm a bit confused. This thread started by liusheng asking to remove the notifications to the bus[1], and I replied saying that we do want to use this capability in Vitrage. What is the notifier plugin that you suggested? Is it the same notifier that liusheng referred to, or something else? [1] https://review.openstack.org/#/c/246727/ Thanks, Ifat. > -Original Message- > From: Julien Danjou [mailto:jul...@danjou.info] > Sent: Friday, January 08, 2016 11:42 AM > > On Thu, Jan 07 2016, AFEK, Ifat (Ifat) wrote: > > > We have two motivations: one is that we want to get notifications on > > every change, but we don't want to register our webhook to each and > > every alarm; and the other is that we already listen to the message > > bus for other openstack components, so we would like to handle aodh > the same way. > > > > If we disable aodh-notifier, won't it have other impacts on the > > system? What if someone else used a webhook, and won't be notified > > because we disabled the notifier? > > We could have a notifier plugin that would send a notification using > oslo.messaging I guess. That shouldn't be a big deal. > > -- > Julien Danjou > ;; Free Software hacker > ;; https://julien.danjou.info __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
> -Original Message- > From: gord chung [mailto:g...@live.ca] > Sent: Wednesday, January 06, 2016 3:52 PM > > On 06/01/2016 8:11 AM, AFEK, Ifat (Ifat) wrote: > > Hi, > > > > We would like to be notified once an alarm state is changed, so we > prefer using the message bus. > > As far as I understand, ceilometer and aodh APIs do not provide > immediate notifications. If we use the APIs, we will have to poll the > data periodically and look for changes, and we will get them in delay. > By listening to the message bus we can get the notifications > immediately, like we do with other openstack components. > > > > Is there an advantage in using the API instead? > > And what do you mean by "aggregation on events data"? what kind of > aggregations can we do? > > > > > is the idea that you don't want to use a webhook to be notified? when > an alarm is computed by aodh-evaluator, it sends this alarm (via > message > queue) to aodh-notifier which in turns does the webhook. if you just > disable aodh-notifier, the alarm won't be consumed and you can listen > to it yourself. We have two motivations: one is that we want to get notifications on every change, but we don't want to register our webhook to each and every alarm; and the other is that we already listen to the message bus for other openstack components, so we would like to handle aodh the same way. If we disable aodh-notifier, won't it have other impacts on the system? What if someone else used a webhook, and won't be notified because we disabled the notifier? > alternatively, it was discussed that maybe adding a zaqar notifier > would be useful. that way, aodh-notifier would send alarm to zaqar and > you could configure it to requeue or maybe send a smtp. What are the advantages of using Zaqar over using the message bus? Is the solution you suggested already supported? Thanks, Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi, We would like to be notified once an alarm state is changed, so we prefer using the message bus. As far as I understand, ceilometer and aodh APIs do not provide immediate notifications. If we use the APIs, we will have to poll the data periodically and look for changes, and we will get them in delay. By listening to the message bus we can get the notifications immediately, like we do with other openstack components. Is there an advantage in using the API instead? And what do you mean by "aggregation on events data"? what kind of aggregations can we do? Thanks, Ifat. > -Original Message- > From: liusheng [mailto:liusheng1...@126.com] > Sent: Wednesday, January 06, 2016 10:10 AM > > Hi Ifat, > > what way do you want to use the alarm change notifications? using > events from Ceilometer API or directly collecting the notifications > from message bus? > if we configure Ceilometer to listen the notifications from aodh, the > notifications about alarm changes will be converted to events and > stored in Ceilometer storage. if you want to use the events about alarm > changes, the same info can be queried by "alarm-history" API, may be > you want to do aggregation on events data ? > > Thanks > Liu sheng > > 在 2016/1/5 21:37, AFEK, Ifat (Ifat) 写道: > >> -Original Message- > >> From: gord chung [mailto:g...@live.ca] > >> Sent: Friday, December 11, 2015 10:17 PM > >> > >> the original reason this was implemented i believe was to track > alarm > >> state changes as an event in ceilometer events. i still see this as > a > >> valid use case but it does duplicate some of the functionality of > >> alarm history. > >> > >> i think the main items are to not flood the message bus with > messages > >> that no one is listening to. but if there is a use case for it, i'm > >> happy to see it remain (and improved). > >> > >> you can check the patch that Liusheng referenced, but the basic > >> concept is as mentioned: send message when an alarm state is > changed. > >> > > Hi Gord, > > > > We had some further discussions about this issue in Vitrage team, and > we decided we do want to use Aodh notifications about alarm state > changes. > > > > One reason is that we want to be notified about every alarm, yet we > don't want to register our web-hook on each alarm definition > separately. The other reason is that we already listen to message bus > notifications of other openstack components (like nova), and we would > like to handle aodh notifications the same way we handle all other > openstack notifications. > > > > We will be happy if this code remains. > > > > Thanks, > > Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
On 06/01/2016 8:11 AM, AFEK, Ifat (Ifat) wrote: Hi, We would like to be notified once an alarm state is changed, so we prefer using the message bus. As far as I understand, ceilometer and aodh APIs do not provide immediate notifications. If we use the APIs, we will have to poll the data periodically and look for changes, and we will get them in delay. By listening to the message bus we can get the notifications immediately, like we do with other openstack components. Is there an advantage in using the API instead? And what do you mean by "aggregation on events data"? what kind of aggregations can we do? Thanks, Ifat. is the idea that you don't want to use a webhook to be notified? when an alarm is computed by aodh-evaluator, it sends this alarm (via message queue) to aodh-notifier which in turns does the webhook. if you just disable aodh-notifier, the alarm won't be consumed and you can listen to it yourself. alternatively, it was discussed that maybe adding a zaqar notifier would be useful. that way, aodh-notifier would send alarm to zaqar and you could configure it to requeue or maybe send a smtp. -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi Ifat, what way do you want to use the alarm change notifications? using events from Ceilometer API or directly collecting the notifications from message bus? if we configure Ceilometer to listen the notifications from aodh, the notifications about alarm changes will be converted to events and stored in Ceilometer storage. if you want to use the events about alarm changes, the same info can be queried by "alarm-history" API, may be you want to do aggregation on events data ? Thanks Liu sheng 在 2016/1/5 21:37, AFEK, Ifat (Ifat) 写道: -Original Message- From: gord chung [mailto:g...@live.ca] Sent: Friday, December 11, 2015 10:17 PM the original reason this was implemented i believe was to track alarm state changes as an event in ceilometer events. i still see this as a valid use case but it does duplicate some of the functionality of alarm history. i think the main items are to not flood the message bus with messages that no one is listening to. but if there is a use case for it, i'm happy to see it remain (and improved). you can check the patch that Liusheng referenced, but the basic concept is as mentioned: send message when an alarm state is changed. Hi Gord, We had some further discussions about this issue in Vitrage team, and we decided we do want to use Aodh notifications about alarm state changes. One reason is that we want to be notified about every alarm, yet we don't want to register our web-hook on each alarm definition separately. The other reason is that we already listen to message bus notifications of other openstack components (like nova), and we would like to handle aodh notifications the same way we handle all other openstack notifications. We will be happy if this code remains. Thanks, Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi Ifat, I meant new API to get statistcs on events data, like "statistics" API on samples data, that has been discussed but not implemented yet. because I thought you'd like to use events data about alarm changes. if you want to directly use the notification messages just because you want to be real-time notified, how about gordon's suggestion ? Thanks Liu sheng 在 2016/1/6 21:11, AFEK, Ifat (Ifat) 写道: Hi, We would like to be notified once an alarm state is changed, so we prefer using the message bus. As far as I understand, ceilometer and aodh APIs do not provide immediate notifications. If we use the APIs, we will have to poll the data periodically and look for changes, and we will get them in delay. By listening to the message bus we can get the notifications immediately, like we do with other openstack components. Is there an advantage in using the API instead? And what do you mean by "aggregation on events data"? what kind of aggregations can we do? Thanks, Ifat. -Original Message- From: liusheng [mailto:liusheng1...@126.com] Sent: Wednesday, January 06, 2016 10:10 AM Hi Ifat, what way do you want to use the alarm change notifications? using events from Ceilometer API or directly collecting the notifications from message bus? if we configure Ceilometer to listen the notifications from aodh, the notifications about alarm changes will be converted to events and stored in Ceilometer storage. if you want to use the events about alarm changes, the same info can be queried by "alarm-history" API, may be you want to do aggregation on events data ? Thanks Liu sheng 在 2016/1/5 21:37, AFEK, Ifat (Ifat) 写道: -Original Message- From: gord chung [mailto:g...@live.ca] Sent: Friday, December 11, 2015 10:17 PM the original reason this was implemented i believe was to track alarm state changes as an event in ceilometer events. i still see this as a valid use case but it does duplicate some of the functionality of alarm history. i think the main items are to not flood the message bus with messages that no one is listening to. but if there is a use case for it, i'm happy to see it remain (and improved). you can check the patch that Liusheng referenced, but the basic concept is as mentioned: send message when an alarm state is changed. Hi Gord, We had some further discussions about this issue in Vitrage team, and we decided we do want to use Aodh notifications about alarm state changes. One reason is that we want to be notified about every alarm, yet we don't want to register our web-hook on each alarm definition separately. The other reason is that we already listen to message bus notifications of other openstack components (like nova), and we would like to handle aodh notifications the same way we handle all other openstack notifications. We will be happy if this code remains. Thanks, Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
> -Original Message- > From: gord chung [mailto:g...@live.ca] > Sent: Friday, December 11, 2015 10:17 PM > > the original reason this was implemented i believe was to track alarm > state changes as an event in ceilometer events. i still see this as a > valid use case but it does duplicate some of the functionality of alarm > history. > > i think the main items are to not flood the message bus with messages > that no one is listening to. but if there is a use case for it, i'm > happy to see it remain (and improved). > > you can check the patch that Liusheng referenced, but the basic concept > is as mentioned: send message when an alarm state is changed. > Hi Gord, We had some further discussions about this issue in Vitrage team, and we decided we do want to use Aodh notifications about alarm state changes. One reason is that we want to be notified about every alarm, yet we don't want to register our web-hook on each alarm definition separately. The other reason is that we already listen to message bus notifications of other openstack components (like nova), and we would like to handle aodh notifications the same way we handle all other openstack notifications. We will be happy if this code remains. Thanks, Ifat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi AFEK, Sorry I was busy in other things and didn't pay much attention to Vitrage project(But I will do :) ), the notification message I metioned doesn't mean the notification based on "alarm_actions". currently, when a alarm's state changed, it will trigger the alarm actions specified in alarm definition, and also send a notification message about the alarm change to notification.info (default config value) meanwhile. thanks Liu sheng At 2015-12-03 00:31:05, "AFEK, Ifat (Ifat)"wrote: >Hi, > >In Vitrage[3] project, we would like to be notified on every alarm that is >triggered, and respond immediately (e.g. by generating RCA insights, or by >triggering new alarms on other resources). We are now in the process of >designing our integration with AODH. > >If I understood you correctly, you want to remove the notifications to the >bus, but keep the alarm_actions in the alarm definition? >I'd be happy to get some more details about the difference between these two >approaches, and why do you think the notifications should be removed. > >[3] https://wiki.openstack.org/wiki/Vitrage > >Thanks, >Ifat. > > >> >> From: liusheng [mailto:liusheng1...@126.com] >> Sent: Tuesday, December 01, 2015 4:32 AM >> To: openstack-dev@lists.openstack.org >> Subject: [openstack-dev] [telemetry][aodh] The purpose of notification about >> alarm updating >> >> Hi folks, >> >> Currently, a notification message will be emitted when updating an alarm >> (state transition, attribute updating, creation), this > functionality was >> added by change[1], but the change didn't describe any purpose. So I wonder >> whether there is any usage of this >> type of notification, we can get the whole details about alarm change by >> alarm-history API. the notification is implicitly >> ignored by default, because the "notification_driver" config option won't be >> configured by default. if we enable this option in >> aodh.conf and enable the "store_events" in ceilometer.conf, this type of >> notifications will be stored as events. so maybe some >> users want to aggregate this with events ? what's your opinion ? >> >> I have made a change try to deprecate this notification, see [2]. >> >> [1] https://review.openstack.org/#/c/48949/ >> [2] https://review.openstack.org/#/c/246727/ >> >> BR >> Liu sheng > >__ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
sorry, my email was blocking the list for a bit so i missed this. the original reason this was implemented i believe was to track alarm state changes as an event in ceilometer events. i still see this as a valid use case but it does duplicate some of the functionality of alarm history. i think the main items are to not flood the message bus with messages that no one is listening to. but if there is a use case for it, i'm happy to see it remain (and improved). you can check the patch that Liusheng referenced, but the basic concept is as mentioned: send message when an alarm state is changed. cheers, -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][aodh][vitrage] The purpose of notification about alarm updating
Hi, In Vitrage[3] project, we would like to be notified on every alarm that is triggered, and respond immediately (e.g. by generating RCA insights, or by triggering new alarms on other resources). We are now in the process of designing our integration with AODH. If I understood you correctly, you want to remove the notifications to the bus, but keep the alarm_actions in the alarm definition? I'd be happy to get some more details about the difference between these two approaches, and why do you think the notifications should be removed. [3] https://wiki.openstack.org/wiki/Vitrage Thanks, Ifat. > > From: liusheng [mailto:liusheng1...@126.com] > Sent: Tuesday, December 01, 2015 4:32 AM > To: openstack-dev@lists.openstack.org > Subject: [openstack-dev] [telemetry][aodh] The purpose of notification about > alarm updating > > Hi folks, > > Currently, a notification message will be emitted when updating an alarm > (state transition, attribute updating, creation), this > functionality was > added by change[1], but the change didn't describe any purpose. So I wonder > whether there is any usage of this > type of notification, we can get the whole details about alarm change by > alarm-history API. the notification is implicitly > ignored by default, because the "notification_driver" config option won't be > configured by default. if we enable this option in > aodh.conf and enable the "store_events" in ceilometer.conf, this type of > notifications will be stored as events. so maybe some > users want to aggregate this with events ? what's your opinion ? > > I have made a change try to deprecate this notification, see [2]. > > [1] https://review.openstack.org/#/c/48949/ > [2] https://review.openstack.org/#/c/246727/ > > BR > Liu sheng __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev