Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-10 Thread Monsyne Dragon
From: Angus Salkeld asalk...@mirantis.commailto:asalk...@mirantis.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Thursday, April 9, 2015 6:43 PM To: OpenStack Development Mailing List

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-09 Thread Flavio Percoco
On 08/04/15 20:23 -0700, Joshua Harlow wrote: Hope this helps: 'Let's do away with' == 'remove it/no longer use it' vs 'let's use RPC-over-AMQP' which means continue using it/use it more. a-ha... That explains it. My, obviously, non-native english translation failed to parse that. Thanks for

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-09 Thread Min Pae
I would agree that the reason monasca had to roll their own notification system is likely because one wasn't available, but whether that should be a separate (stand alone separate project) vs integrated service (part of something like monasca) is, to some extent, debatable. No argument on the

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-09 Thread Zane Bitter
On 06/04/15 22:55, Angus Salkeld wrote: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that is interacting with the client). What do I mean by user messages, and how do they differ from our

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-09 Thread Sandy Walsh
From: Angus Salkeld asalk...@mirantis.com Sent: Wednesday, April 8, 2015 8:24 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [all] how to send messages (and events) to our users I also want to point out that what I'd actually rather see is that

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Clint Byrum
Excerpts from Angus Salkeld's message of 2015-04-06 19:55:37 -0700: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that is interacting with the client). What do I mean by user messages, and

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Duncan Thomas
From a security point, it certainly scares the hell out of me On 7 April 2015 at 08:45, Chris Friesen chris.frie...@windriver.com wrote: On 04/06/2015 10:08 PM, Angus Salkeld wrote: On Tue, Apr 7, 2015 at 1:53 PM, Chris Friesen chris.frie...@windriver.com mailto:chris.frie...@windriver.com

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Chris Dent
On Wed, 8 Apr 2015, Sandy Walsh wrote: Yes. It would be so good to pull apart the state-machine that is Nova and just emit completed actions via notifications. Then, have something like TaskFlow externalize the orchestration. Do away with RPC-over-AMQP. YES! I've got notes going back to my

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
Yeah, I don't think anyone would give access to the production rabbitmq directly. We use Yagi [1] to pipe it to AtomHopper [2] for downstream consumption/sanitizing. -S [1] https://github.com/rackerlabs/yagi [2] http://atomhopper.org/ From: Duncan

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
From: Ryan Brown rybr...@redhat.com Sent: Wednesday, April 8, 2015 9:42 AM The trend in the monitoring space seems to be: 1. Alarms are issued from Metrics as Events. (events can issue alarms too, but conventional alarming is metric based) 2. Multiple events are analyzed to produce

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Ryan Brown
On 04/07/2015 02:34 PM, Sandy Walsh wrote: Tooling in general seems to be moving towards richer event data as well. The logging tools (Loggly/Logstash/PaperTrail/zillions of others) are intended to take your unstructured logs and turn them into events, so why not have Heat output structured

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread gordon chung
Yeah, I'd really like to have a schema for Heat events so we can have a single event stream and repackage events for different consumption goals (metrics, notifications, programmatic interaction, etc). Keystone and parts of Ceilometer use the CADF schema to build notification messages[1].

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Flavio Percoco
On 07/04/15 12:55 +1000, Angus Salkeld wrote: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that is interacting with the client). What do I mean by user messages, and how do they differ from our

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
From: Clint Byrum cl...@fewbar.com Sent: Wednesday, April 8, 2015 1:15 PM There's this: https://wiki.openstack.org/wiki/Cue Hmm, that looks interesting. Will read. I also want to point out that what I'd actually rather see is that all of the services

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Flavio Percoco
On 08/04/15 16:38 +, Sandy Walsh wrote: From: Clint Byrum cl...@fewbar.com Sent: Wednesday, April 8, 2015 1:15 PM There's this: https://wiki.openstack.org/wiki/Cue Hmm, that looks interesting. Will read. I also want to point out that what I'd

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Duncan Thomas
On 8 April 2015 at 20:34, Chris Dent chd...@redhat.com wrote: On Wed, 8 Apr 2015, Sandy Walsh wrote: Yes. It would be so good to pull apart the state-machine that is Nova and just emit completed actions via notifications. Then, have something like TaskFlow externalize the orchestration. Do

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Flavio Percoco
On 08/04/15 15:35 -0700, Min Pae wrote: Uh sorry to nitpick, I think he said “let’s do away with” not “let’s use” RPC-over-AMQP How is that different? I honestly don't see the difference but I'm surre I'm missing something in my translation. On Wed, Apr 8, 2015 at 10:56 AM, Flavio Percoco

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Joshua Harlow
Sandy Walsh wrote: From: Clint Byrumcl...@fewbar.com Sent: Wednesday, April 8, 2015 1:15 PM There's this: https://wiki.openstack.org/wiki/Cue Hmm, that looks interesting. Will read. I also want to point out that what I'd actually rather see is that

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Joshua Harlow
Hope this helps: 'Let's do away with' == 'remove it/no longer use it' vs 'let's use RPC-over-AMQP' which means continue using it/use it more. Flavio Percoco wrote: On 08/04/15 15:35 -0700, Min Pae wrote: Uh sorry to nitpick, I think he said “let’s do away with” not “let’s use” RPC-over-AMQP

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Min Pae
Uh sorry to nitpick, I think he said “let’s do away with” not “let’s use” RPC-over-AMQP On Wed, Apr 8, 2015 at 10:56 AM, Flavio Percoco fla...@redhat.com wrote: On 08/04/15 16:38 +, Sandy Walsh wrote: From: Clint Byrum cl...@fewbar.com Sent:

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Vipul Sabhaya
On Wed, Apr 8, 2015 at 4:45 PM, Min Pae sputni...@gmail.com wrote: an under-the-clould service ? - That is not what I am after here. I think the thread went off on a tangent and this point got lost. A user facing notification system absolutely should be a web centric protocol, as I

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Halterman, Jonathan
The ability to send general purpose notifications is clearly a cross-cutting concern. The absence of an AWS SNS like service in OpenStack is the reason that services like Monasca had to roll their own notifications. This has been a gaping hole in the OpenStack portfolio for a while, and I I think

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Min Pae
an under-the-clould service ? - That is not what I am after here. I think the thread went off on a tangent and this point got lost. A user facing notification system absolutely should be a web centric protocol, as I imagine one of the big consumers of such a system will be monitoring

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Angus Salkeld
On Thu, Apr 9, 2015 at 2:15 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Angus Salkeld's message of 2015-04-06 19:55:37 -0700: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Min Pae
+1 for using taskflow to implement workflows… workflow reliability is a non-trivial problem that is best solved in one place imho I have doubts as to whether AMQP is the right protocol for notifications. Web service interfaces, thus far, being the “standard” interface for interacting with

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Sandy Walsh
Notifications were added for this very purpose. At Rax, we have a downstream consumer (yagi) that routes notifications to an appropriate ATOM/pubsub feed (based on tenant-id). Notifications are only as heavy as you want to make them. The only required fields are event_type and timestamp. ?

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread gordon chung
- Specific user oriented log messages (distinct from our normal operator logs) - Deprecation messages (if they are using old resource properties/template features) - Progress and resource state changes (an application doesn't want to poll an api for a state change) - Automated actions

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Ryan Brown
On 04/07/2015 10:10 AM, gordon chung wrote: - Specific user oriented log messages (distinct from our normal operator logs) - Deprecation messages (if they are using old resource properties/template features) - Progress and resource state changes (an application doesn't want to poll an api

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread gordon chung
This could be extended to richer JSON events that include the stack, resources affected in the update, stats like num-deleted-resources or num-replaced-resources, autoscaling actions, and info about stack errors. Is there a way for users as-is to view those raw notifications, not just the

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Sandy Walsh
Tooling in general seems to be moving towards richer event data as well. The logging tools (Loggly/Logstash/PaperTrail/zillions of others) are intended to take your unstructured logs and turn them into events, so why not have Heat output structured events that we can present to the user with

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Jay Pipes
On 04/07/2015 11:34 AM, Sandy Walsh wrote: log({'message':The %(foo)s did %(thing)s, 'foo':'x', 'thing':'action'}) I pity the foo that did that thing. -jay __ OpenStack Development Mailing List (not for usage questions)

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-06 Thread Angus Salkeld
On Tue, Apr 7, 2015 at 1:53 PM, Chris Friesen chris.frie...@windriver.com wrote: On 04/06/2015 08:55 PM, Angus Salkeld wrote: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that is

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-06 Thread Chris Friesen
On 04/06/2015 08:55 PM, Angus Salkeld wrote: Hi all For quite some time we (Heat team) have wanted to be able to send messages to our users (by user I do not mean the Operator, but the User that is interacting with the client). What do I mean by user messages, and how do they differ from our

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-06 Thread Chris Friesen
On 04/06/2015 10:08 PM, Angus Salkeld wrote: On Tue, Apr 7, 2015 at 1:53 PM, Chris Friesen chris.frie...@windriver.com mailto:chris.frie...@windriver.com wrote: On 04/06/2015 08:55 PM, Angus Salkeld wrote: Hi all For quite some time we (Heat team) have wanted to be able to