Planning to use apache Qpid implementation of AMQP for publishing cloudstack events. Amazon like notifications system ala AWS SNS can be added to this eventually.
-abhi >-----Original Message----- >From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] >Sent: Friday, May 18, 2012 10:40 PM >To: George Reese; cloudstack-dev@incubator.apache.org >Cc: Nitin Mehta; Kishan Kavala; Dean >Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack >Questions ) > >While embedding the publisher inside CloudStack could be one way to go, it is >probably more immediately useful if we push events to a popular queueing >system such as AMQP? >Having CS have its own unique way of publishing events does make it more >work for others to utilize these events. >Agree on having a design that allows for other systems of notifications. > >From: George Reese ><george.re...@enstratus.com<mailto:george.re...@enstratus.com>> >Date: Fri, 18 May 2012 05:24:42 -0700 >To: "cloudstack-dev@incubator.apache.org<mailto:cloudstack- >d...@incubator.apache.org>" <cloudstack- >d...@incubator.apache.org<mailto:cloudstack-dev@incubator.apache.org>> >Cc: Nitin Mehta <nitin.me...@citrix.com<mailto:nitin.me...@citrix.com>>, >Kishan Kavala <kishan.kav...@citrix.com<mailto:kishan.kav...@citrix.com>>, >Dean <cl...@tizatron.com<mailto:cl...@tizatron.com>>, Chiradeep Vittal ><chiradeep.vit...@citrix.com<mailto:chiradeep.vit...@citrix.com>> >Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack >Questions ) > >Here's the way event pub sub should work... > >People interested in events will subscribe to them via API and CloudStack will >maintain a record that includes: > >* What type of event(s) the subscriber cares about >* The URI of an endpoint to notify on events >* An optional API key for signing event publications > >If you want it to be really rich, you allow for multiple notifications >endpoints, >including SMS and email. > >When the existing event management system in CloudStack identifies a state >change, it published it to a message queue. > >A new CloudStack component is subscribed to the message queue and pulls >the state change off. It then sends the event to all interested parties. It >does >not keep delivery state. It sends and forgets. The only thing resembling state >is the counting of failures, eventually killing an endpoint that fails to >often. > >-George > >On May 17, 2012, at 10:53 PM, Abhinandan Prateek wrote: > >Cloudstack already have a well-defined set of events, currently being used by >the Usage module. >We can extend this to publish the events to registered listeners/subscribers. > >Nitin, > We should evaluate the google guava package for this purpose. > >-abhi > >-----Original Message----- >From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] >Sent: Saturday, May 12, 2012 3:48 AM >To: cloudstack-dev@incubator.apache.org<mailto:cloudstack- >d...@incubator.apache.org> >Cc: Dean >Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack >Questions ) > >+1. >This could also be used for example to update DNS records on a DNS server >whenever a VM is started / stopped in a network. >A subscriber to the event bus can do this, and potentially things like >updating a >CMDB or an LDAP server. > >The important caveat is that this will be asynchronous to the stuff happening >inside the management server: it has limited utility for doing stuff that is >required to succeed before an API operation can succeed. >For example, if a VM start requires a NAT rule to be programmed on the >firewall before the start can be considered successful. You could write a event >bus subscriber that does this, but the management server would not know if >it was successful or not. > >There's at least a couple of entry points: >1. com.cloud.utils.fsm.StateListener. Classes that implement this listener get >notified on finite state event transitions. Examples include VM start / stop 2. >com.cloud.network.element.NetworkElement. Classes that implement this >get notified of network state transitions. Examples include adding nics and >removing nics to/from a network. > >I think the first question regarding monitoring VMs on the isolated VLAN had a >different origin though. >The intention there was to have a monitoring service (e.g., Nagios) reach into >the VM and monitor stuff like CPU, IO, or even applications. >The issue there was around network reachability. > >-- >Chiradeep > > > >On 5/11/12 2:09 PM, "Adrian Cole" ><adr...@jclouds.org<mailto:adr...@jclouds.org>> wrote: > >+1 > >Makes sense to have pubsub. Inside the java codebase, we could consider a >clean and idiomatic lib like guava which is easy to unit test. > >http://codingjunkie.net/guava-eventbus/ > >Then, expose out-of-JVM hooks for any of the popular services people use. > >-A >On May 11, 2012 1:58 PM, "Dean" ><cl...@tizatron.com<mailto:cl...@tizatron.com>> wrote: > >Cross reference to: > > > >http://mail-archives.apache.org/mod_mbox/incubator-cloudstack- >dev/201204. >mbox/browser > >[ from: Marlon Davids ] >< munch > >2) How do we monitor VM's that are in Cloudstack when they are in an >isolated VLAN does anyone have a clever workaround? >3) Has anyone developed a script for parsing and alerting on warning events in >the management Log yet? > >I would like to propose cloudstack consider a pub/sub model for event >handling to complement API calls like listEvents. > >Polling can be problematic and sensitive to scaling. > >A simple example would be state change on a physical device. The admin >server can simply publish a message on a network socket indicating that the >device has changed it's state. > >If a subscriber was interested in that device, it could make an api call to >the >admin server for state change information for that device only. >The >admin server may choose to validate that physical device against the current >state table in the database. > >The API would reply that this node changed it's state from Operational to >Prep For Maintenance. (or whatever the transition state would be) > >The message exchange could be wrapped around vm states, resource >additions/removals etc. > >Using a library like zeromq, a developer can write any number of consumers >in any language they wanted to subscribe to the Event Bus. > >Comments? > >-- >-Dean > > >-- >George Reese - Chief Technology Officer, enStratus >e: george.re...@enstratus.com<mailto:george.re...@enstratus.com> >Skype: nspollution t: @GeorgeReese p: +1.207.956.0217 >enStratus: Enterprise Cloud Management - @enStratus - >http://www.enstratus.com<http://www.enstratus.com/> >To schedule a meeting with me: http://tungle.me/GeorgeReese
