Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh
On 08/01/2013 09:19 AM, Julien Danjou wrote: On Thu, Aug 01 2013, Sandy Walsh wrote: Hmm, if notifications are turned on, it should fill up. For billing purposes we don't want to lose events simply because there is no consumer. Operations would alert on it and someone would need to put out

Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh
On 08/01/2013 10:25 AM, Julien Danjou wrote: On Thu, Aug 01 2013, Sandy Walsh wrote: Right, that is a concern. Within RAX we have two downstream services that consume notifications (StackTach and Yagi) and we've configured nova to write to two queues. --notifications_topics can take a list

Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-01 Thread Sandy Walsh
On 08/01/2013 07:22 PM, Doug Hellmann wrote: On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com wrote: Hey y'all, I've had a little thorn in my claw on this topic for a while and thought I'd ask the larger group

Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-02 Thread Sandy Walsh
On 08/02/2013 08:38 AM, Doug Hellmann wrote: On Thu, Aug 1, 2013 at 8:52 PM, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com wrote: On 08/01/2013 07:22 PM, Doug Hellmann wrote: On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh

Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-02 Thread Sandy Walsh
On 08/02/2013 12:27 PM, Eoghan Glynn wrote: On 08/01/2013 07:22 PM, Doug Hellmann wrote: On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com wrote: Hey y'all, I've had a little thorn in my claw on this topic for a while

Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-05 Thread Sandy Walsh
On 08/04/2013 08:24 PM, Angus Salkeld wrote: On 02/08/13 13:26 -0300, Sandy Walsh wrote: On 08/02/2013 12:27 PM, Eoghan Glynn wrote: On 08/01/2013 07:22 PM, Doug Hellmann wrote: On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com

Re: [openstack-dev] [Ceilometer] Ceilometer and nova compute cells

2013-08-05 Thread Sandy Walsh
On 08/05/2013 04:49 AM, Julien Danjou wrote: On Fri, Aug 02 2013, Doug Hellmann wrote: On Fri, Aug 2, 2013 at 7:47 AM, Julien Danjou jul...@danjou.info wrote: That would need the RPC layer to connect to different rabbitmq server. Not sure that's supported yet. We'll have that problem in

Re: [openstack-dev] [Ceilometer] Another introduction

2013-08-06 Thread Sandy Walsh
On 08/06/2013 08:31 AM, Nejc Saje wrote: Hey everyone, I'm a developer at XLAB d.o.o. in Ljubljana. My colleagues are part of the EU research project Contrail, dealing with cloud federation, and I got hired to work on Contrail-Openstack integration. Firstly I'm trying to integrate

Re: [openstack-dev] [Heat] event table is a ticking time bomb

2013-08-09 Thread Sandy Walsh
On 08/08/2013 11:36 PM, Angus Salkeld wrote: On 08/08/13 13:16 -0700, Clint Byrum wrote: Last night while reviewing a feature which would add more events to the event table, it dawned on me that the event table really must be removed. https://bugs.launchpad.net/heat/+bug/1209492

Re: [openstack-dev] [glance] [ceilometer] Periodic Auditing In Glance

2013-08-13 Thread Sandy Walsh
On 08/13/2013 04:35 PM, Andrew Melton wrote: I'm just concerned with the type of notification you'd send. It has to be enough fine grained so we don't lose too much information. It's a tough situation, sending out an image.exists for each image with the same payload as say image.upload

Re: [openstack-dev] [Ceilometer] Concerning get_resources/get_meters and the Ceilometer API

2013-08-14 Thread Sandy Walsh
On 08/14/2013 06:17 AM, Julien Danjou wrote: On Tue, Aug 13 2013, Thomas Maddox wrote: Hi Thomas, * Driving get_resources() with the Meter table instead of the Resource table. This is mainly because of the additional filtering available in the Meter table, which allows us to

[openstack-dev] [oslo] ack(), reject() and requeue() support in rpc ...

2013-08-14 Thread Sandy Walsh
At Eric's request in https://review.openstack.org/#/c/41979/ I'm bringing this to the ML for feedback. Currently, oslo-common rpc behaviour is to always ack() a message no matter what. For billing purposes we can't afford to drop important notifications (like *.exists). We only want to ack() if

[openstack-dev] [Ceilometer] Pipeline Retry Semantics ...

2013-08-15 Thread Sandy Walsh
Recently I've been focused on ensuring we don't drop notifications in CM. But problems still exist downstream, after we've captured the raw event. From the efforts going on with the Ceilometer sample pipeline, the new dispatcher model and the upcoming trigger pipeline, the discussion around retry

Re: [openstack-dev] [oslo] ack(), reject() and requeue() support in rpc ...

2013-08-15 Thread Sandy Walsh
On 08/15/2013 02:00 PM, Eric Windisch wrote: On Wed, Aug 14, 2013 at 4:08 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: At Eric's request in https://review.openstack.org/#/c/41979/ I'm bringing this to the ML for feedback. Thank you Sandy. Currently, oslo-common rpc behaviour

Re: [openstack-dev] [Ceilometer] Concerning get_resources/get_meters and the Ceilometer API

2013-08-19 Thread Sandy Walsh
On 08/18/2013 04:04 PM, Jay Pipes wrote: On 08/17/2013 03:10 AM, Julien Danjou wrote: On Fri, Aug 16 2013, Jay Pipes wrote: Actually, that's the opposite of what I'm suggesting :) I'm suggesting getting rid of the resource_metadata column in the meter table and using the resource table in

Re: [openstack-dev] [Ceilometer] Concerning get_resources/get_meters and the Ceilometer API

2013-08-19 Thread Sandy Walsh
On 08/19/2013 09:40 AM, Julien Danjou wrote: On Mon, Aug 19 2013, Sandy Walsh wrote: On 08/19/2013 05:08 AM, Julien Danjou wrote: On Sun, Aug 18 2013, Jay Pipes wrote: I'm proposing that in these cases, a *new* resource would be added to the resource table (and its ID inserted in meter

Re: [openstack-dev] [glance] [ceilometer] Periodic Auditing In Glance

2013-08-19 Thread Sandy Walsh
On 08/16/2013 04:58 PM, Doug Hellmann wrote: The notification messages don't translate 1:1 to database records. Even if the notification payload includes multiple resources, we will store those as multiple individual records so we can query against them. So it seems like sending individual

Re: [openstack-dev] [Ceilometer] Concerning get_resources/get_meters and the Ceilometer API

2013-08-20 Thread Sandy Walsh
On 08/20/2013 10:42 AM, Thomas Maddox wrote: On 8/19/13 8:21 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: On 08/18/2013 04:04 PM, Jay Pipes wrote: On 08/17/2013 03:10 AM, Julien Danjou wrote: On Fri, Aug 16 2013, Jay Pipes wrote: Actually, that's the opposite of what I'm suggesting

Re: [openstack-dev] [Ceilometer] Need help with Alembic...

2013-08-26 Thread Sandy Walsh
I'm getting the same problem with a different migration (mine is complaining that a column already exists) http://paste.openstack.org/show/44512/ I've compared it to the other migrations and it seems fine. -S On 08/26/2013 02:34 PM, Jay Pipes wrote: Hey all, I'm trying to figure out what

Re: [openstack-dev] [Nova] Frustrations with review wait times

2013-08-28 Thread Sandy Walsh
On 08/28/2013 10:58 AM, Thierry Carrez wrote: Robert Collins wrote: So I'd like to throw two ideas into the mix. Firstly, consider having a rota - ideally 24x5 but that will need some more geographical coverage I suspect for many projects - of folk who spend a dedicated time period only

Re: [openstack-dev] [Ceilometer] what's in scope of Ceilometer

2013-08-29 Thread Sandy Walsh
On 08/29/2013 05:20 AM, Julien Danjou wrote: On Thu, Aug 29 2013, Gordon Chung wrote: the first question is, Ceilometer currently does metering/alarming/maybe a few other things... will it go beyond that? specifically: capacity planning, optimization, dashboard(i assume this falls under

[openstack-dev] [Ceilometer][IceHouse] Ceilometer + Kibana + ElasticSearch Integration

2013-09-23 Thread Sandy Walsh
Nachi ... love it! It would be very cool to see how this would work with Events, since they have much more metadata associated with them. This is the sort of stuff I think we should be doing a lot more of. There are so many excellent open source monitoring components out there. Better

[openstack-dev] Reporting framework in OpenStack ...

2013-10-07 Thread Sandy Walsh
Hey y'all, I've been looking at the reporting framework being put together in oslo [1] and was wondering about using this for other purposes than guru meditation [2]. In StackTach we generate a number of critical reports around usage, billing, performance and errors. Currently we store these

Re: [openstack-dev] Where should Schema files live?

2014-12-08 Thread Sandy Walsh
From: Sandy Walsh [sandy.wa...@rackspace.com] Monday, December 01, 2014 9:29 AM From: Duncan Thomas [duncan.tho...@gmail.com] Sent: Sunday, November 30, 2014 5:40 AM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] Where should Schema files live? Duncan Thomas On Nov 27, 2014

Re: [openstack-dev] Notification Schemas ...

2015-01-19 Thread Sandy Walsh
Thanks Jay, good feedback. Comments inline ... From: Jay Pipes [jaypi...@gmail.com] Sent: Sunday, January 18, 2015 10:47 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] Notification Schemas ... On 01/18/2015 04:39 PM, Sandy

[openstack-dev] Notification Schemas ...

2015-01-18 Thread Sandy Walsh
Hey y'all Eddie Sheffield has pulled together a strawman set of notification schemas for Nova and Glance. Looks like a great start for further discussion. He's going to add JSON-Schema validation next as a form of unit test. Then I guess we have to start thinking about a library to digest

Re: [openstack-dev] [stacktach] [oslo] stachtach - kombu - pika ??

2015-03-16 Thread Sandy Walsh
From: Sean Dague s...@dague.net Sent: Thursday, March 12, 2015 9:09 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [stacktach] [oslo] stachtach - kombu - pika ?? On 03/10/2015 12:56 PM, Joshua Harlow wrote: I saw the following on

Re: [openstack-dev] [stacktach] [oslo] stachtach - kombu - pika ??

2015-03-10 Thread Sandy Walsh
No, we're adding this to Yagi first and perhaps Notabene later. We don't need rpc support, so it's too big a change for us to take on. From: gordon chung g...@live.ca Sent: Tuesday, March 10, 2015 3:58 PM To: OpenStack Development Mailing List not for usage

Re: [openstack-dev] [stacktach] [oslo] stachtach - kombu - pika ??

2015-03-10 Thread Sandy Walsh
Hey J, Our (old) notification consumer was using carrot, which is dead but worked. Lately though there have been conflicts with carrot and msgpack, so we had to change. Around the same time, we ran into a bug where we were writing to an unnamed exchange (completely valid, but too easy to do

Re: [openstack-dev] [oslo_messaging]Notification listener for nova

2015-03-04 Thread Sandy Walsh
I assume you have notifications enabled in nova.conf? See the Enabling notifications in OpenStack section http://www.stacktach.com/about.html Oh, there are couple more flags that could be useful too: notify_on_state_change=vm_and_task_state notify_on_any_change=True ?

Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Sandy Walsh
Cool, I'm interested in creating some notification drivers outside of olso-messaging as well (for Kafka support and schema-based notifications). Do you have a repo started on this? I'd be keen to have a look. -S From: Boden Russell boden...@gmail.com

[openstack-dev] StackTach.v3 docs ...

2015-02-26 Thread Sandy Walsh
First pass at new docs are available here http://www.stacktach.com/ (API and glossary to follow) Feedback and patches welcome! -Sandy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:

Re: [openstack-dev] [all][tc] SQL Schema Downgrades and Related Issues

2015-01-30 Thread Sandy Walsh
From: Johannes Erdfelt [johan...@erdfelt.com] Sent: Thursday, January 29, 2015 9:18 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [all][tc] SQL Schema Downgrades and Related Issues On Thu, Jan 29, 2015,

[openstack-dev] StackTach.v3 now in production ...

2015-04-22 Thread Sandy Walsh
(sorry for cross-post, but this is appropriate to both audiences) Hey y'all! For those of you that don't know, StackTach is a notification-based debugging, monitoring and usage tool for OpenStack. We're happy to announce that we've recently rolled StackTach.v3 into production at one of the

Re: [openstack-dev] [nova][oslo_messaging]Enabling notifications from nova

2015-04-16 Thread Sandy Walsh
http://www.stacktach.com/about.html#enabling? From: Eduard Matei eduard.ma...@cloudfounders.com Sent: Thursday, April 16, 2015 6:04 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [nova][oslo_messaging]Enabling

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Sandy Walsh
Notifications were added for this very purpose. At Rax, we have a downstream consumer (yagi) that routes notifications to an appropriate ATOM/pubsub feed (based on tenant-id). Notifications are only as heavy as you want to make them. The only required fields are event_type and timestamp. ?

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-07 Thread Sandy Walsh
Tooling in general seems to be moving towards richer event data as well. The logging tools (Loggly/Logstash/PaperTrail/zillions of others) are intended to take your unstructured logs and turn them into events, so why not have Heat output structured events that we can present to the user with

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
Yeah, I don't think anyone would give access to the production rabbitmq directly. We use Yagi [1] to pipe it to AtomHopper [2] for downstream consumption/sanitizing. -S [1] https://github.com/rackerlabs/yagi [2] http://atomhopper.org/ From: Duncan

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
From: Ryan Brown rybr...@redhat.com Sent: Wednesday, April 8, 2015 9:42 AM The trend in the monitoring space seems to be: 1. Alarms are issued from Metrics as Events. (events can issue alarms too, but conventional alarming is metric based) 2. Multiple events are analyzed to produce

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-08 Thread Sandy Walsh
From: Clint Byrum cl...@fewbar.com Sent: Wednesday, April 8, 2015 1:15 PM There's this: https://wiki.openstack.org/wiki/Cue Hmm, that looks interesting. Will read. I also want to point out that what I'd actually rather see is that all of the services

Re: [openstack-dev] [all] how to send messages (and events) to our users

2015-04-09 Thread Sandy Walsh
From: Angus Salkeld asalk...@mirantis.com Sent: Wednesday, April 8, 2015 8:24 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [all] how to send messages (and events) to our users I also want to point out that what I'd actually rather see is that

<    1   2