Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-08 Thread Doug Hellmann
On Tue, Oct 8, 2013 at 9:44 AM, Thomas Maddox
thomas.mad...@rackspace.comwrote:

   On 10/7/13 3:49 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote:




 On Mon, Oct 7, 2013 at 4:23 PM, Thomas Maddox thomas.mad...@rackspace.com
  wrote:

   On 10/7/13 1:55 PM, Doug Hellmann doug.hellm...@dreamhost.com
 wrote:




 On Mon, Oct 7, 2013 at 1:44 PM, Thomas Maddox 
 thomas.mad...@rackspace.com wrote:

  On 10/3/13 4:09 PM, Thomas Maddox thomas.mad...@rackspace.com
 wrote:

 On 10/3/13 8:53 AM, Julien Danjou jul...@danjou.info wrote:
 
 On Thu, Oct 03 2013, Thomas Maddox wrote:
 
  Interesting point, Doug and Julien. I'm thinking out loud, but if we
 wanted
  to use pipeline.yaml, we could have an 'enabled' attribute for each
  pipeline?
 
 That would be an option, for sure. But just removing all of them should
 also work.
 
  I'm curious, does the pipeline dictate whether its resulting
  sample is stored, or if no pipeline is configured, will it just store
 the
  sample according to the plugins in */notifications.py? I will test
 this
 out.
 
 If there's no pipeline, there's no sample, so nothing's stored.
 
  For additional context, the intent of the feature is to allow a
 deployer
  more flexibility. Like, say we wanted to only enable storing
 white-listed
  event traits and using trigger pipelines (to come) for notification
 based
  alerting/monitoring?
 
 This is already supported by the pipeline as you can list the meters
 you
 want or not.
 
 I poked around a bunch today; yep, you're right - we can just drop
 samples
 on the floor by negating all meters in pipeline.yaml. I didn't have much
 luck just removing all pipeline definitions or using a blank one (it
 puked, and anything other than negating all samples felt too hacky to be
 viable with trusted behavior).
 
 I had my semantics and understanding of the workflow from the collector
 to
 the pipeline to the dispatcher all muddled and was set straight today.
 =]
 I will think on this some more.
 
 I was also made aware of some additional Stevedore functionality, like
 NamedExtensionManager, that should allow us to completely enable/disable
 any handlers we don't want to load and the pipelines with just config
 changes, and easily (thanks, Dragon!).
 
 I really appreciate the time you all take to help us less experienced
 developers learn on a daily basis! =]

  I tried two approaches from this:

 1. Using NamedExtensionManager and passing in an empty list of names, I
 get the same RuntimeError[1]
 2. Using EnabledExtensionManager (my preference since the use case for
 disabling is lesser than enabling) and passing in a black list check,
 with
 which I received the same Runtime error when an empty list of extensions
 was the result.

 I was thinking that, with the white-list/black-list capability of [Named,
 Enabled]ExtensionManager, it would behave more like an iterator. If the
 manager didn't load any Extensions, then it would just no op on
 operations
 on said extensions it owns and the application would carry on as always.

 Is this something that we could change in Stevedore? I wanted to get your
 thoughts before opening an issue there, in case this was intended
 behavior
 for some benefit I'm not aware of.


  The exception is intended to prevent the app from failing silently if
 it cannot load any plugins for some reason, but
 stevedore should throw a different exception for the could not load any
 plugins and I was told not to use any plugins and then told to do some
 work cases.


   Thanks, Doug!

  I poked around a bit more. This is being raised in the map function:
 https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L135-L137,
 not at load time. I see a separate try/except block for a failure to load,
 it looks like:
 https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L85-L97.
 Is that what you're referring to?


  The exception is raised when the manager is used, because the manager
 might have been created as a module or application global object in a place
 where the traceback wouldn't have been logged properly.


  I don't understand. Why wouldn't it have been logged properly when it
 fails in the _load_plugins(…) method? Due to implementor's code, Stevedore
 code, or some other reason(s) that I'm missing?


If the manager is instantiated before logging is configured that log call
won't do anything (or might do the wrong thing).

Stevedore is trying to prevent an app from failing to start if some or all
of the extensions don't load, but then complain noisily later when the
extensions are actually *used*. The trade-off for protecting against those
cases is what you're running into -- sometimes it is OK to not load or
invoke any plugins. But stevedore does not know when that is OK, so it is
left up to the caller to either catch the exception and ignore it or
perform some sort of check explicitly before calling the manager.


 In this particular case, though, the thing calling the extension 

Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-07 Thread Thomas Maddox
On 10/3/13 4:09 PM, Thomas Maddox thomas.mad...@rackspace.com wrote:

On 10/3/13 8:53 AM, Julien Danjou jul...@danjou.info wrote:

On Thu, Oct 03 2013, Thomas Maddox wrote:

 Interesting point, Doug and Julien. I'm thinking out loud, but if we
wanted
 to use pipeline.yaml, we could have an 'enabled' attribute for each
 pipeline?

That would be an option, for sure. But just removing all of them should
also work.

 I'm curious, does the pipeline dictate whether its resulting
 sample is stored, or if no pipeline is configured, will it just store
the
 sample according to the plugins in */notifications.py? I will test this
out.

If there's no pipeline, there's no sample, so nothing's stored.

 For additional context, the intent of the feature is to allow a
deployer
 more flexibility. Like, say we wanted to only enable storing
white-listed
 event traits and using trigger pipelines (to come) for notification
based
 alerting/monitoring?

This is already supported by the pipeline as you can list the meters you
want or not.

I poked around a bunch today; yep, you're right - we can just drop samples
on the floor by negating all meters in pipeline.yaml. I didn't have much
luck just removing all pipeline definitions or using a blank one (it
puked, and anything other than negating all samples felt too hacky to be
viable with trusted behavior).

I had my semantics and understanding of the workflow from the collector to
the pipeline to the dispatcher all muddled and was set straight today. =]
I will think on this some more.

I was also made aware of some additional Stevedore functionality, like
NamedExtensionManager, that should allow us to completely enable/disable
any handlers we don't want to load and the pipelines with just config
changes, and easily (thanks, Dragon!).

I really appreciate the time you all take to help us less experienced
developers learn on a daily basis! =]

I tried two approaches from this:

1. Using NamedExtensionManager and passing in an empty list of names, I
get the same RuntimeError[1]
2. Using EnabledExtensionManager (my preference since the use case for
disabling is lesser than enabling) and passing in a black list check, with
which I received the same Runtime error when an empty list of extensions
was the result.

I was thinking that, with the white-list/black-list capability of [Named,
Enabled]ExtensionManager, it would behave more like an iterator. If the
manager didn't load any Extensions, then it would just no op on operations
on said extensions it owns and the application would carry on as always.

Is this something that we could change in Stevedore? I wanted to get your
thoughts before opening an issue there, in case this was intended behavior
for some benefit I'm not aware of.

-Thomas

[1]:'RuntimeError: No ceilometer.collector extensions found'



Cheers!

-Thomas


-- 
Julien Danjou
-- Free Software hacker - independent consultant
-- http://julien.danjou.info




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-07 Thread Doug Hellmann
On Mon, Oct 7, 2013 at 1:44 PM, Thomas Maddox
thomas.mad...@rackspace.comwrote:

 On 10/3/13 4:09 PM, Thomas Maddox thomas.mad...@rackspace.com wrote:

 On 10/3/13 8:53 AM, Julien Danjou jul...@danjou.info wrote:
 
 On Thu, Oct 03 2013, Thomas Maddox wrote:
 
  Interesting point, Doug and Julien. I'm thinking out loud, but if we
 wanted
  to use pipeline.yaml, we could have an 'enabled' attribute for each
  pipeline?
 
 That would be an option, for sure. But just removing all of them should
 also work.
 
  I'm curious, does the pipeline dictate whether its resulting
  sample is stored, or if no pipeline is configured, will it just store
 the
  sample according to the plugins in */notifications.py? I will test this
 out.
 
 If there's no pipeline, there's no sample, so nothing's stored.
 
  For additional context, the intent of the feature is to allow a
 deployer
  more flexibility. Like, say we wanted to only enable storing
 white-listed
  event traits and using trigger pipelines (to come) for notification
 based
  alerting/monitoring?
 
 This is already supported by the pipeline as you can list the meters you
 want or not.
 
 I poked around a bunch today; yep, you're right - we can just drop samples
 on the floor by negating all meters in pipeline.yaml. I didn't have much
 luck just removing all pipeline definitions or using a blank one (it
 puked, and anything other than negating all samples felt too hacky to be
 viable with trusted behavior).
 
 I had my semantics and understanding of the workflow from the collector to
 the pipeline to the dispatcher all muddled and was set straight today. =]
 I will think on this some more.
 
 I was also made aware of some additional Stevedore functionality, like
 NamedExtensionManager, that should allow us to completely enable/disable
 any handlers we don't want to load and the pipelines with just config
 changes, and easily (thanks, Dragon!).
 
 I really appreciate the time you all take to help us less experienced
 developers learn on a daily basis! =]

 I tried two approaches from this:

 1. Using NamedExtensionManager and passing in an empty list of names, I
 get the same RuntimeError[1]
 2. Using EnabledExtensionManager (my preference since the use case for
 disabling is lesser than enabling) and passing in a black list check, with
 which I received the same Runtime error when an empty list of extensions
 was the result.

 I was thinking that, with the white-list/black-list capability of [Named,
 Enabled]ExtensionManager, it would behave more like an iterator. If the
 manager didn't load any Extensions, then it would just no op on operations
 on said extensions it owns and the application would carry on as always.

 Is this something that we could change in Stevedore? I wanted to get your
 thoughts before opening an issue there, in case this was intended behavior
 for some benefit I'm not aware of.


The exception is intended to prevent the app from failing silently if it
cannot load any plugins for some reason, but
stevedore should throw a different exception for the could not load any
plugins and I was told not to use any plugins and then told to do some
work cases.

In this particular case, though, the thing calling the extension manager
knows what the pipeline configuration is, and could just skip the call if
there are no publishers in the pipeline.

Doug




 -Thomas

 [1]:'RuntimeError: No ceilometer.collector extensions found'


 
 Cheers!
 
 -Thomas
 
 
 --
 Julien Danjou
 -- Free Software hacker - independent consultant
 -- http://julien.danjou.info
 
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-07 Thread Thomas Maddox
On 10/7/13 1:55 PM, Doug Hellmann 
doug.hellm...@dreamhost.commailto:doug.hellm...@dreamhost.com wrote:




On Mon, Oct 7, 2013 at 1:44 PM, Thomas Maddox 
thomas.mad...@rackspace.commailto:thomas.mad...@rackspace.com wrote:
On 10/3/13 4:09 PM, Thomas Maddox 
thomas.mad...@rackspace.commailto:thomas.mad...@rackspace.com wrote:

On 10/3/13 8:53 AM, Julien Danjou 
jul...@danjou.infomailto:jul...@danjou.info wrote:

On Thu, Oct 03 2013, Thomas Maddox wrote:

 Interesting point, Doug and Julien. I'm thinking out loud, but if we
wanted
 to use pipeline.yaml, we could have an 'enabled' attribute for each
 pipeline?

That would be an option, for sure. But just removing all of them should
also work.

 I'm curious, does the pipeline dictate whether its resulting
 sample is stored, or if no pipeline is configured, will it just store
the
 sample according to the plugins in */notifications.py? I will test this
out.

If there's no pipeline, there's no sample, so nothing's stored.

 For additional context, the intent of the feature is to allow a
deployer
 more flexibility. Like, say we wanted to only enable storing
white-listed
 event traits and using trigger pipelines (to come) for notification
based
 alerting/monitoring?

This is already supported by the pipeline as you can list the meters you
want or not.

I poked around a bunch today; yep, you're right - we can just drop samples
on the floor by negating all meters in pipeline.yaml. I didn't have much
luck just removing all pipeline definitions or using a blank one (it
puked, and anything other than negating all samples felt too hacky to be
viable with trusted behavior).

I had my semantics and understanding of the workflow from the collector to
the pipeline to the dispatcher all muddled and was set straight today. =]
I will think on this some more.

I was also made aware of some additional Stevedore functionality, like
NamedExtensionManager, that should allow us to completely enable/disable
any handlers we don't want to load and the pipelines with just config
changes, and easily (thanks, Dragon!).

I really appreciate the time you all take to help us less experienced
developers learn on a daily basis! =]

I tried two approaches from this:

1. Using NamedExtensionManager and passing in an empty list of names, I
get the same RuntimeError[1]
2. Using EnabledExtensionManager (my preference since the use case for
disabling is lesser than enabling) and passing in a black list check, with
which I received the same Runtime error when an empty list of extensions
was the result.

I was thinking that, with the white-list/black-list capability of [Named,
Enabled]ExtensionManager, it would behave more like an iterator. If the
manager didn't load any Extensions, then it would just no op on operations
on said extensions it owns and the application would carry on as always.

Is this something that we could change in Stevedore? I wanted to get your
thoughts before opening an issue there, in case this was intended behavior
for some benefit I'm not aware of.

The exception is intended to prevent the app from failing silently if it cannot 
load any plugins for some reason, but
stevedore should throw a different exception for the could not load any 
plugins and I was told not to use any plugins and then told to do some work 
cases.

Thanks, Doug!

I poked around a bit more. This is being raised in the map function: 
https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L135-L137,
 not at load time. I see a separate try/except block for a failure to load, it 
looks like: 
https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L85-L97.
 Is that what you're referring to?


In this particular case, though, the thing calling the extension manager knows 
what the pipeline configuration is, and could just skip the call if there are 
no publishers in the pipeline.

This seems like it'd have the desired end result, but then this logic would 
have to live in two places for the collector service - both at collector 
initialization as well as each time a notification is processed. If Stevedore 
ExtensionManager behaved like an iterator map, where extensions = [] and 
map(func, extensions) just returns None, we would handle the empty case in one 
place (but silently, indeed). Otherwise, We'd have to check for publishers in 
the init function and, since that causes no notification managers to load,  we 
also have to check for the existence of notification managers in the callback 
for each notification. I realize I'm speaking specifically to Ceilometer's 
collector service, so generally speaking, what I'm suggesting is to fall back 
to how Python handles this naturally, since Stevedore is using a map function 
as syntactic sugar for an iterator of available extensions; seems the simplest 
approach to me.

Another suggestion (thanks, Dragon!), is we could just subclass the 
ExtensionManager for a different way of handling the various iterator 
operations?

So, 

Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-07 Thread Doug Hellmann
On Mon, Oct 7, 2013 at 4:23 PM, Thomas Maddox
thomas.mad...@rackspace.comwrote:

   On 10/7/13 1:55 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote:




 On Mon, Oct 7, 2013 at 1:44 PM, Thomas Maddox thomas.mad...@rackspace.com
  wrote:

  On 10/3/13 4:09 PM, Thomas Maddox thomas.mad...@rackspace.com wrote:

 On 10/3/13 8:53 AM, Julien Danjou jul...@danjou.info wrote:
 
 On Thu, Oct 03 2013, Thomas Maddox wrote:
 
  Interesting point, Doug and Julien. I'm thinking out loud, but if we
 wanted
  to use pipeline.yaml, we could have an 'enabled' attribute for each
  pipeline?
 
 That would be an option, for sure. But just removing all of them should
 also work.
 
  I'm curious, does the pipeline dictate whether its resulting
  sample is stored, or if no pipeline is configured, will it just store
 the
  sample according to the plugins in */notifications.py? I will test
 this
 out.
 
 If there's no pipeline, there's no sample, so nothing's stored.
 
  For additional context, the intent of the feature is to allow a
 deployer
  more flexibility. Like, say we wanted to only enable storing
 white-listed
  event traits and using trigger pipelines (to come) for notification
 based
  alerting/monitoring?
 
 This is already supported by the pipeline as you can list the meters you
 want or not.
 
 I poked around a bunch today; yep, you're right - we can just drop
 samples
 on the floor by negating all meters in pipeline.yaml. I didn't have much
 luck just removing all pipeline definitions or using a blank one (it
 puked, and anything other than negating all samples felt too hacky to be
 viable with trusted behavior).
 
 I had my semantics and understanding of the workflow from the collector
 to
 the pipeline to the dispatcher all muddled and was set straight today. =]
 I will think on this some more.
 
 I was also made aware of some additional Stevedore functionality, like
 NamedExtensionManager, that should allow us to completely enable/disable
 any handlers we don't want to load and the pipelines with just config
 changes, and easily (thanks, Dragon!).
 
 I really appreciate the time you all take to help us less experienced
 developers learn on a daily basis! =]

  I tried two approaches from this:

 1. Using NamedExtensionManager and passing in an empty list of names, I
 get the same RuntimeError[1]
 2. Using EnabledExtensionManager (my preference since the use case for
 disabling is lesser than enabling) and passing in a black list check, with
 which I received the same Runtime error when an empty list of extensions
 was the result.

 I was thinking that, with the white-list/black-list capability of [Named,
 Enabled]ExtensionManager, it would behave more like an iterator. If the
 manager didn't load any Extensions, then it would just no op on operations
 on said extensions it owns and the application would carry on as always.

 Is this something that we could change in Stevedore? I wanted to get your
 thoughts before opening an issue there, in case this was intended behavior
 for some benefit I'm not aware of.


  The exception is intended to prevent the app from failing silently if it
 cannot load any plugins for some reason, but
 stevedore should throw a different exception for the could not load any
 plugins and I was told not to use any plugins and then told to do some
 work cases.


  Thanks, Doug!

  I poked around a bit more. This is being raised in the map function:
 https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L135-L137,
 not at load time. I see a separate try/except block for a failure to load,
 it looks like:
 https://github.com/dreamhost/stevedore/blob/master/stevedore/extension.py#L85-L97.
 Is that what you're referring to?


The exception is raised when the manager is used, because the manager might
have been created as a module or application global object in a place where
the traceback wouldn't have been logged properly.




  In this particular case, though, the thing calling the extension manager
 knows what the pipeline configuration is, and could just skip the call if
 there are no publishers in the pipeline.


  This seems like it'd have the desired end result, but then this logic
 would have to live in two places for the collector service - both at
 collector initialization as well as each time a notification is processed.
 If Stevedore ExtensionManager behaved like an iterator map, where
 extensions = [] and map(func, extensions) just returns None, we would
 handle the empty case in one place (but silently, indeed). Otherwise, We'd
 have to check for publishers in the init function and, since that causes no
 notification managers to load,  we also have to check for the existence of
 notification managers in the callback for each notification. I realize I'm
 speaking specifically to Ceilometer's collector service, so generally
 speaking, what I'm suggesting is to fall back to how Python handles this
 naturally, since Stevedore is using a map function as 

Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-03 Thread Julien Danjou
On Wed, Oct 02 2013, Thomas Maddox wrote:

 I'm working to make the sample pipeline optional and I'm stuck at a
 decision point about whether I ought to use a collector config option
 (like 'enable_sample_pipelines'), or let it be driven by setup.cfg (i.e.
 the existence of sample plugin references). My favorite right now is the
 former, but I wanted to entertain the latter and learn in the process.

What about having an empty pipeline.yml?

-- 
Julien Danjou
;; Free Software hacker ; independent consultant
;; http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-03 Thread Doug Hellmann
On Thu, Oct 3, 2013 at 5:54 AM, Julien Danjou jul...@danjou.info wrote:

 On Wed, Oct 02 2013, Thomas Maddox wrote:

  I'm working to make the sample pipeline optional and I'm stuck at a
  decision point about whether I ought to use a collector config option
  (like 'enable_sample_pipelines'), or let it be driven by setup.cfg (i.e.
  the existence of sample plugin references). My favorite right now is the
  former, but I wanted to entertain the latter and learn in the process.

 What about having an empty pipeline.yml?


Modifying the pipeline's configuration file is the right way to go. If an
empty file isn't valid, then a single pipeline that subscribes to no events
may work.

The other proposed solution isn't going to work. setup.cfg is not meant to
be a deployer-facing configuration file. It's a packaging file that tells
the system what files are part of the app or library.

Doug
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-03 Thread Thomas Maddox
On 10/3/13 8:15 AM, Doug Hellmann 
doug.hellm...@dreamhost.commailto:doug.hellm...@dreamhost.com wrote:




On Thu, Oct 3, 2013 at 5:54 AM, Julien Danjou 
jul...@danjou.infomailto:jul...@danjou.info wrote:
On Wed, Oct 02 2013, Thomas Maddox wrote:

 I'm working to make the sample pipeline optional and I'm stuck at a
 decision point about whether I ought to use a collector config option
 (like 'enable_sample_pipelines'), or let it be driven by setup.cfg (i.e.
 the existence of sample plugin references). My favorite right now is the
 former, but I wanted to entertain the latter and learn in the process.

What about having an empty pipeline.yml?

Modifying the pipeline's configuration file is the right way to go. If an empty 
file isn't valid, then a single pipeline that subscribes to no events may work.

Interesting point, Doug and Julien. I'm thinking out loud, but if we wanted to 
use pipeline.yaml, we could have an 'enabled' attribute for each pipeline? I'm 
curious, does the pipeline dictate whether its resulting sample is stored, or 
if no pipeline is configured, will it just store the sample according to the 
plugins in */notifications.py? I will test this out.

For additional context, the intent of the feature is to allow a deployer more 
flexibility. Like, say we wanted to only enable storing white-listed event 
traits and using trigger pipelines (to come) for notification based 
alerting/monitoring?


The other proposed solution isn't going to work. setup.cfg is not meant to be a 
deployer-facing configuration file. It's a packaging file that tells the system 
what files are part of the app or library.

Agreed. Thanks for the explanation! =]


Doug



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-03 Thread Julien Danjou
On Thu, Oct 03 2013, Thomas Maddox wrote:

 Interesting point, Doug and Julien. I'm thinking out loud, but if we wanted
 to use pipeline.yaml, we could have an 'enabled' attribute for each
 pipeline?

That would be an option, for sure. But just removing all of them should
also work.

 I'm curious, does the pipeline dictate whether its resulting
 sample is stored, or if no pipeline is configured, will it just store the
 sample according to the plugins in */notifications.py? I will test this out.

If there's no pipeline, there's no sample, so nothing's stored.

 For additional context, the intent of the feature is to allow a deployer
 more flexibility. Like, say we wanted to only enable storing white-listed
 event traits and using trigger pipelines (to come) for notification based
 alerting/monitoring?

This is already supported by the pipeline as you can list the meters you
want or not.

-- 
Julien Danjou
-- Free Software hacker - independent consultant
-- http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-03 Thread Thomas Maddox
On 10/3/13 8:53 AM, Julien Danjou jul...@danjou.info wrote:

On Thu, Oct 03 2013, Thomas Maddox wrote:

 Interesting point, Doug and Julien. I'm thinking out loud, but if we
wanted
 to use pipeline.yaml, we could have an 'enabled' attribute for each
 pipeline?

That would be an option, for sure. But just removing all of them should
also work.

 I'm curious, does the pipeline dictate whether its resulting
 sample is stored, or if no pipeline is configured, will it just store
the
 sample according to the plugins in */notifications.py? I will test this
out.

If there's no pipeline, there's no sample, so nothing's stored.

 For additional context, the intent of the feature is to allow a deployer
 more flexibility. Like, say we wanted to only enable storing
white-listed
 event traits and using trigger pipelines (to come) for notification
based
 alerting/monitoring?

This is already supported by the pipeline as you can list the meters you
want or not.

I poked around a bunch today; yep, you're right - we can just drop samples
on the floor by negating all meters in pipeline.yaml. I didn't have much
luck just removing all pipeline definitions or using a blank one (it
puked, and anything other than negating all samples felt too hacky to be
viable with trusted behavior).

I had my semantics and understanding of the workflow from the collector to
the pipeline to the dispatcher all muddled and was set straight today. =]
I will think on this some more.

I was also made aware of some additional Stevedore functionality, like
NamedExtensionManager, that should allow us to completely enable/disable
any handlers we don't want to load and the pipelines with just config
changes, and easily (thanks, Dragon!).

I really appreciate the time you all take to help us less experienced
developers learn on a daily basis! =]

Cheers!

-Thomas


-- 
Julien Danjou
-- Free Software hacker - independent consultant
-- http://julien.danjou.info




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Ceilometer] Looking for input on optional sample pipelines branch

2013-10-02 Thread Thomas Maddox
Hey All!

I'm working to make the sample pipeline optional and I'm stuck at a
decision point about whether I ought to use a collector config option
(like 'enable_sample_pipelines'), or let it be driven by setup.cfg (i.e.
the existence of sample plugin references). My favorite right now is the
former, but I wanted to entertain the latter and learn in the process.

In the case of driving it via setup.cfg, I am not sure how Stevedore could
handle a blank namespace. As it is implemented in the CollectorService
now, it's raising RuntimeError: No namespace extensions found. So,
that's my first question: is there a better way to handle that using
existing Stevedore functionality? The second would be, if we drive the
enabled/disabled pipelines by the existence of references in setup.cfg,
how then could an operator disable it without having to reinstall CM? Just
wanted to be sure I covered my options before picking something and having
to undo my branch (again). =]

Here's the CollectorService from my branch:
https://github.com/TMaddox/ceilometer/blob/optional_samples/ceilometer/coll
ector/service.py. The changes are a WIP, though any feedback is definitely
appreciated!

Thanks in advance for the help!

-Thomas


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev