Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-03-11 Thread Boden Russell
Regarding bug 1426046 (see below) -- Is this just a matter of making the
classes public or are you thinking the driver interface needs more
thought + solidifying before making something extendable?

Perhaps I can donate a cycle to 2 to help get this in.

 On 2/26/15 10:33 AM, Doug Hellmann wrote:
 
 Yes, I don't recommend relying on anything in private modules. It looks
 like even the base class for the notification drivers is private, right
 now. We should probably change that, so I filed
 https://bugs.launchpad.net/oslo.messaging/+bug/1426046.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Doug Hellmann


On Thu, Feb 26, 2015, at 07:24 AM, Boden Russell wrote:
 What's the suggested approach for implementing a custom oslo messaging
 driver given the existing impl [1] is private?
 
 e.g. I want to provide my own notification messaging driver which adds
 functionality atop the existing driver [1]. This can obviously be done
 by extending the oslo.messaging.notify._impl_messaging.MessagingDriver
 and registering the custom impl as a stevedore plugin. e.g.
 
 ---
 from oslo.messaging.notify import _impl_messaging as messaging
 
 
 class MyMessagingDriver(messaging.MessagingDriver):
 # impl below
 ---
 
 
 This works, but given the private nature of oslo's _impl_messaging I'm
 likely not following best practices.
 
 Any feedback is appreciated.

Yes, I don't recommend relying on anything in private modules. It looks
like even the base class for the notification drivers is private, right
now. We should probably change that, so I filed
https://bugs.launchpad.net/oslo.messaging/+bug/1426046.

Maybe if you can give more details about what your driver wants to do, I
can provide better feedback about a short-term approach for you.

Doug
 
 Thanks
 
 
 [1] http://goo.gl/cRGNwJ
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Boden Russell
I don't have a public repo -- have been PoCing using a private gitlab to
date... I figured any interest in the driver impl would come out of this
email discussion.

More than happy to provide my PoC code publicly (after a little
clean-up) if there's an interest.


 On 2/26/15 12:01 PM, Sandy Walsh wrote:
 Cool, I'm interested in creating some notification drivers outside of 
 olso-messaging as well (for Kafka support and schema-based notifications). 
 
 Do you have a repo started on this? I'd be keen to have a look.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Sandy Walsh
Cool, I'm interested in creating some notification drivers outside of 
olso-messaging as well (for Kafka support and schema-based notifications). 

Do you have a repo started on this? I'd be keen to have a look.

-S


From: Boden Russell boden...@gmail.com
Sent: Thursday, February 26, 2015 2:41 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [oslo.messaging] extending notification
MessagingDriver

Thanks for filing the bug report...

My driver implementation effectively allows you to filter on
notification events and multicast matches to a given list of topics.

I've been calling it an messaging multicast notification driver and thus
the plugin stevedore entry point I've called 'messaging-multicast'.

The impl basically wraps the oslo messaging driver and does it's
matching and multicasting after the parent... e.g.

def notify(self, ctxt, msg, priority, retry):
_notify = super(AMQPMulticastDriver, self).notify(
ctxt, msg, priority, retry)

# check filters on msg and priority and notify on configured topics
# for any matches below
... code ...

return _notify


The driver is a drop-in replacement anywhere oslo.messaging is used for
notifications and accepts some conf props to set it up.

example config for usage (glance in this example):

rpc_backend = rabbit
...
notification_driver = messaging-multicast
multicast_events = image.upload,image.delete
multicast_topic_prefix = glance.multicast.
publisher_id = GLANCE:MASTER
image.delete = host1,host2
image.upload = host1,host2


The above will send a copy of image.delete and image.upload events to
the glance.multicast.host1 and glance.multicast.host2 topics. It will
use 'GLANCE:MASTER' as the publisher ID in the multicasted events.

Your feedback is appreciated.


 On Thu, Feb 26, 2015, at 07:24 AM, Boden Russell wrote:
 What's the suggested approach for implementing a custom oslo messaging
 driver given the existing impl [1] is private?
 [1] http://goo.gl/cRGNwJ

 On 2/26/15 10:33 AM, Doug Hellmann wrote:
 Yes, I don't recommend relying on anything in private modules. It looks
 like even the base class for the notification drivers is private, right
 now. We should probably change that, so I filed
 https://bugs.launchpad.net/oslo.messaging/+bug/1426046.

 Maybe if you can give more details about what your driver wants to do, I
 can provide better feedback about a short-term approach for you.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Boden Russell
Thanks for filing the bug report...

My driver implementation effectively allows you to filter on
notification events and multicast matches to a given list of topics.

I've been calling it an messaging multicast notification driver and thus
the plugin stevedore entry point I've called 'messaging-multicast'.

The impl basically wraps the oslo messaging driver and does it's
matching and multicasting after the parent... e.g.

def notify(self, ctxt, msg, priority, retry):
_notify = super(AMQPMulticastDriver, self).notify(
ctxt, msg, priority, retry)

# check filters on msg and priority and notify on configured topics
# for any matches below
... code ...

return _notify


The driver is a drop-in replacement anywhere oslo.messaging is used for
notifications and accepts some conf props to set it up.

example config for usage (glance in this example):

rpc_backend = rabbit
...
notification_driver = messaging-multicast
multicast_events = image.upload,image.delete
multicast_topic_prefix = glance.multicast.
publisher_id = GLANCE:MASTER
image.delete = host1,host2
image.upload = host1,host2


The above will send a copy of image.delete and image.upload events to
the glance.multicast.host1 and glance.multicast.host2 topics. It will
use 'GLANCE:MASTER' as the publisher ID in the multicasted events.

Your feedback is appreciated.


 On Thu, Feb 26, 2015, at 07:24 AM, Boden Russell wrote:
 What's the suggested approach for implementing a custom oslo messaging
 driver given the existing impl [1] is private?
 [1] http://goo.gl/cRGNwJ

 On 2/26/15 10:33 AM, Doug Hellmann wrote:
 Yes, I don't recommend relying on anything in private modules. It looks
 like even the base class for the notification drivers is private, right
 now. We should probably change that, so I filed
 https://bugs.launchpad.net/oslo.messaging/+bug/1426046.
 
 Maybe if you can give more details about what your driver wants to do, I
 can provide better feedback about a short-term approach for you.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] extending notification MessagingDriver

2015-02-26 Thread Doug Hellmann


On Thu, Feb 26, 2015, at 01:41 PM, Boden Russell wrote:
 Thanks for filing the bug report...
 
 My driver implementation effectively allows you to filter on
 notification events and multicast matches to a given list of topics.
 
 I've been calling it an messaging multicast notification driver and thus
 the plugin stevedore entry point I've called 'messaging-multicast'.
 
 The impl basically wraps the oslo messaging driver and does it's
 matching and multicasting after the parent... e.g.
 
 def notify(self, ctxt, msg, priority, retry):
 _notify = super(AMQPMulticastDriver, self).notify(
 ctxt, msg, priority, retry)
 
 # check filters on msg and priority and notify on configured topics
 # for any matches below
 ... code ...
 
 return _notify
 
 
 The driver is a drop-in replacement anywhere oslo.messaging is used for
 notifications and accepts some conf props to set it up.

The notification drivers config option supports multiple values, so you
don't need to subclass in order to make that work. You could implement a
completely separate driver that does different notifications than the
built-in version. Of course, we should probably make that base class
public but you don't have to actually subclass it for your driver to
work AFAICT.

You might also want to look at the existing RoutingDriver. It does not
do quite what you want, but some of the concepts might be useful. It
would be useful to be able to compose the routing notifier with other
notifiers to do the sort of thing you're talking about without having to
build a custom driver.

 
 example config for usage (glance in this example):
 
   rpc_backend = rabbit
 ...
 notification_driver = messaging-multicast
 multicast_events = image.upload,image.delete
 multicast_topic_prefix = glance.multicast.
 publisher_id = GLANCE:MASTER
 image.delete = host1,host2
 image.upload = host1,host2
 
 
 The above will send a copy of image.delete and image.upload events to
 the glance.multicast.host1 and glance.multicast.host2 topics. It will
 use 'GLANCE:MASTER' as the publisher ID in the multicasted events.
 
 Your feedback is appreciated.
 
 
  On Thu, Feb 26, 2015, at 07:24 AM, Boden Russell wrote:
  What's the suggested approach for implementing a custom oslo messaging
  driver given the existing impl [1] is private?
  [1] http://goo.gl/cRGNwJ
 
  On 2/26/15 10:33 AM, Doug Hellmann wrote:
  Yes, I don't recommend relying on anything in private modules. It looks
  like even the base class for the notification drivers is private, right
  now. We should probably change that, so I filed
  https://bugs.launchpad.net/oslo.messaging/+bug/1426046.
  
  Maybe if you can give more details about what your driver wants to do, I
  can provide better feedback about a short-term approach for you.
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev