[openstack-dev] [cue] PTL candidacy

[Min Pae]

I am submitting myself as a candidate as Cue PTL for the Newton cycle.

I have been involved in the Cue project since its inception and would like
to help continue its development to make it a production ready message
broker provisioning/management service.

Some things I’d like to work on for Newton:

- Improve reliability of the broker cluster creation process
- Improve monitoring of broker instances to detect failures in broker
cluster members
- Introduce healing functions for recovering failed clusters

As a stretch goal I would also like to see a second broker type be
introduced alongside RabbitMQ, as Cue was never intended to be a "RabbitMQ
vending machine”.

Thank you,
- Min Pae
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Oslo][TaskFlow] Proposal for new core reviewer (greg hill)

[Min Pae]

+1 Greg has been actively contributing to taskflow, both code, code review,
and general discussions and helping users.  It would be great to have him
as a core.

On Wed, Nov 11, 2015 at 12:02 PM, Joshua Harlow 
wrote:

> Greetings all stackers,
>
> I propose that we add Greg Hill[1] to the taskflow-core[2] team.
>
> Greg (aka jimbo) has been actively contributing to taskflow for a
> while now, both in helping make taskflow better via code
> contribution(s) and by helping spread more usage/knowledge of taskflow
> at rackspace (since the big-data[3] team uses taskflow internally).
> He has helped provided quality reviews and is doing an awesome job
> with the various taskflow concepts and helping make taskflow the best
> it can be!
>
> Overall I think he would make a great addition to the core review team.
>
> Please respond with +1/-1.
>
> Thanks much!
>
> - Joshua Harlow
>
> [1] https://launchpad.net/~greg-hill
> [2] https://launchpad.net/taskflow
> [3] http://www.rackspace.com/cloud/big-data
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Oslo][Automaton] Proposal for new core reviewer (min pae)

[Min Pae]

Thanks Davanum :)

On Mon, Jun 8, 2015 at 12:00 PM, Davanum Srinivas dava...@gmail.com wrote:

 +1 from me Josh. welcome Min Pae.

 -- dims

 On Mon, Jun 8, 2015 at 2:31 PM, Joshua Harlow harlo...@outlook.com
 wrote:
  Greetings all stackers,
 
  I propose that we add Min Pae (sputnik13) to the automaton-core team.
 
  Min has been actively contributing to taskflow for a while now and as
  automaton (the library came out of taskflow) is a new library it would be
  great to have his participation there as well. He is willing (and able!)
 to
  help with the review load when he can. He has provided quality reviews in
  other projects and would be a welcome addition in helping make automaton
 the
  best library it can be!
 
  Overall I think he would make a great addition to the core review team.
 
  Please respond with +1/-1.
 
  Thanks much!
 
  --
 
  Joshua Harlow
 
  It's openstack, relax... | harlo...@yahoo-inc.com
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 --
 Davanum Srinivas :: https://twitter.com/dims

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] how to send messages (and events) to our users

[Min Pae]

I would agree that the reason monasca had to roll their own notification
system is likely because one wasn't available, but whether that should be a
separate (stand alone separate project) vs integrated service (part of
something like monasca) is, to some extent, debatable.

No argument on the fact that this is a cross-cutting concern, and as
previous posts said it would be great to have a common mechanism for
publishing notifications to users.

Whether the notification system/service is separated or integrated, what
would be the best method to use?  Angus started the thread asking whether
such a service should be something that pushes to other existing endpoints
that the user supplies (syslog), or a publicly available message queue
(zaqar), and there was a suggestion to use something like AMQP as well.

I assert a public/user facing notification system should be web
centric/native for the reason I cited before, one of the consumers of such
a notification system will very likely be web browsers (perhaps even
horizon itself).  If there’s agreement on this point (which I guess there
isn’t, or nobody has really chimed in on it yet), then the next thing would
be to identify the best protocol/transport for communicating the
notifications.  I threw out Atom as a potential method, but by no means am
I advocating Atom, just that it be a web centric protocol.

Also, I’m of the mind that notification/event messaging there should be a
multi-tiered approach to notification/event messaging, where there’s
probably an internal message bus (be it rabbitmq, kafka, activemq, or what
have you) that all services publish to for consumption by other services,
and a consumer of said internal message bus that then publishes the events
publicly to users in a web native protocol.  BTW, I don’t mean open access
public, I mean public facing public.  There should still be access controls
on consuming the public facing notifications.

- Min

On Wed, Apr 8, 2015, 5:46 PM Halterman, Jonathan jonathan.halter...@hp.com
wrote:

 The ability to send general purpose notifications is clearly a
 cross-cutting concern. The absence of an AWS SNS like service in OpenStack
 is the reason that services like Monasca had to roll their own
 notifications. This has been a gaping hole in the OpenStack portfolio for a
 while, and I I think the right way to think of a solution is as a new
 service built around a pub/sub notification API (again, see SNS) as opposed
 to something which merely exposes OpenStack’s internal messaging
 infrastructure in some way (that would be inappropriate).

 Cheers,
 Jonathan

 From: Vipul Sabhaya vip...@gmail.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Wednesday, April 8, 2015 at 5:18 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org

 Subject: Re: [openstack-dev] [all] how to send messages (and events) to
 our users

 On Wed, Apr 8, 2015 at 4:45 PM, Min Pae sputni...@gmail.com wrote:



 an under-the-clould service ? - That is not what I am after here.

 I think the thread went off on a tangent and this point got lost.  A
 user facing notification system absolutely should be a web centric
 protocol, as I imagine one of the big consumers of such a system will be
 monitoring dashboards which is trending more and more toward rich client
 side “Single Page Applications”.  AMQP would not work well in such cases.



 So is the yagi + atom hopper solution something we can point end-users
 to?
 Is it per-tenant etc...


 While I haven’t seen it yet, if that solution provides a means to expose
 the atom events to end users, it seems like a promising start.  The thing
 that’s required, though, is authentication/authorization that’s tied in to
 keystone, so that notification regarding a tenant’s resource is available
 only to that tenant.


 Sandy, do you have a write up somewhere on how to set this up so I can
 experiment a bit?

 Maybe this needs to be a part of Cue?


 Sorry, Cue’s goal is to provision Message Queue/Broker services and
 manage them, just like Trove provisions and manages databases.  Cue would
 be ideally used to stand up and scale the RabbitMQ cluster providing
 messaging for an application backend, but it does not provide messaging
 itself (that would be Zaqar).



 Agree — I don’t think a multi-tenant notification service (which we seem
 to be after here) is the goal of Cue.

 That said, Monasca https://wiki.openstack.org/wiki/Monasca seems have
 implemented the collection, aggregation, and notification of these events.
 What may be missing is in Monasca is a mechanism for the tenant to consume
 these events via something other than AMQP.



 - Min

 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http

Re: [openstack-dev] [all] how to send messages (and events) to our users

[Min Pae]

Uh sorry to nitpick, I think he said “let’s do away with” not “let’s use”
RPC-over-AMQP

On Wed, Apr 8, 2015 at 10:56 AM, Flavio Percoco fla...@redhat.com wrote:

 On 08/04/15 16:38 +, Sandy Walsh wrote:

 

 From: Clint Byrum cl...@fewbar.com
 Sent: Wednesday, April 8, 2015 1:15 PM

 There's this:

 https://wiki.openstack.org/wiki/Cue


 Hmm, that looks interesting. Will read.

  I also want to point out that what I'd actually rather see is that all
 of the services provide functionality like this. Users would be served
 by having an event stream from Nova telling them when their instances
 are active, deleted, stopped, started, error, etc.

 Also, I really liked Sandy's suggestion to use the notifications on the
 backend, and then funnel them into something that the user can consume.
 The project they have, yagi, for putting them into atom feeds is pretty
 interesting. If we could give people a simple API that says subscribe
 to Nova/Cinder/Heat/etc. notifications for instance X, and put them
 in an atom feed, that seems like something that would make sense as
 an under-the-cloud service that would be relatively low cost and would
 ultimately reduce load on API servers.


 THIS!

 Yes. It would be so good to pull apart the state-machine that is Nova and
 just emit completed actions via notifications. Then, have something like
 TaskFlow externalize the orchestration. Do away with RPC-over-AMQP.


 Sorry for being nitpicky but, saying RPC-over-AMQP is way too
 generic. What AMQP version? On top of what technology?

 Considering all the issues OPs have with our current broker story, I
 think considering implementing this on top of pure AMQP (which is how
 that phrase reads) would not be good.

 If you meant RPC-over-messaging then I think you should just keep
 using oslo.nmessaging, which abstracts the problem of picking one
 broker.

 Unfortunately, this means users will need to consume this messages
 from the messaging source using oslo.messaging as well. I say
 unfortunately because I believe the API - or even the protocol - as
 it is exposed through this library - or simply the broker - is not
 something users should deal with. There are services that try to make
 this interaction simpler - yes, Zaqar.

 Flavio



 And, anyone that is interested in the transitions can eavesdrop on the
 notifications.

 In our transition from StackTach.v2 to StackTach.v3 in production we
 simply
 cloned the notification feeds so the two systems can run in parallel*. No
 changes to OpenStack, no disruption of service. Later, we'll just kill off
 the v2 queues.

 -S

 * we did this in Yagi, since olso-messaging doesn't support multiple
 queues
 from one routing key.
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 --
 @flaper87
 Flavio Percoco

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] how to send messages (and events) to our users

[Min Pae]

 an under-the-clould service ? - That is not what I am after here.

 I think the thread went off on a tangent and this point got lost.  A user
facing notification system absolutely should be a web centric protocol, as
I imagine one of the big consumers of such a system will be monitoring
dashboards which is trending more and more toward rich client side “Single
Page Applications”.  AMQP would not work well in such cases.



 So is the yagi + atom hopper solution something we can point end-users to?
 Is it per-tenant etc...


While I haven’t seen it yet, if that solution provides a means to expose
the atom events to end users, it seems like a promising start.  The thing
that’s required, though, is authentication/authorization that’s tied in to
keystone, so that notification regarding a tenant’s resource is available
only to that tenant.


 Sandy, do you have a write up somewhere on how to set this up so I can
 experiment a bit?

 Maybe this needs to be a part of Cue?


Sorry, Cue’s goal is to provision Message Queue/Broker services and manage
them, just like Trove provisions and manages databases.  Cue would be
ideally used to stand up and scale the RabbitMQ cluster providing messaging
for an application backend, but it does not provide messaging itself (that
would be Zaqar).

- Min
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] how to send messages (and events) to our users

[Min Pae]

+1 for using taskflow to implement workflows…  workflow reliability is a
non-trivial problem that is best solved in one place imho

I have doubts as to whether AMQP is the right protocol for notifications.
Web service interfaces, thus far, being the “standard” interface for
interacting with Openstack, it would seem like the best thing to do is to
stay with a web centric protocol like Atom or RSS for delivering
notifications.

Also, as to where the notification should be served out of, whether it’s
provided by the service itself vs a common shared service (like Ceilometer
or Zaqar), I feel like the service APIs are management interfaces and
should not be for event notifications, since the scaling requirements on a
management interface (which might have to support complex workflows) and a
notification interface (which would be simple in logic but very high in
volume) would be different.  Does that mean each service should provide a
separate web stack for serving notifications?  I think there’s a benefit to
being able to go to a single place for notifications, and if I’m an
operator I’d rather have a fewer things to maintain, so a centralized
notification service seems like the better choice.

- Min

On Wed, Apr 8, 2015 at 3:14 PM, Joshua Harlow harlo...@outlook.com wrote:

 Sandy Walsh wrote:

 

 From: Clint Byrumcl...@fewbar.com
 Sent: Wednesday, April 8, 2015 1:15 PM

 There's this:

 https://wiki.openstack.org/wiki/Cue


 Hmm, that looks interesting. Will read.

  I also want to point out that what I'd actually rather see is that all
 of the services provide functionality like this. Users would be served
 by having an event stream from Nova telling them when their instances
 are active, deleted, stopped, started, error, etc.

 Also, I really liked Sandy's suggestion to use the notifications on the
 backend, and then funnel them into something that the user can consume.
 The project they have, yagi, for putting them into atom feeds is pretty
 interesting. If we could give people a simple API that says subscribe
 to Nova/Cinder/Heat/etc. notifications for instance X, and put them
 in an atom feed, that seems like something that would make sense as
 an under-the-cloud service that would be relatively low cost and would
 ultimately reduce load on API servers.


 THIS!

 Yes. It would be so good to pull apart the state-machine that is Nova and
 just emit completed actions via notifications. Then, have something like
 TaskFlow externalize the orchestration. Do away with RPC-over-AMQP.


 Sounds good to me ;)

 http://docs.openstack.org/developer/taskflow/notifications.html were
 designed for this purpose; some of the implementations at:

 http://docs.openstack.org/developer/taskflow/notifications.html#
 implementations

 I know that http://www.rackspace.com/cloud/big-data/ is using one of
 these (among others) and using it to do various tracking of
 workflows/tasks, and such and gathering the information at whatever level
 is desired.

 The neat thing is that it allows for post-workflow addition of listeners
 so if at some future point you want to know the timing of your workflow,
 well u can just plug another listener in that gathers this information (for
 example http://docs.openstack.org/developer/taskflow/
 notifications.html#taskflow.listeners.timing.EventTimeListener)...

 -Josh



 And, anyone that is interested in the transitions can eavesdrop on the
 notifications.

 In our transition from StackTach.v2 to StackTach.v3 in production we
 simply
 cloned the notification feeds so the two systems can run in parallel*. No
 changes to OpenStack, no disruption of service. Later, we'll just kill off
 the v2 queues.

 -S

 * we did this in Yagi, since olso-messaging doesn't support multiple
 queues
 from one routing key.
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Oslo][TaskFlow] Proposal for new core reviewer (min pae)

[Min Pae]

I’ll take that as a prod to step up on my review contributions, and
definitely will be doing so ;-)

Thank you for the vote of confidence, I’m glad to be helping with what I
think will be an essential part of the underlying software infrastructure
for Openstack services.

On Mon, Mar 23, 2015 at 1:58 PM, Doug Hellmann d...@doughellmann.com
wrote:

 Excerpts from Joshua Harlow's message of 2015-03-23 10:40:09 -0700:
  Greetings all stackers,
 
  I propose that we add Min Pae[1] to the taskflow-core team[2].
 
  Min has been actively contributing to taskflow for a while now, both in
  helping prove taskflow out (by being a user via the project that he is
  using it in @ https://wiki.openstack.org/wiki/Cue) and helping with the
  review load when he can. He has provided quality reviews and is doing an
  awesome job with the various taskflow concepts and helping make taskflow
  the best library it can be!
 
  Overall I think he would make a great addition to the core review team.
 
  Please respond with +1/-1.
 
  Thanks much!
 

 Min Pae's review stats look a little low, but after talking with Josh
 I'm comfortable with the amount of discussion and helping that has
 happened outside of existing reviews.

 The usual process is to give nominations a week to collect comments, so
 let's do that.

 Doug

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev