Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Todd Johnson]

Hi.  A colleague and i have been experimenting and prototyping an 
implementation for listening for ampq notifications and kicking off mistral 
workflows as a mistral service.  This is for a use case that is different than 
discussed here, but seems like it might be useful to this use case as well.  
The service reads a mapping file that lists the notifications to listen for and 
which workflow to run when that notification is received.  Workflows are 
started using the context of the notification or, if no context is present, the 
mistral service context.  Since we use the message’s security context you can 
have project scoped workflows that do different things even though they have 
the same name.  The service also passes the notification event type and message 
payload to the workflow.  This allows the workflow to make decisions on what to 
do based on the message contents and/or the type of event.   We just submitted 
a work in progress patch to mistral:  https://review.openstack.org/#/c/248813/ 
.  Please take a look at provide 
feedback.

thanks,
todd__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Renat Akhmerov]

> On 12 Nov 2015, at 00:11, Clint Byrum  wrote:
> 
> Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:
>> 1. Keystone (or some Rabbit->Zaqar proxy service reading notifications 
>> from Keystone) sends "new federated user" notification out via Zaqar.
>> 2. Mistral picks up the message and checks policy to see what should be 
>> done.
>> 3. Mistral calls either Heat or Keystone to autoprovision user.
>> 
> 
> Zane I like most of what you said here, and agree with nearly all of it.
> I actually started typing a question asking why Zaqar, but I think I
> understand, and you can correct me if I'm wrong.
> 
> There's a notification bus. It is generally accessible to all of the
> things run by the operator if the operator wants it to be. Zaqar is for
> communication toward the user, whether from user hosted apps or operator
> hosted services. The thing we're discussiong seems entirely operator
> hosted, to operator hosted. Which to me, at first, meant we should just
> teach Mistral to listen to Keystone notifications and to run workflows
> using trusts acquired similarly to the way Heat acquires them.

Mistral uses trusts I think the same way that Heat does.

Renat

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Zane Bitter]

On 11/11/15 13:11, Clint Byrum wrote:

Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:

1. Keystone (or some Rabbit->Zaqar proxy service reading notifications
from Keystone) sends "new federated user" notification out via Zaqar.
2. Mistral picks up the message and checks policy to see what should be
done.
3. Mistral calls either Heat or Keystone to autoprovision user.



Zane I like most of what you said here, and agree with nearly all of it.
I actually started typing a question asking why Zaqar, but I think I
understand, and you can correct me if I'm wrong.

There's a notification bus. It is generally accessible to all of the
things run by the operator if the operator wants it to be. Zaqar is for
communication toward the user, whether from user hosted apps or operator
hosted services. The thing we're discussiong seems entirely operator
hosted, to operator hosted. Which to me, at first, meant we should just
teach Mistral to listen to Keystone notifications and to run workflows
using trusts acquired similarly to the way Heat acquires them.


Candidly, I'm using this as an opportunity to push my pet feature ;)

I genuinely think it could go either way in this case. What makes the 
difference for me is that if we teach Mistral to read notifications from 
Rabbit then only operators can use that feature, but if we teach Mistral 
to read messages from a Zaqar queue then users can make use of this 
feature too. We get more bang for our buck.


I think if this feature already existed, it would be an obvious 
candidate for this use case. And since it should exist anyway, we should 
just implement it instead of implementing another thing that isn't 
really useful for any other purposes.


As a (theoretical) user I really, really want this. A couple of example 
use cases:
- Heat sends a message and pauses for a response before updating or 
deleting a server resource. The user can configure Mistral to run a 
workflow to quiesce the server and send the response that allows Heat to 
continue.
- Ceilometer sends a message in the event of an alarm. Instead of piping 
it directly to Heat autoscaling, the user configures Mistral to listen 
on the queue and do signal conditioning on the alarm.


Some of the requirements that fall out of this make Zaqar an obvious choice:
- Needs to be user-facing (Keystone auth)
- We MUST NOT lose the message...
- even if the listener is temporarily down


However, it just ocurred to me that if we teach Mistral to read messages
in a Zaqar queue belonging to a user, then there's no weirdness around
user authentication and admin powers. Messages in a user's queue are
entirely acted on using trusts for that user.


Yes, that is also a very nice property.


That said, I think this is overly abstracted. I'd rather just see
operator hosted services listen to the notification bus and react to the
notifications they care about. You have to teach Mistral about trusts
either way so it can do things as a user, and having the notification
go an extra step:

Keystone->[notifications]->Zaqar->Mistral

vs.

Keystone->[notifications]->Mistral

Doesn't make a ton of sense to me.


I agree to some extent, I think I would prefer:

 Keystone->Zaqar->Mistral

Although, honestly, it's only Keystone's desire not to be tightly 
coupled with this that would prevent them from calling Mistral directly.


However, I think:

  Keystone->[notifications]->Zaqar->Mistral

would make sense if:

- Keystone were ordinarily connected to the notification bus; and
- everyone were doing it (i.e. some service existed to routinely pick 
notifications off the notification bus firehose, filter and sanitise 
them for user consumption and redirect them to Zaqar)


Kevin indicates in another part of the thread that the former may not be 
the case in a multi-region deployment, so that is one point against it.


The latter certainly isn't the case *yet*. But it has been discussed 
quite a bit. (Most recently 
https://etherpad.openstack.org/p/mitaka-horizon-async) A lot of things 
need it (it would be really useful for Heat, for example). I wouldn't 
say it's looking inevitable, but there's a good chance it may happen at 
some point.


cheers,
Zane.


But as Renat mentioned, the part about triggering Mistral workflows from
a message does not yet exist. As Tim pointed out, Congress could be a
solution to that (listening for a message and then starting the Mistral
workflow). That may be OK in the short term, but in the long term I'd
prefer that we implement the triggering thing in Mistral (since there
are *lots* of end-user use cases for this too), and have the workflow
optionally query Congress for the policy rather than having Congress in
the loop.



I agree 100% on the positioning of Congress vs. Mistral here.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Clint Byrum]

Excerpts from Renat Akhmerov's message of 2015-11-12 07:52:42 -0800:
> 
> > On 12 Nov 2015, at 00:11, Clint Byrum  wrote:
> > 
> > Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:
> >> 1. Keystone (or some Rabbit->Zaqar proxy service reading notifications 
> >> from Keystone) sends "new federated user" notification out via Zaqar.
> >> 2. Mistral picks up the message and checks policy to see what should be 
> >> done.
> >> 3. Mistral calls either Heat or Keystone to autoprovision user.
> >> 
> > 
> > Zane I like most of what you said here, and agree with nearly all of it.
> > I actually started typing a question asking why Zaqar, but I think I
> > understand, and you can correct me if I'm wrong.
> > 
> > There's a notification bus. It is generally accessible to all of the
> > things run by the operator if the operator wants it to be. Zaqar is for
> > communication toward the user, whether from user hosted apps or operator
> > hosted services. The thing we're discussiong seems entirely operator
> > hosted, to operator hosted. Which to me, at first, meant we should just
> > teach Mistral to listen to Keystone notifications and to run workflows
> > using trusts acquired similarly to the way Heat acquires them.
> 
> Mistral uses trusts I think the same way that Heat does.
> 

Well thats good news. So, keeping things simple, I think this is a case
of just being able to let admins set up a workflow that gets run based
on notifications, and that would need to include setting the user to
run as from content in the notification.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Fox, Kevin M]

+1 for Zane's suggestion. Getting Mistral+Zaqar working together has many other 
side benefits that are great enough that it will help push both Mistral and 
Zaqar to be more commonly deployed.

It also may be able to solve the single keystone, multiple region case more 
easily

If you have keystone sending only to one zaqar in only one region, you could 
still have Mistral in other regions accessing that zaqar to get the 
notifications, so long as the user had the choice of which region to talk to in 
their workflow. This would be much better then the rabbit solution which might 
require rabbit shovels or other cleverness to try and deal with the fact there 
might be multiple regions.

Thanks,
Kevin


From: Zane Bitter [zbit...@redhat.com]
Sent: Thursday, November 12, 2015 10:22 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, 
per-user projects, and Federation

On 11/11/15 13:11, Clint Byrum wrote:
> Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:
>> 1. Keystone (or some Rabbit->Zaqar proxy service reading notifications
>> from Keystone) sends "new federated user" notification out via Zaqar.
>> 2. Mistral picks up the message and checks policy to see what should be
>> done.
>> 3. Mistral calls either Heat or Keystone to autoprovision user.
>>
>
> Zane I like most of what you said here, and agree with nearly all of it.
> I actually started typing a question asking why Zaqar, but I think I
> understand, and you can correct me if I'm wrong.
>
> There's a notification bus. It is generally accessible to all of the
> things run by the operator if the operator wants it to be. Zaqar is for
> communication toward the user, whether from user hosted apps or operator
> hosted services. The thing we're discussiong seems entirely operator
> hosted, to operator hosted. Which to me, at first, meant we should just
> teach Mistral to listen to Keystone notifications and to run workflows
> using trusts acquired similarly to the way Heat acquires them.

Candidly, I'm using this as an opportunity to push my pet feature ;)

I genuinely think it could go either way in this case. What makes the
difference for me is that if we teach Mistral to read notifications from
Rabbit then only operators can use that feature, but if we teach Mistral
to read messages from a Zaqar queue then users can make use of this
feature too. We get more bang for our buck.

I think if this feature already existed, it would be an obvious
candidate for this use case. And since it should exist anyway, we should
just implement it instead of implementing another thing that isn't
really useful for any other purposes.

As a (theoretical) user I really, really want this. A couple of example
use cases:
- Heat sends a message and pauses for a response before updating or
deleting a server resource. The user can configure Mistral to run a
workflow to quiesce the server and send the response that allows Heat to
continue.
- Ceilometer sends a message in the event of an alarm. Instead of piping
it directly to Heat autoscaling, the user configures Mistral to listen
on the queue and do signal conditioning on the alarm.

Some of the requirements that fall out of this make Zaqar an obvious choice:
- Needs to be user-facing (Keystone auth)
- We MUST NOT lose the message...
- even if the listener is temporarily down

> However, it just ocurred to me that if we teach Mistral to read messages
> in a Zaqar queue belonging to a user, then there's no weirdness around
> user authentication and admin powers. Messages in a user's queue are
> entirely acted on using trusts for that user.

Yes, that is also a very nice property.

> That said, I think this is overly abstracted. I'd rather just see
> operator hosted services listen to the notification bus and react to the
> notifications they care about. You have to teach Mistral about trusts
> either way so it can do things as a user, and having the notification
> go an extra step:
>
> Keystone->[notifications]->Zaqar->Mistral
>
> vs.
>
> Keystone->[notifications]->Mistral
>
> Doesn't make a ton of sense to me.

I agree to some extent, I think I would prefer:

  Keystone->Zaqar->Mistral

Although, honestly, it's only Keystone's desire not to be tightly
coupled with this that would prevent them from calling Mistral directly.

However, I think:

   Keystone->[notifications]->Zaqar->Mistral

would make sense if:

- Keystone were ordinarily connected to the notification bus; and
- everyone were doing it (i.e. some service existed to routinely pick
notifications off the notification bus firehose, filter and sanitise
them for user consumption and redirect them to Zaqar)

Kevin indicates in another part of the thread

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Zane Bitter]

On 10/11/15 11:32, Adam Young wrote:

On 11/10/2015 10:28 AM, Renat Akhmerov wrote:




On 09 Nov 2015, at 20:43, Adam Young  wrote:

On 11/06/2015 06:28 PM, Tim Hinrichs wrote:

Congress allows users to write a policy that executes an action
under certain conditions.

The conditions can be based on any data Congress has access to,
which includes nova servers, neutron networks, cinder storage,
keystone users, etc.  We also have some Ceilometer statistics; I'm
not sure about whether it's easy to get the Keystone notifications
that you're talking about today, but notifications are on our
roadmap.  If the user's login is reflected in the Keystone API, we
may already be getting that event.

The action could in theory be a mistral/heat API or an arbitrary
script.  Right now we're set up to invoke any method on any of the
python-clients we've integrated with.  We've got an integration with
heat but not mistral.  New integrations are typically easy.

Sounds like Mistral and Congress are competing here, then.  Maybe we
should merge those efforts.

I may be wrong on this but the difference is that Mistral provides
workflow. Meaning you can have a graph of tasks related by conditional
logic whereas Congress action is something simple like calling a
function. Correct me if my understanding is wrong. I actually don’t
know at this point whether a workflow is really needed, IMO it does
make sense if we need to create a bunch of heavy resources so it
should be an HA service managing the process of configuring/creating
the new tenant. The power of workflow is in automating long-running
stuff.

But both technologies are missing notifications part now.


This does not need to be super complicated; we need a listener that can
kick off workflows.  If congress is that listener, super.


Listening to what?

If it's the RabbitMQ bus then there has been a lot of talk about having 
some service listen to that and emit notifications to actual end-users. 
Ceilometer has been mentioned in the context of these discussions. I 
would love to see this happen. I know we discussed it in the Horizon 
track at Summit, and Heat also has use cases for this.


At the user level, there are many, many use cases that could benefit 
from being able to trigger a Mistral workflow from a Zaqar message 
(Renat and I have discussed this on a number of occasions), and this is 
possibly another.


The two are not mutually exclusive, either - potentially Ceilometer 
could be listening on the RabbitMQ bus and emitting notifications to 
Zaqar that could be picked up by Mistral. It sounds like that's not 
necessary though; Keystone could probably emit the notification wherever 
it likes.



I would think that it would then be

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be
done.
3. Congress calls Heat to fire off template for autoprovisioning user.


This is the right solution if:
- The user template is somewhat complex
- Congress always needs control over the policy
- All we ever need to do is encapsulated in the Heat template


It could also be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Murano Picks up the message and checks policy to see what should be
done.
3. Murano calls Heat to fire off template for autoprovisioning user.


Assuming s/Murano/Mistral/, this is the right solution if:

- The user template is somewhat complex
- We want it to also work without Congress in the loop

Another possibility:

1. Keystone sends "new federated user" notification out on the message bus.
2. Mistral picks up the message and checks policy to see what should be 
 done.

3. Mistral creates the autoprovisioning user directly.

This is the right solution if:
- Creating the user is no harder than creating a Heat stack
- We want it to also work without Congress in the loop


You are suggesting it should be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be
done.
3. Congress calls Murano to fire off template for autoprovisioning user.


This is the right solution if:

- Creating the user is no harder than creating a Heat stack
- Congress always needs control over the policy
- The action may need to be customised in ways not encapsulated in a 
Heat template



And, the most complex solution:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be
done.
3. Congress calls Murano to fire off template for autoprovisioning user.
4. Murano calls Heat to fire off template for autoprovisioning user.


This is the right solution if:
- The user template is somewhat complex
- Congress always needs control over the policy
- The action may need to be customised in ways not encapsulated in a 
Heat template




Personally, I would prefer:

1. 

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Fox, Kevin M]

I'm in the middle of deploying a keystone cluster to support multiple regions. 
We were told we wouldn't need rabbit for this case.

Just something to keep in mind. Keystone's kind of unique in that it can be 
shared between regions.

Thanks,
Kevin

From: Zane Bitter [zbit...@redhat.com]
Sent: Wednesday, November 11, 2015 9:43 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, 
per-user projects, and Federation

On 10/11/15 11:32, Adam Young wrote:
> On 11/10/2015 10:28 AM, Renat Akhmerov wrote:
>>
>>
>>> On 09 Nov 2015, at 20:43, Adam Young <ayo...@redhat.com> wrote:
>>>
>>> On 11/06/2015 06:28 PM, Tim Hinrichs wrote:
>>>> Congress allows users to write a policy that executes an action
>>>> under certain conditions.
>>>>
>>>> The conditions can be based on any data Congress has access to,
>>>> which includes nova servers, neutron networks, cinder storage,
>>>> keystone users, etc.  We also have some Ceilometer statistics; I'm
>>>> not sure about whether it's easy to get the Keystone notifications
>>>> that you're talking about today, but notifications are on our
>>>> roadmap.  If the user's login is reflected in the Keystone API, we
>>>> may already be getting that event.
>>>>
>>>> The action could in theory be a mistral/heat API or an arbitrary
>>>> script.  Right now we're set up to invoke any method on any of the
>>>> python-clients we've integrated with.  We've got an integration with
>>>> heat but not mistral.  New integrations are typically easy.
>>> Sounds like Mistral and Congress are competing here, then.  Maybe we
>>> should merge those efforts.
>> I may be wrong on this but the difference is that Mistral provides
>> workflow. Meaning you can have a graph of tasks related by conditional
>> logic whereas Congress action is something simple like calling a
>> function. Correct me if my understanding is wrong. I actually don’t
>> know at this point whether a workflow is really needed, IMO it does
>> make sense if we need to create a bunch of heavy resources so it
>> should be an HA service managing the process of configuring/creating
>> the new tenant. The power of workflow is in automating long-running
>> stuff.
>>
>> But both technologies are missing notifications part now.
>
> This does not need to be super complicated; we need a listener that can
> kick off workflows.  If congress is that listener, super.

Listening to what?

If it's the RabbitMQ bus then there has been a lot of talk about having
some service listen to that and emit notifications to actual end-users.
Ceilometer has been mentioned in the context of these discussions. I
would love to see this happen. I know we discussed it in the Horizon
track at Summit, and Heat also has use cases for this.

At the user level, there are many, many use cases that could benefit
from being able to trigger a Mistral workflow from a Zaqar message
(Renat and I have discussed this on a number of occasions), and this is
possibly another.

The two are not mutually exclusive, either - potentially Ceilometer
could be listening on the RabbitMQ bus and emitting notifications to
Zaqar that could be picked up by Mistral. It sounds like that's not
necessary though; Keystone could probably emit the notification wherever
it likes.

> I would think that it would then be
>
> 1. Keystone sends "new federated user" notification out on the message bus.
> 2. Congress Picks up the message and checks policy to see what should be
> done.
> 3. Congress calls Heat to fire off template for autoprovisioning user.

This is the right solution if:
- The user template is somewhat complex
- Congress always needs control over the policy
- All we ever need to do is encapsulated in the Heat template

> It could also be:
>
> 1. Keystone sends "new federated user" notification out on the message bus.
> 2. Murano Picks up the message and checks policy to see what should be
> done.
> 3. Murano calls Heat to fire off template for autoprovisioning user.

Assuming s/Murano/Mistral/, this is the right solution if:

- The user template is somewhat complex
- We want it to also work without Congress in the loop

Another possibility:

1. Keystone sends "new federated user" notification out on the message bus.
2. Mistral picks up the message and checks policy to see what should be
  done.
3. Mistral creates the autoprovisioning user directly.

This is the right solution if:
- Creating the user is no harder than creating a Heat stack
- We want it to also work without Congress in the loop

> You are suggesting it

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Clint Byrum]

Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:
> 1. Keystone (or some Rabbit->Zaqar proxy service reading notifications 
> from Keystone) sends "new federated user" notification out via Zaqar.
> 2. Mistral picks up the message and checks policy to see what should be 
> done.
> 3. Mistral calls either Heat or Keystone to autoprovision user.
> 

Zane I like most of what you said here, and agree with nearly all of it.
I actually started typing a question asking why Zaqar, but I think I
understand, and you can correct me if I'm wrong.

There's a notification bus. It is generally accessible to all of the
things run by the operator if the operator wants it to be. Zaqar is for
communication toward the user, whether from user hosted apps or operator
hosted services. The thing we're discussiong seems entirely operator
hosted, to operator hosted. Which to me, at first, meant we should just
teach Mistral to listen to Keystone notifications and to run workflows
using trusts acquired similarly to the way Heat acquires them.

However, it just ocurred to me that if we teach Mistral to read messages
in a Zaqar queue belonging to a user, then there's no weirdness around
user authentication and admin powers. Messages in a user's queue are
entirely acted on using trusts for that user.

That said, I think this is overly abstracted. I'd rather just see
operator hosted services listen to the notification bus and react to the
notifications they care about. You have to teach Mistral about trusts
either way so it can do things as a user, and having the notification
go an extra step:

Keystone->[notifications]->Zaqar->Mistral

vs.

Keystone->[notifications]->Mistral

Doesn't make a ton of sense to me.

> But as Renat mentioned, the part about triggering Mistral workflows from 
> a message does not yet exist. As Tim pointed out, Congress could be a 
> solution to that (listening for a message and then starting the Mistral 
> workflow). That may be OK in the short term, but in the long term I'd 
> prefer that we implement the triggering thing in Mistral (since there 
> are *lots* of end-user use cases for this too), and have the workflow 
> optionally query Congress for the policy rather than having Congress in 
> the loop.
> 

I agree 100% on the positioning of Congress vs. Mistral here.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Tim Hinrichs]

Excerpts from Wed, Nov 11, 2015 at 10:14 AM Clint Byrum 
wrote:

> > But as Renat mentioned, the part about triggering Mistral workflows from
> > a message does not yet exist. As Tim pointed out, Congress could be a
> > solution to that (listening for a message and then starting the Mistral
> > workflow). That may be OK in the short term, but in the long term I'd
> > prefer that we implement the triggering thing in Mistral (since there
> > are *lots* of end-user use cases for this too), and have the workflow
> > optionally query Congress for the policy rather than having Congress in
> > the loop.
> >
>
> I agree 100% on the positioning of Congress vs. Mistral here.
>
>
One problem that I'd imagine Mistral would want to solve if it's picking up
events off the bus and executing workflows is how the operator configures
the event-to-workflow mapping logic.  In Adam's example, the operator would
want to say that every time the 'new-user-login-event' shows up on the bus
that Mistral should kick off the 'create-quota' workflow and the
'create-role' workflow.  In simple cases, this would just be a dictionary,
but what happens if the operator wants to condition workflow execution on
an AND/OR/NOT expression evaluated over state from different projects (e.g.
run 'create-quota' when a new user logs in and that user doesn't already
have a nova quota).

For the operator, the problem becomes more complicated when multiple
OpenStack projects are listening to the bus and kicking off
workflows/scripts/etc. The operator now has N projects to configure
(possibly in different ways) and needs to feel confident that there's not
some (rare) sequence of events that puts OpenStack as a whole into a bad
state because the events/workflows she configured have opposing goals.

The benefit of Congress is that there's one rich, declarative language that
operators can use to control the event-to-workflow mapping.  The operator
dictates which events/states (drawn from any collection of OpenStack
projects) should cause which workflows/templates/APIs (again from any
OpenStack project) to be executed.  And because the mapping is written
declaratively, it's feasible to do some conflict detection.

I'm not arguing that Mistral can't or shouldn't be adapted as was
suggested.  I'm just articulating what Congress brings to the table.

Tim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

[Adam Young]

On 11/10/2015 10:28 AM, Renat Akhmerov wrote:




On 09 Nov 2015, at 20:43, Adam Young  wrote:

On 11/06/2015 06:28 PM, Tim Hinrichs wrote:

Congress allows users to write a policy that executes an action under certain 
conditions.

The conditions can be based on any data Congress has access to, which includes 
nova servers, neutron networks, cinder storage, keystone users, etc.  We also 
have some Ceilometer statistics; I'm not sure about whether it's easy to get 
the Keystone notifications that you're talking about today, but notifications 
are on our roadmap.  If the user's login is reflected in the Keystone API, we 
may already be getting that event.

The action could in theory be a mistral/heat API or an arbitrary script.  Right 
now we're set up to invoke any method on any of the python-clients we've 
integrated with.  We've got an integration with heat but not mistral.  New 
integrations are typically easy.

Sounds like Mistral and Congress are competing here, then.  Maybe we should 
merge those efforts.

I may be wrong on this but the difference is that Mistral provides workflow. 
Meaning you can have a graph of tasks related by conditional logic whereas 
Congress action is something simple like calling a function. Correct me if my 
understanding is wrong. I actually don’t know at this point whether a workflow 
is really needed, IMO it does make sense if we need to create a bunch of heavy 
resources so it should be an HA service managing the process of 
configuring/creating the new tenant. The power of workflow is in automating 
long-running stuff.

But both technologies are missing notifications part now.


This does not need to be super complicated; we need a listener that can 
kick off workflows.  If congress is that listener, super.



I would think that it would then be

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Heat to fire off template for autoprovisioning user.

It could also be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Murano Picks up the message and checks policy to see what should be done.
3. Murano calls Heat to fire off template for autoprovisioning user.


You are suggesting it should be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Murano to fire off template for autoprovisioning user.

And, the most complex solution:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Murano to fire off template for autoprovisioning user.
4. Murano calls Heat to fire off template for autoprovisioning user.

Personally, I would prefer:

1. Keystone sends "new federated user" notification out on the message bus.
2. Heat Picks up the message and checks policy to see what should be done.
3. Heat fires off template for autoprovisioning user.

Why the last:  Heat already exists and already has a message listener.








Renat Akhmerov
@ Mirantis Inc.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev