Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-17 Thread David Chadwick
Hi Adam

prior art is the publish-subscribe mechanism. I dont know if Keystone
already has this implemented or not, or if Python implementation exists
or not, without doing some research

regards

David


On 16/03/2015 18:08, Sumit Naiksatam wrote:
 On Mon, Mar 16, 2015 at 8:10 AM, Adam Young ayo...@redhat.com wrote:
 Oslo policy has been released as a stand alone library.  This is great, in
 that the rules engine is relatively non-applicaition specific, and I assume
 that all of the policy based project are planning to migrate over to using
 the policy library instead of the incubated version.

 Part of the push toward a more dynamic policy mechanism is figuring out how
 to fetch and cache the policy files from Keystone.  I suspect that the other
 services have the same issue.

 1.  How long should a service hold on to the cached version of the policy
 file?
 2.  How can we avoid the stampeding herd if Keystone pushes out a
 notification change event?
 3.  How do we securely cache the file and still make it possible to debug.

 The PKI tokens have a lot of these same issues, and have a one-off mechanism
 for handling it.  We should probably look in to commonizing this function.

 There general mechanism should be fetch and cache but I think it should
 not be tied to keystone token validation so much as capable of using it if
 necessary.  I'm guessing that access to policy rules are typically
 controlled by auth token validated services.  Is this correct?

 Maybe the right level of abstraction is a callback function for fetching the
 file to be cached, with the default being something that uses
 python-requests, and then an  auth plugin based alternative for those that
 require Keystone tokens.

 Before I go off and write a spec, I'd like to know what the prior art is
 here.  I'd also like to know if there oslo policy library is part of the
 plans for the other groups that are doing policy based work?

 
 Thanks Adam for bringing this up. As regards the group-based-policy
 (GBP) project, we leverage the access control policy just like other
 projects do, so the questions you raise above are definitely relevant
 to GBP. We do not manage the lifecycle of this aspect of policy, so we
 hope to use whatever comes out of this discussion.
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Adam Young

On 03/16/2015 03:24 PM, Doug Hellmann wrote:

Excerpts from Adam Young's message of 2015-03-16 14:17:16 -0400:

On 03/16/2015 01:45 PM, Doug Hellmann wrote:

All of these are reasons we have so far resisted building a service to
deploy updates to oslo.config's input files, and rely on provisioning
tools to update them.

Have we consider using normal provisioning tools for pushing out
changes to policy files, and having the policy library look at the
timestamp of the file(s) to decide if it needs to re-read them
before evaluating a rule? Maybe we wouldn't always scan the file
system, but wait for some sort of signal that the scan needs to be
done.

I like this last idea.  Thew trigger needs to be app specific, I think.

I was thinking of a callback to be triggered by 'kill -HUP $pid'. We can
make a little framework for registering callbacks on signals (if there
isn't something like that already) to allow multiple refresh actions on
the signal.


Doug

I think policy files are not config files.  We've treated them as such
in the past as they are not dynamic, but I don't think I want to *have*
to do this:

1.  Change policy in keystone (somehow)
2.  Tell Puppet that there is a new file
3.  Have puppet pick up the3 new file and sync it to the servers.

Right, I wouldn't do that. I would modify the file in my puppet
repository and then push that out all at once. Keystone would receive
the policy files the same way as the other services.



Although I would say that we should make it easy to support this workflow.

For one thing, it assumes that all of the comsuers are talking to the
same config management system, which is only true for a subset of the
services.

I'm not sure what you mean here. Do you mean that in a given deployment
you would expect some services to be configured by puppet and others to
be configured a different way?


I mean that there could be a puppet server for the core infrastructure, 
and another one (or Ansible or Chef) for Hadoop on top of that.  There 
is no one puppet master that we can assume to be controlling all of the 
servers.  They might be run by different organizations.





I see a case for doing this same kind of management for Many of the
files Keystone produces.  Service catalog is the most obvious candidate.

Yes, that's another good example, although in that case we do already
have an API that lets a cloud consumer access the service catalog data
so it might be viewed as different from the policy rules or oslo.config
files (the latter at least typically have private data we wouldn't want
to share through an API).



We don't have a single, monolithic Service catalog (anymore) and, with 
endpoint filtering, we expect multiple service catalogs to be the norm.


I want to pursue the idea of git style file identification here, (hash 
of the file as identifier) as that works to split the service catalog 
from the token, and still have multiple service catalogs, but ensure 
that they are correctly linked in remote systems.   It doesn't have to 
be hash, but it makes the process much more verifiable.  This is also 
true for policy files;  there can be more than one active at any given 
point in time, fetchable by remote identifier.  Even as we push towards 
common  rules for defininng the RBAC section, we have to be aware that 
different endpoints might need different policy files.






If we could have a workflow for managing : PKI certs, Federatiomn
mappings and  (Group only?) Role Assignments we could decentralize token
validation.

When doing the PKI tokens, we discussed this, and ended up with a t
fetch first policy toward the certs.

Puppet does not know how to get a token, so it can't call the keystone
token-protected APIs to fetch new data.  What forms of authentication do
the config managment systems support?  Is this an argument for tokenless
operations against Keystone?

In my scenario puppet (or chef or whatever) is the source of truth for
the configuration file, not one of our services. So there's no need for
the configuration management tool to talk to any of our services beyond
sending the HUP signal telling us to re-read the file(s).


So services would generate files to be published to Puppet.  As I said, 
that would work for a subset of use cases, and probably makes sense for 
core infrastructure, but was cannot assume all consumers are talking to 
puppet, or even if they are, talking to the same puppet master.





Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Jamie Lennox


- Original Message -
 From: Adam Young ayo...@redhat.com
 To: openstack-dev@lists.openstack.org
 Sent: Tuesday, March 17, 2015 8:59:17 AM
 Subject: Re: [openstack-dev] [keystone][congress][group-policy] Fetching 
 policy from a remote source
 
 On 03/16/2015 03:24 PM, Doug Hellmann wrote:
  Excerpts from Adam Young's message of 2015-03-16 14:17:16 -0400:
  On 03/16/2015 01:45 PM, Doug Hellmann wrote:
  All of these are reasons we have so far resisted building a service to
  deploy updates to oslo.config's input files, and rely on provisioning
  tools to update them.
 
  Have we consider using normal provisioning tools for pushing out
  changes to policy files, and having the policy library look at the
  timestamp of the file(s) to decide if it needs to re-read them
  before evaluating a rule? Maybe we wouldn't always scan the file
  system, but wait for some sort of signal that the scan needs to be
  done.
  I like this last idea.  Thew trigger needs to be app specific, I think.
  I was thinking of a callback to be triggered by 'kill -HUP $pid'. We can
  make a little framework for registering callbacks on signals (if there
  isn't something like that already) to allow multiple refresh actions on
  the signal.
 
  Doug
  I think policy files are not config files.  We've treated them as such
  in the past as they are not dynamic, but I don't think I want to *have*
  to do this:

(sorry about the bad threading, zimbra...) 

To pick up on that policy files are not config files, my last impression of 
this from Paris was that keystone was working towards having more fine grained 
roles, eventually pushing these roles to essentially being capabilities. If we 
get to this point - aren't the policy files just config files? I like the 
automatic update approach but at what frequency do we anticipate them being 
updated, and what is a reasonable amount of time for the change to propogate? 



 
  1.  Change policy in keystone (somehow)
  2.  Tell Puppet that there is a new file
  3.  Have puppet pick up the3 new file and sync it to the servers.
  Right, I wouldn't do that. I would modify the file in my puppet
  repository and then push that out all at once. Keystone would receive
  the policy files the same way as the other services.
 
 
  Although I would say that we should make it easy to support this workflow.
 
  For one thing, it assumes that all of the comsuers are talking to the
  same config management system, which is only true for a subset of the
  services.
  I'm not sure what you mean here. Do you mean that in a given deployment
  you would expect some services to be configured by puppet and others to
  be configured a different way?
 
 I mean that there could be a puppet server for the core infrastructure,
 and another one (or Ansible or Chef) for Hadoop on top of that.  There
 is no one puppet master that we can assume to be controlling all of the
 servers.  They might be run by different organizations.
 
 
  I see a case for doing this same kind of management for Many of the
  files Keystone produces.  Service catalog is the most obvious candidate.
  Yes, that's another good example, although in that case we do already
  have an API that lets a cloud consumer access the service catalog data
  so it might be viewed as different from the policy rules or oslo.config
  files (the latter at least typically have private data we wouldn't want
  to share through an API).
 
 
 We don't have a single, monolithic Service catalog (anymore) and, with
 endpoint filtering, we expect multiple service catalogs to be the norm.
 
 I want to pursue the idea of git style file identification here, (hash
 of the file as identifier) as that works to split the service catalog
 from the token, and still have multiple service catalogs, but ensure
 that they are correctly linked in remote systems.   It doesn't have to
 be hash, but it makes the process much more verifiable.  This is also
 true for policy files;  there can be more than one active at any given
 point in time, fetchable by remote identifier.  Even as we push towards
 common  rules for defininng the RBAC section, we have to be aware that
 different endpoints might need different policy files.
 
 
 
  If we could have a workflow for managing : PKI certs, Federatiomn
  mappings and  (Group only?) Role Assignments we could decentralize token
  validation.
 
  When doing the PKI tokens, we discussed this, and ended up with a t
  fetch first policy toward the certs.
 
  Puppet does not know how to get a token, so it can't call the keystone
  token-protected APIs to fetch new data.  What forms of authentication do
  the config managment systems support?  Is this an argument for tokenless
  operations against Keystone?
  In my scenario puppet (or chef or whatever) is the source of truth for
  the configuration file, not one of our services. So there's no need for
  the configuration management tool to talk to any of our services beyond
  sending the HUP signal

Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Dolph Mathews
On Mon, Mar 16, 2015 at 10:10 AM, Adam Young ayo...@redhat.com wrote:

 Oslo policy has been released as a stand alone library.  This is great, in
 that the rules engine is relatively non-applicaition specific, and I assume
 that all of the policy based project are planning to migrate over to using
 the policy library instead of the incubated version.

 Part of the push toward a more dynamic policy mechanism is figuring out
 how to fetch and cache the policy files from Keystone.  I suspect that the
 other services have the same issue.

 1.  How long should a service hold on to the cached version of the policy
 file?
 2.  How can we avoid the stampeding herd if Keystone pushes out a
 notification change event?


Why do you think this will present a problem?


 3.  How do we securely cache the file and still make it possible to debug.


Why would security for policy blobs need to differ from today's
implementation? (specifically referring to the side of the fence where
policy blobs are consumed -- which is the only side of the fence today)



 The PKI tokens have a lot of these same issues, and have a one-off
 mechanism for handling it.  We should probably look in to commonizing this
 function.

 There general mechanism should be fetch and cache but I think it should
 not be tied to keystone token validation so much as capable of using it if
 necessary.


I'm guessing that access to policy rules are typically controlled by auth
 token validated services.  Is this correct?


Can you clarify this question/concern?



 Maybe the right level of abstraction is a callback function for fetching
 the file to be cached, with the default being something that uses
 python-requests, and then an  auth plugin based alternative for those that
 require Keystone tokens.

 Before I go off and write a spec, I'd like to know what the prior art is
 here.  I'd also like to know if there oslo policy library is part of the
 plans for the other groups that are doing policy based work?

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Adam Young
Oslo policy has been released as a stand alone library.  This is great, 
in that the rules engine is relatively non-applicaition specific, and I 
assume that all of the policy based project are planning to migrate over 
to using the policy library instead of the incubated version.


Part of the push toward a more dynamic policy mechanism is figuring out 
how to fetch and cache the policy files from Keystone.  I suspect that 
the other services have the same issue.


1.  How long should a service hold on to the cached version of the 
policy file?
2.  How can we avoid the stampeding herd if Keystone pushes out a 
notification change event?

3.  How do we securely cache the file and still make it possible to debug.

The PKI tokens have a lot of these same issues, and have a one-off 
mechanism for handling it.  We should probably look in to commonizing 
this function.


There general mechanism should be fetch and cache but I think it 
should not be tied to keystone token validation so much as capable of 
using it if necessary.  I'm guessing that access to policy rules are 
typically controlled by auth token validated services.  Is this correct?


Maybe the right level of abstraction is a callback function for fetching 
the file to be cached, with the default being something that uses 
python-requests, and then an  auth plugin based alternative for those 
that require Keystone tokens.


Before I go off and write a spec, I'd like to know what the prior art is 
here.  I'd also like to know if there oslo policy library is part of the 
plans for the other groups that are doing policy based work?


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Sumit Naiksatam
On Mon, Mar 16, 2015 at 8:10 AM, Adam Young ayo...@redhat.com wrote:
 Oslo policy has been released as a stand alone library.  This is great, in
 that the rules engine is relatively non-applicaition specific, and I assume
 that all of the policy based project are planning to migrate over to using
 the policy library instead of the incubated version.

 Part of the push toward a more dynamic policy mechanism is figuring out how
 to fetch and cache the policy files from Keystone.  I suspect that the other
 services have the same issue.

 1.  How long should a service hold on to the cached version of the policy
 file?
 2.  How can we avoid the stampeding herd if Keystone pushes out a
 notification change event?
 3.  How do we securely cache the file and still make it possible to debug.

 The PKI tokens have a lot of these same issues, and have a one-off mechanism
 for handling it.  We should probably look in to commonizing this function.

 There general mechanism should be fetch and cache but I think it should
 not be tied to keystone token validation so much as capable of using it if
 necessary.  I'm guessing that access to policy rules are typically
 controlled by auth token validated services.  Is this correct?

 Maybe the right level of abstraction is a callback function for fetching the
 file to be cached, with the default being something that uses
 python-requests, and then an  auth plugin based alternative for those that
 require Keystone tokens.

 Before I go off and write a spec, I'd like to know what the prior art is
 here.  I'd also like to know if there oslo policy library is part of the
 plans for the other groups that are doing policy based work?


Thanks Adam for bringing this up. As regards the group-based-policy
(GBP) project, we leverage the access control policy just like other
projects do, so the questions you raise above are definitely relevant
to GBP. We do not manage the lifecycle of this aspect of policy, so we
hope to use whatever comes out of this discussion.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Doug Hellmann
Excerpts from Adam Young's message of 2015-03-16 11:10:01 -0400:
 Oslo policy has been released as a stand alone library.  This is great, 
 in that the rules engine is relatively non-applicaition specific, and I 
 assume that all of the policy based project are planning to migrate over 
 to using the policy library instead of the incubated version.
 
 Part of the push toward a more dynamic policy mechanism is figuring out 
 how to fetch and cache the policy files from Keystone.  I suspect that 
 the other services have the same issue.
 
 1.  How long should a service hold on to the cached version of the 
 policy file?
 2.  How can we avoid the stampeding herd if Keystone pushes out a 
 notification change event?
 3.  How do we securely cache the file and still make it possible to debug.

All of these are reasons we have so far resisted building a service to
deploy updates to oslo.config's input files, and rely on provisioning
tools to update them.

Have we consider using normal provisioning tools for pushing out
changes to policy files, and having the policy library look at the
timestamp of the file(s) to decide if it needs to re-read them
before evaluating a rule? Maybe we wouldn't always scan the file
system, but wait for some sort of signal that the scan needs to be
done.

Doug

 
 The PKI tokens have a lot of these same issues, and have a one-off 
 mechanism for handling it.  We should probably look in to commonizing 
 this function.
 
 There general mechanism should be fetch and cache but I think it 
 should not be tied to keystone token validation so much as capable of 
 using it if necessary.  I'm guessing that access to policy rules are 
 typically controlled by auth token validated services.  Is this correct?
 
 Maybe the right level of abstraction is a callback function for fetching 
 the file to be cached, with the default being something that uses 
 python-requests, and then an  auth plugin based alternative for those 
 that require Keystone tokens.
 
 Before I go off and write a spec, I'd like to know what the prior art is 
 here.  I'd also like to know if there oslo policy library is part of the 
 plans for the other groups that are doing policy based work?
 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Tim Hinrichs
Hi Adam,

For the most part we've been looking at Congress policy as complementary to 
Oslo policy, so we haven’t yet tried to incorporate Oslo policy into Congress 
(though I did some experiments with that a while back).   But looking forward, 
it would definitely be convenient if there were some way to fetch Oslo policy 
from each of the components.

More inline...

 On Mar 16, 2015, at 8:10 AM, Adam Young ayo...@redhat.com wrote:
 
 Oslo policy has been released as a stand alone library.  This is great, in 
 that the rules engine is relatively non-applicaition specific, and I assume 
 that all of the policy based project are planning to migrate over to using 
 the policy library instead of the incubated version.
 
 Part of the push toward a more dynamic policy mechanism is figuring out how 
 to fetch and cache the policy files from Keystone.  I suspect that the other 
 services have the same issue.

I thought each service would provide an API endpoint that would allow us to 
fetch their policy.json.  Why would we go through Keystone?

 
 1.  How long should a service hold on to the cached version of the policy 
 file?
 2.  How can we avoid the stampeding herd if Keystone pushes out a 
 notification change event?
 3.  How do we securely cache the file and still make it possible to debug.
 

I’m guessing performance won’t be a problem.  Policy.json is small, and there 
are likely few services listening to updates.  

Is the *content* of policy.json something sensitive that needs high security?  
Months ago there was a thread about building a tool for analyzing policy.json 
to tell people which groups they would need to execute a given API call.  
Wouldn’t that mean we’re probably not too concerned about people seeing the 
contents of policy.json?   I’m not saying we should broadcast policy.json to 
everyone who wants it, but it’s not clear to me we need to worry about 
protecting policy.json any more than administrator-level data returned by 
today’s API calls.

 The PKI tokens have a lot of these same issues, and have a one-off mechanism 
 for handling it.  We should probably look in to commonizing this function.
 
 There general mechanism should be fetch and cache but I think it should not 
 be tied to keystone token validation so much as capable of using it if 
 necessary.  I'm guessing that access to policy rules are typically controlled 
 by auth token validated services.  Is this correct?
 

If there were an API call for fetching policy, I would expect it to be 
protected just like any other API call.  And sure we’d provide whatever 
credentials are necessary to make that call.

Tim

 Maybe the right level of abstraction is a callback function for fetching the 
 file to be cached, with the default being something that uses 
 python-requests, and then an  auth plugin based alternative for those that 
 require Keystone tokens.
 
 Before I go off and write a spec, I'd like to know what the prior art is 
 here.  I'd also like to know if there oslo policy library is part of the 
 plans for the other groups that are doing policy based work?
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Doug Hellmann
Excerpts from Adam Young's message of 2015-03-16 14:17:16 -0400:
 On 03/16/2015 01:45 PM, Doug Hellmann wrote:
  All of these are reasons we have so far resisted building a service to
  deploy updates to oslo.config's input files, and rely on provisioning
  tools to update them.
 
  Have we consider using normal provisioning tools for pushing out
  changes to policy files, and having the policy library look at the
  timestamp of the file(s) to decide if it needs to re-read them
  before evaluating a rule? Maybe we wouldn't always scan the file
  system, but wait for some sort of signal that the scan needs to be
  done.
 I like this last idea.  Thew trigger needs to be app specific, I think.

I was thinking of a callback to be triggered by 'kill -HUP $pid'. We can
make a little framework for registering callbacks on signals (if there
isn't something like that already) to allow multiple refresh actions on
the signal.

 
 
  Doug
 
 I think policy files are not config files.  We've treated them as such 
 in the past as they are not dynamic, but I don't think I want to *have* 
 to do this:
 
 1.  Change policy in keystone (somehow)
 2.  Tell Puppet that there is a new file
 3.  Have puppet pick up the3 new file and sync it to the servers.

Right, I wouldn't do that. I would modify the file in my puppet
repository and then push that out all at once. Keystone would receive
the policy files the same way as the other services.

 
 
 Although I would say that we should make it easy to support this workflow.
 
 For one thing, it assumes that all of the comsuers are talking to the 
 same config management system, which is only true for a subset of the 
 services.

I'm not sure what you mean here. Do you mean that in a given deployment
you would expect some services to be configured by puppet and others to
be configured a different way?

 
 I see a case for doing this same kind of management for Many of the 
 files Keystone produces.  Service catalog is the most obvious candidate.

Yes, that's another good example, although in that case we do already
have an API that lets a cloud consumer access the service catalog data
so it might be viewed as different from the policy rules or oslo.config
files (the latter at least typically have private data we wouldn't want
to share through an API).

 
 If we could have a workflow for managing : PKI certs, Federatiomn 
 mappings and  (Group only?) Role Assignments we could decentralize token 
 validation.
 
 When doing the PKI tokens, we discussed this, and ended up with a t 
 fetch first policy toward the certs.
 
 Puppet does not know how to get a token, so it can't call the keystone 
 token-protected APIs to fetch new data.  What forms of authentication do 
 the config managment systems support?  Is this an argument for tokenless 
 operations against Keystone?

In my scenario puppet (or chef or whatever) is the source of truth for
the configuration file, not one of our services. So there's no need for
the configuration management tool to talk to any of our services beyond
sending the HUP signal telling us to re-read the file(s).

Doug

 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Adam Young

On 03/16/2015 02:59 PM, Tim Hinrichs wrote:

Hi Adam,

For the most part we've been looking at Congress policy as complementary to 
Oslo policy, so we haven’t yet tried to incorporate Oslo policy into Congress 
(though I did some experiments with that a while back).   But looking forward, 
it would definitely be convenient if there were some way to fetch Oslo policy 
from each of the components.


Good stuff.  Oslo Policy should be a generic rules engine.  We've been 
pretty good at keeping things that way, but Roles checking has Creeped 
in.  It should not mess up the other services to ignore that.


Probably a better, more scalable approach would be for an inheritance 
scheme, where by policy enforcement can inherit the core rules as well 
as rules specific to each project.  Fodder for the next summit.




More inline...


On Mar 16, 2015, at 8:10 AM, Adam Young ayo...@redhat.com wrote:

Oslo policy has been released as a stand alone library.  This is great, in that 
the rules engine is relatively non-applicaition specific, and I assume that all 
of the policy based project are planning to migrate over to using the policy 
library instead of the incubated version.

Part of the push toward a more dynamic policy mechanism is figuring out how to 
fetch and cache the policy files from Keystone.  I suspect that the other 
services have the same issue.

I thought each service would provide an API endpoint that would allow us to 
fetch their policy.json.  Why would we go through Keystone?
We agree.  Keystone here meant for the Keystone policy files.  Each 
service would supply its own form of content.

1.  How long should a service hold on to the cached version of the policy file?
2.  How can we avoid the stampeding herd if Keystone pushes out a notification 
change event?
3.  How do we securely cache the file and still make it possible to debug.


I’m guessing performance won’t be a problem.  Policy.json is small, and there 
are likely few services listening to updates.
For Keystone, it is potentially every service in the catalog.  That 
might have an impact.




Is the *content* of policy.json something sensitive that needs high security?  
Months ago there was a thread about building a tool for analyzing policy.json 
to tell people which groups they would need to execute a given API call.  
Wouldn’t that mean we’re probably not too concerned about people seeing the 
contents of policy.json?   I’m not saying we should broadcast policy.json to 
everyone who wants it, but it’s not clear to me we need to worry about 
protecting policy.json any more than administrator-level data returned by 
today’s API calls.
So, to get policy from Keysteon needs a Keystone token, which allows us 
to control who gets what  and I assume the other services want the 
potential to do the same thing.





The PKI tokens have a lot of these same issues, and have a one-off mechanism 
for handling it.  We should probably look in to commonizing this function.

There general mechanism should be fetch and cache but I think it should not 
be tied to keystone token validation so much as capable of using it if necessary.  I'm 
guessing that access to policy rules are typically controlled by auth token validated 
services.  Is this correct?


If there were an API call for fetching policy, I would expect it to be 
protected just like any other API call.  And sure we’d provide whatever 
credentials are necessary to make that call.

Agreed.



Tim


Maybe the right level of abstraction is a callback function for fetching the 
file to be cached, with the default being something that uses python-requests, 
and then an  auth plugin based alternative for those that require Keystone 
tokens.

Before I go off and write a spec, I'd like to know what the prior art is here.  
I'd also like to know if there oslo policy library is part of the plans for the 
other groups that are doing policy based work?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone][congress][group-policy] Fetching policy from a remote source

2015-03-16 Thread Adam Young

On 03/16/2015 01:45 PM, Doug Hellmann wrote:

All of these are reasons we have so far resisted building a service to
deploy updates to oslo.config's input files, and rely on provisioning
tools to update them.

Have we consider using normal provisioning tools for pushing out
changes to policy files, and having the policy library look at the
timestamp of the file(s) to decide if it needs to re-read them
before evaluating a rule? Maybe we wouldn't always scan the file
system, but wait for some sort of signal that the scan needs to be
done.

I like this last idea.  Thew trigger needs to be app specific, I think.



Doug


I think policy files are not config files.  We've treated them as such 
in the past as they are not dynamic, but I don't think I want to *have* 
to do this:


1.  Change policy in keystone (somehow)
2.  Tell Puppet that there is a new file
3.  Have puppet pick up the3 new file and sync it to the servers.


Although I would say that we should make it easy to support this workflow.

For one thing, it assumes that all of the comsuers are talking to the 
same config management system, which is only true for a subset of the 
services.


I see a case for doing this same kind of management for Many of the 
files Keystone produces.  Service catalog is the most obvious candidate.


If we could have a workflow for managing : PKI certs, Federatiomn 
mappings and  (Group only?) Role Assignments we could decentralize token 
validation.


When doing the PKI tokens, we discussed this, and ended up with a t 
fetch first policy toward the certs.


Puppet does not know how to get a token, so it can't call the keystone 
token-protected APIs to fetch new data.  What forms of authentication do 
the config managment systems support?  Is this an argument for tokenless 
operations against Keystone?





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev