Re: [openstack-dev] [tripleo] Policy Managment and distribution.
On Tue, Mar 29, 2016 at 07:37:02PM -0400, Emilien Macchi wrote: > On Tue, Mar 29, 2016 at 6:18 PM, Adam Young wrote: > > Keystone has a policy API, but no one uses it. It allows us to associate a > > policy file with an endpoint. Upload a json blob, it gets a uuid. Associate > > the UUID with the endpoint. It could also be associated with a service, and > > then it is associated with all endpoint for that service unless explicitly > > overwritten. > > > > Assuming all of the puppet modules for all of the services support managing > > the policy files, how hard would it be to synchronize between the database > > and what we distribute to the nodes? Something along the lines of: if I > > put a file in this directory, I want puppet to use it the next time I do a > > redeploy, and I also want it uploaded to Keystone and associate with the > > endpoint? > > > > As a start, I want to be able to replace the baseline policy.json file with > > the cloudsample. We ship both. > > > > > > We have policy.pp in Puppet Keystone for this use case. > > In tripleO, we could create a parameter that uses would use to > > configure specific policies. It would be an hash and puppet will > > manage the policies. This would handle the Keystone case, but we need > > to customize all of the policy files, for all of the services, for > > example, to add the is_admin_project check. I'd like to get this mechanism > > in place before I start that work, so I can easily test changes. > > ++ > > the keystone::policy (and generally neutron::policy, nova::policy, > etc...) class is pretty robust: > > * it creates news policies or update existing ones. > * it does not delete old policies, that are already in the file. > * notify keystone service on every change. > > Please use it and let us know if we need to change something in > puppet-keystone, that would help you to deploy the use-case. I tried driving the heat::policy class today via our existing extraconfig parameter, it works like this: parameter_defaults: controllerExtraConfig: controller_classes: - heat::policy heat::policy::policies: context_is_admin: key: context_is_admin value: foo:bar Just include this in an environment file, and the policy.json for heat will be updated by puppet. The only issue I found was the json formatting is lost when puppet renders the file, it becomes a giant json blob (newlines/formatting gone). Other than that it works well, so I assume the other interfaces will work similarly for all other services. The other approch I tried was using the new DeployArtifacts interface, which isn't yet documented, but was proposed in this spec: https://github.com/openstack/tripleo-specs/blob/master/specs/mitaka/puppet-modules-deployment-via-swift.rst This was originally envisaged as a method to deploy updated puppet modules (when not delivered via package updates), but it actually works great as a general purpose deploy-files mechanism too: Here's how it works: 1. Clone https://github.com/openstack/tripleo-common (not sure if the script needed is packaged yet) 2. mkdir -p policyfiles/etc/heat/ && cd policyfiles 3. vim etc/heat/policy.json (copy the initial file from somewhere) 4. tar -cvzf policy123.tgz etc # Note the tarball is expanded from "/" so ensure path prefixes are as required 5. ./tripleo-common/scripts/upload-swift-artifacts -f policyfiles/policy123.tgz This creates a special environment file here: cat /home/stack/.tripleo/environments/deployment-artifacts.yaml *NOTE* this will *always* be used now until you delete it, you don't need to explicitly specify it via "-e" I guess there are pros/cons to both methods, and neither uses keystone as the policy store - I'm not sure how important that is vs having them stored "somewhere", e.g it seems like they could just as well be stored in a git repo or swift on the undercloud, there's not huge benefit to storing them in the overcloud unless other services support automatically consuming them? Cheers, Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Policy Managment and distribution.
On Tue, Mar 29, 2016 at 6:18 PM, Adam Young wrote: > Keystone has a policy API, but no one uses it. It allows us to associate a > policy file with an endpoint. Upload a json blob, it gets a uuid. Associate > the UUID with the endpoint. It could also be associated with a service, and > then it is associated with all endpoint for that service unless explicitly > overwritten. > > Assuming all of the puppet modules for all of the services support managing > the policy files, how hard would it be to synchronize between the database > and what we distribute to the nodes? Something along the lines of: if I > put a file in this directory, I want puppet to use it the next time I do a > redeploy, and I also want it uploaded to Keystone and associate with the > endpoint? > > As a start, I want to be able to replace the baseline policy.json file with > the cloudsample. We ship both. > > > We have policy.pp in Puppet Keystone for this use case. > In tripleO, we could create a parameter that uses would use to > configure specific policies. It would be an hash and puppet will > manage the policies. This would handle the Keystone case, but we need > to customize all of the policy files, for all of the services, for > example, to add the is_admin_project check. I'd like to get this mechanism > in place before I start that work, so I can easily test changes. ++ the keystone::policy (and generally neutron::policy, nova::policy, etc...) class is pretty robust: * it creates news policies or update existing ones. * it does not delete old policies, that are already in the file. * notify keystone service on every change. Please use it and let us know if we need to change something in puppet-keystone, that would help you to deploy the use-case. > > The workflow needs to be something like this: > > Bring up Keystone with Bootstrap. > > For each service: > Fetch its policy file from the RPM location. > Upload to Keystone. > Set the service-policy association in Keystone. > Deploy the service. > Copy over the policy file from Keystone. > > > In order to make a change, say to specialize for an endpoint: > > Upload new policy file to Keystone > Set the Endpoint Association for the Policy File > run overcloud deploy and sync all of the policy files down again. > > We don't have to use the Policy API, but we would end up re-implementing > some aspect of it. > By using the Keystone API, we also provide a way to query "what is the > policy for this endpoint?" > > I don't think this would be a radical departure from what the Rest of > OpenStack would do. > > I can see Kolla using the same approach, or something like it. > > Feedback, before I write this up as a spec? > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Policy Managment and distribution.
Keystone has a policy API, but no one uses it. It allows us to associate a policy file with an endpoint. Upload a json blob, it gets a uuid. Associate the UUID with the endpoint. It could also be associated with a service, and then it is associated with all endpoint for that service unless explicitly overwritten. Assuming all of the puppet modules for all of the services support managing the policy files, how hard would it be to synchronize between the database and what we distribute to the nodes? Something along the lines of: if I put a file in this directory, I want puppet to use it the next time I do a redeploy, and I also want it uploaded to Keystone and associate with the endpoint? As a start, I want to be able to replace the baseline policy.json file with the cloudsample. We ship both. We have policy.pp in Puppet Keystone for this use case. In tripleO, we could create a parameter that uses would use to configure specific policies. It would be an hash and puppet will manage the policies. This would handle the Keystone case, but we need to customize all of the policy files, for all of the services, for example, to add the is_admin_project check. I'd like to get this mechanism in place before I start that work, so I can easily test changes. The workflow needs to be something like this: Bring up Keystone with Bootstrap. For each service: Fetch its policy file from the RPM location. Upload to Keystone. Set the service-policy association in Keystone. Deploy the service. Copy over the policy file from Keystone. In order to make a change, say to specialize for an endpoint: Upload new policy file to Keystone Set the Endpoint Association for the Policy File run overcloud deploy and sync all of the policy files down again. We don't have to use the Policy API, but we would end up re-implementing some aspect of it. By using the Keystone API, we also provide a way to query "what is the policy for this endpoint?" I don't think this would be a radical departure from what the Rest of OpenStack would do. I can see Kolla using the same approach, or something like it. Feedback, before I write this up as a spec? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
