Re: [openstack-dev] [qa] Cleaning OpenStack resources
Hi David, I have been working on an OpenStack resources cleanup script that allows you to wipe out all resources in a given project, I guess you could use / adapt it for your own case. It is available on github there: https://github.com/cloudwatt/ospurge You can also install it with pip (pip install ospurge). As for the floating ips, you should be able to list and remove them, by using the neutron CLI: * neutron floatingip-list * neutron floatingip-delete Forent Flament - Original Message - From: David Kranz dkr...@redhat.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Wednesday, February 19, 2014 10:15:12 PM Subject: [openstack-dev] [qa] Cleaning OpenStack resources I was looking at https://review.openstack.org/#/c/73274/1 which makes it configurable whether a brute-force cleanup of resources is done after success. This got my wondering how this should really be done. As admin, there are some resources that can be cleaned and some that I don't know how. For example, as admin you can list all servers and delete them with the --all-tenants flag. But for floating ips I don't see a way to list all of them even as admin through the apis. Is there a way that an admin can, through the api, locate all resources used by a particular tenant? -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova] Re: Hierarchicical Multitenancy Discussion
Hi Vish, You're approach looks very interesting. I especially like the idea of 'walking the tree of parent projects, to construct the set of roles'. Here are some issues that came to my mind: Regarding policy rules enforcement: Considering the following projects: * orga * orga.projecta * orga.projectb Let's assume that Joe has the following roles: * `Member` of `orga` * `admin` of `orga.projectb` Now Joe wishes to launch a VM on `orga.projecta` and grant a role to some user on `orga.projectb` (which rights he has). He would like to be able to do all of this with the same token (scoped on project `orga`?). For this scenario to be working, we would need to be able to store multiple roles (a tree of roles?) in the token, so that services would know which role is granted to the user on which project. In a first time, I guess we could stay with the roles scoped to a unique project. Joe would be able to do what he wants, by getting a first token on `orga` or `orga.projecta` with a `Member` role, then a second token on `orga.projectb` with the `admin` role. Considering quotas enforcement: Let's say we wants set the following limits: * `orga` : max 10 VMs * ̀ orga.projecta` : max 8 VMs * `orga.projectb` : max 8 VMs The idea would be that the `admin` of `orga` wishes to allow 8 VMs to projects ̀`orga.projecta` or `orga.projectb`, but doesn't care how these VMs are spread. Although he wishes to keep 2 VMs in `orga` for himself. Then to be able to enforce these quotas, Nova (and all other services) would have to keep track of the tree of quotas, and update the appropriate nodes. By the way, I'm wondering if it wouldn't be DRYer to centralize the RBAC and Quotas logic in a unique service (Keystone?). Openstack services (Nova, Cinder, ...) would just have to ask this centralized access management service whether an action is authorized for a given token? Florent Flament - Original Message - From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, February 3, 2014 10:58:28 PM Subject: [openstack-dev] [keystone][nova] Re: Hierarchicical Multitenancy Discussion Hello Again! At the meeting last week we discussed some options around getting true multitenancy in nova. The use case that we are trying to support can be described as follows: Martha, the owner of ProductionIT provides it services to multiple Enterprise clients. She would like to offer cloud services to Joe at WidgetMaster, and Sam at SuperDevShop. Joe is a Development Manager for WidgetMaster and he has multiple QA and Development teams with many users. Joe needs the ability create users, projects, and quotas, as well as the ability to list and delete resources across WidgetMaster. Martha needs to be able to set the quotas for both WidgetMaster and SuperDevShop; manage users, projects, and objects across the entire system; and set quotas for the client companies as a whole. She also needs to ensure that Joe can't see or mess with anything owned by Sam. As per the plan I outlined in the meeting I have implemented a Proof-of-Concept that would allow me to see what changes were required in nova to get scoped tenancy working. I used a simple approach of faking out heirarchy by prepending the id of the larger scope to the id of the smaller scope. Keystone uses uuids internally, but for ease of explanation I will pretend like it is using the name. I think we can all agree that ‘orga.projecta’ is more readable than ‘b04f9ea01a9944ac903526885a2666dec45674c5c2c6463dad3c0cb9d7b8a6d8’. The code basically creates the following five projects: orga orga.projecta orga.projectb orgb orgb.projecta I then modified nova to replace everywhere where it searches or limits policy by project_id to do a prefix match. This means that someone using project ‘orga’ should be able to list/delete instances in orga, orga.projecta, and orga.projectb. You can find the code here: https://github.com/vishvananda/devstack/commit/10f727ce39ef4275b613201ae1ec7655bd79dd5f https://github.com/vishvananda/nova/commit/ae4de19560b0a3718efaffb6c205c7a3c372412f Keeping in mind that this is a prototype, but I’m hoping to come to some kind of consensus as to whether this is a reasonable approach. I’ve compiled a list of pros and cons. Pros: * Very easy to understand * Minimal changes to nova * Good performance in db (prefix matching uses indexes) * Could be extended to cover more complex scenarios like multiple owners or multiple scopes Cons: * Nova has no map of the hierarchy * Moving projects would require updates to ownership inside of nova * Complex scenarios involving delegation of roles may be a bad fit * Database upgrade to hierarchy could be tricky If this seems like a reasonable set of tradeoffs, there are a few things that need to be done inside of nova bring this to a complete solution
Re: [openstack-dev] Hierarchicical Multitenancy Discussion
Hi Vishvananda, I would be interested in such a working group. Can you please confirm the meeting hour for this Friday ? I've seen 1600 UTC in your email and 2100 UTC in the wiki ( https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting ). As I'm in Europe I'd prefer 1600 UTC. Florent Flament - Original Message - From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, January 28, 2014 7:35:15 PM Subject: [openstack-dev] Hierarchicical Multitenancy Discussion Hi Everyone, I apologize for the obtuse title, but there isn't a better succinct term to describe what is needed. OpenStack has no support for multiple owners of objects. This means that a variety of private cloud use cases are simply not supported. Specifically, objects in the system can only be managed on the tenant level or globally. The key use case here is to delegate administration rights for a group of tenants to a specific user/role. There is something in Keystone called a “domain” which supports part of this functionality, but without support from all of the projects, this concept is pretty useless. In IRC today I had a brief discussion about how we could address this. I have put some details and a straw man up here: https://wiki.openstack.org/wiki/HierarchicalMultitenancy I would like to discuss this strawman and organize a group of people to get actual work done by having an irc meeting this Friday at 1600UTC. I know this time is probably a bit tough for Europe, so if we decide we need a regular meeting to discuss progress then we can vote on a better time for this meeting. https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting Please note that this is going to be an active team that produces code. We will *NOT* spend a lot of time debating approaches, and instead focus on making something that works and learning as we go. The output of this team will be a MultiTenant devstack install that actually works, so that we can ensure the features we are adding to each project work together. Vish ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain
Hi Jamie, Indeed, it is more important to be able to set quotas on every resource in the context of public clouds, than in the context of private clouds. With public cloud, we cannot assume that the user will not (deliberately or not) create millions of users if he can. I agree that there are several ways to implement quotas (e.g. in a dedicated service or on the backend). From my point of view, it makes sense to have it implemented in Keystone, since it manages these resources, as it is done with other services. Also, I understand that this may not be the priority for Keystone right now. Florent Flament - Original Message - From: Jamie Lennox jamielen...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, January 28, 2014 1:59:41 AM Subject: Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain - Original Message - From: Florent Flament florent.flament-...@cloudwatt.com To: openstack-dev@lists.openstack.org Sent: Friday, 24 January, 2014 8:07:28 AM Subject: Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain I understand that not everyone may be interested in such feature. On the other hand, some (maybe shallow) Openstack users may be interested in setting quotas on users or projects. Also, this feature wouldn't do any harm to the other users who wouldn't use it. If some contributors are willing to spend some time in adding this feature to Openstack, is there any reason not to accept it ? I have in general no problem with users/projects/domains/etc being quota-ed for a business decision (and i don't work for a provider) but as part of a more global initiative that all resource types in OpenStack can be quotaed and this would be managed by some other service (This i think would be a difficult service to write). I don't see the point in implementing this directly as a keystone feature. As Dolph mentioned these are not resource heavy concepts that we have a practical need to limit. In most situations i imagine service providers who want this have means to achieve it via the backend they use store to. Note that the idea of storing quota data in keystone has come up before and has generally never gained much traction. Jamie On Thu, 2014-01-23 at 14:55 -0600, Dolph Mathews wrote: On Thu, Jan 23, 2014 at 9:59 AM, Florent Flament florent.flament-...@cloudwatt.com wrote: Hi, Although it is true that projects and users don't consume a lot of resources, I think that there may be cases where setting quotas (possibly large) may be useful. For instance, a cloud provider may wish to prevent domain administrators to mistakingly create an infinite number of users and/or projects, by calling APIs in a bugging loop. That sounds like it would be better solved by API rate limiting, not quotas. Moreover, if quotas can be disabled, I don't see any reason not to allow cloud operators to set quotas on users and/or projects if they wishes to do so for whatever marketing reason (e.g. charging more to allow more users or projects). That's the shallow business decision I was alluding to, which I don't think we have any reason to support in-tree. Regards, Florent Flament __ From: Dolph Mathews dolph.math...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, January 23, 2014 3:09:51 PM Subject: Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain ... why? It strikes me as a rather shallow business decision to limit the number of users or projects in a system, as neither are actually cost-consuming resources. On Thu, Jan 23, 2014 at 6:43 AM, Matthieu Huin matthieu.h...@enovance.com wrote: Hello, I'd be interested in opinions and feedback on the following blueprint: https://blueprints.launchpad.net/keystone/+spec/tenants-users-quotas The idea is to add a mechanism preventing the creation of users or projects once a quota per domain is met. I believe this could be interesting for cloud providers who delegate administrative rights under domains
Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain
Hi, Although it is true that projects and users don't consume a lot of resources, I think that there may be cases where setting quotas (possibly large) may be useful. For instance, a cloud provider may wish to prevent domain administrators to mistakingly create an infinite number of users and/or projects, by calling APIs in a bugging loop. Moreover, if quotas can be disabled, I don't see any reason not to allow cloud operators to set quotas on users and/or projects if they wishes to do so for whatever marketing reason (e.g. charging more to allow more users or projects). Regards, Florent Flament - Original Message - From: Dolph Mathews dolph.math...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, January 23, 2014 3:09:51 PM Subject: Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain ... why? It strikes me as a rather shallow business decision to limit the number of users or projects in a system, as neither are actually cost-consuming resources. On Thu, Jan 23, 2014 at 6:43 AM, Matthieu Huin matthieu.h...@enovance.com wrote: Hello, I'd be interested in opinions and feedback on the following blueprint: https://blueprints.launchpad.net/keystone/+spec/tenants-users-quotas The idea is to add a mechanism preventing the creation of users or projects once a quota per domain is met. I believe this could be interesting for cloud providers who delegate administrative rights under domains to their customers. I'd like to hear the community's thoughts on this, especially in terms of viability. Many thanks, Matthieu Huin m...@enovance.com http://www.enovance.com eNovance SaS - 10 rue de la Victoire 75009 Paris - France ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] domain admin role query
Hi, Following-up on this thread (although late), I have detailed the steps allowing to have Keystone with multiple domains properly set: http://www.florentflament.com/blog/setting-keystone-v3-domains.html I hope that it may be useful for people willing to play with the Identity v3 API and domains. Florent Flament On Wed, 2013-12-18 at 12:10 -0800, Ravi Chunduru wrote: Thanks Dolph, It worked now. I specified domain id in the scope. -Ravi. On Wed, Dec 18, 2013 at 12:05 PM, Ravi Chunduru ravi...@gmail.com wrote: Hi Dolph, I dont have project yet to use in the scope. The intention is to get a token using domain admin credentials and create project using it. Thanks, -Ravi. On Wed, Dec 18, 2013 at 11:39 AM, Dolph Mathews dolph.math...@gmail.com wrote: On Wed, Dec 18, 2013 at 12:48 PM, Ravi Chunduru ravi...@gmail.com wrote: Thanks all for the information. I have now v3 policies in place, the issue is that as a domain admin I could not create a project in the domain. I get 403 unauthorized status. I see that when as a 'domain admin' request a token, the response did not have any roles. In the token request, I couldnt specify the project - as we are about to create the project in next step. Specify a domain as the scope to obtain domain-level authorization in the resulting token. See the third example under Scope: https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md#scope-scope Here is the complete request/response of all the steps done. https://gist.github.com/kumarcv/8015275 I am assuming its a bug. Please let me know your opinions. Thanks, -Ravi. On Thu, Dec 12, 2013 at 3:00 PM, Henry Nash hen...@linux.vnet.ibm.com wrote: Hi So the idea wasn't the you create a domain with the id of 'domain_admin_id', rather that you create the domain that you plan to use for your admin domain, and then paste its (auto-generated) domain_id into the policy file. Henry On 12 Dec 2013, at 03:11, Paul Belanger paul.belan...@polybeacon.com wrote: On 13-12-11 11:18 AM, Lyle, David wrote: +1 on moving the domain admin role rules to the default policy.json -David Lyle From: Dolph Mathews [mailto:dolph.math...@gmail.com] Sent: Wednesday, December 11, 2013 9:04 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [keystone] domain admin role query On Tue, Dec 10, 2013 at 10:49 PM, Jamie Lennox jamielen...@redhat.com wrote: Using the default policies it will simply check for the admin role
Re: [openstack-dev] [DevStack] Nominate Chmouel Boudjnah for core team
+1 - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, January 6, 2014 5:30:09 PM Subject: Re: [openstack-dev] [DevStack] Nominate Chmouel Boudjnah for core team On 01/06/2014 11:26 AM, Dean Troyer wrote: With the new year comes a long-overdue cleanup to the devstack-core membership and the desire to expand he team a bit. I propose to add Chmouel Boudjnah as he has been a steady contributor for some time, doing much of the Swift implementation. dt -- Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev +1 -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Gating-Failures] Docs creation is failing
Hi, The 1.2 rule for Sphinx doesn't help, as pointed out by Sean Dague here: http://lists.openstack.org/pipermail/openstack-dev/2013-December/021921.html The 1.1.99 rule he proposes works for me (on python-swiftclient): https://review.openstack.org/#/c/61378/ Regards, Florent Flament - Original Message - From: Sergey Lukjanov slukja...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, December 11, 2013 2:45:02 PM Subject: Re: [openstack-dev] [Gating-Failures] Docs creation is failing Hey, doc jobs fails because sphinx 1.2 used, all project are now applying 1.2 rule for sphinx to fix it. Here is a thread with additional info: http://lists.openstack.org/pipermail/openstack-dev/2013-December/021863.html Thanks. On Wed, Dec 11, 2013 at 1:30 PM, Wangpan hzwang...@corp.netease.com wrote: +1 http://logs.openstack.org/10/61310/2/check/gate-nova-docs/e4ca63f/console.html 2013-12-11 Wangpan 发件人： Gary Kotton gkot...@vmware.com 发送时间： 2013-12-11 15:22 主题： [openstack-dev] [Gating-Failures] Docs creation is failing 收件人： OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org 抄送： Hi, An example for this is: http://logs.openstack.org/94/59994/10/check/gate-nova-docs/b0f3910/console.html Any ideas? Thanks Gary ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Adding notifications to Horizon
Thanks for your feedback. Sandy, thank you for the link. I agree that the .start/.end notification pattern that you propose seems to be the most appropriate to monitor actions launched through the Horizon dashboard. In the case of .start/.end notifications, the decorator should do the job and has the advantage of being less intrusive than the function call. Specific notifications may then be sent through ad-hoc functions. Lance, I agree with you that generic notification patterns should be moved to Oslo-incubator. While Keystone's `notifications.py` module implements the CrUD pattern, I believe that the .start/.end pattern makes more sense in the case of Horizon. I'll try and propose a generic .start/.end pattern implementation to the Oslo-incubator, that will be based on Keystone's decorator implementation. Regards, Florent Flament - Original Message - From: Lance D Bragstad ldbra...@us.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, November 25, 2013 6:21:55 PM Subject: Re: [openstack-dev] Adding notifications to Horizon Sandy Walsh sandy.wa...@rackspace.com wrote on 11/25/2013 10:30:05 AM: From: Sandy Walsh sandy.wa...@rackspace.com To: openstack-dev@lists.openstack.org, Date: 11/25/2013 10:34 AM Subject: Re: [openstack-dev] Adding notifications to Horizon +1 on the inline method. It makes it clear when a notification should be emitted and, as you say, handles the exception handling better. This might be a good opportunity to add the decorator from Keystone's notification module to Oslo-incubator, and recycle some of that code. https://github.com/openstack/keystone/blob/master/keystone/notifications.py#L26 I know some projects may require more information to be sent in the event payload: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L783 but a general case (like Keystone) that requires only a UUID of the resource and the type of action being created, the current decorator does this pretty well. https://github.com/openstack/keystone/blob/master/keystone/assignment/core.py#L66 If this is the direction of event notifications in Horizon, it would be nice to settle on one implementation. Also, if it makes sense for Horizon, consider bracketing long-running operations in .start/.end pairs. This will help with performance tuning and early error detection. More info on well behaved notifications in here: http://www.sandywalsh.com/2013/09/notification-usage-in-openstack-report.html Great to see! -S On 11/25/2013 11:58 AM, Florent Flament wrote: Hi, I am interested in adding AMQP notifications to the Horizon dashboard, as described in the following blueprint: https://blueprints.launchpad.net/horizon/+spec/horizon-notifications There are currently several implementations in Openstack. While Nova and Cinder define `notify_about_*` methods that are called whenever a notification has to be sent, Keystone uses decorators, which send appropriate notifications when decorated methods are called. I fed the blueprint's whiteboard with an implementation proposal, based on Nova and Cinder implementation. I would be interested in having your opinion about which method would fit best, and whether these notifications make sense at all. Cheers, Florent Flament ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Best Regards, Lance Bragstad ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Adding notifications to Horizon
Hi, I am interested in adding AMQP notifications to the Horizon dashboard, as described in the following blueprint: https://blueprints.launchpad.net/horizon/+spec/horizon-notifications There are currently several implementations in Openstack. While Nova and Cinder define `notify_about_*` methods that are called whenever a notification has to be sent, Keystone uses decorators, which send appropriate notifications when decorated methods are called. I fed the blueprint's whiteboard with an implementation proposal, based on Nova and Cinder implementation. I would be interested in having your opinion about which method would fit best, and whether these notifications make sense at all. Cheers, Florent Flament ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
