Re: [openstack-dev] Resources owned by a project/tenant are not cleaned up after that project is deleted from keystone
On Mon, Feb 02, 2015 at 11:46:53AM -0600, Matt Riedemann wrote: This came up in the operators mailing list back in June [1] but given the subject probably didn't get much attention. Basically there is a really old bug [2] from Grizzly that is still a problem and affects multiple projects. A tenant can be deleted in Keystone even though other resources in other projects are under that project, and those resources aren't cleaned up. I agree this probably can be a major pain point for users. We've had to work around it in tempest by creating things like: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/cleanup_service.py and http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/cleanup.py to ensure we aren't dangling resources after a run. But, this doesn't work in all cases either. (like with tenant isolation enabled) I also know there is a stackforge project that is attempting something similar here: http://git.openstack.org/cgit/stackforge/ospurge/ It would be much nicer if the burden for doing this was taken off users and this was just handled cleanly under the covers. Keystone implemented event notifications back in Havana [3] but the other projects aren't listening on them to know when a project has been deleted and act accordingly. The bug has several people saying we should talk about this at the summit for several summits, but I can't find any discussion or summit sessions related back to the bug. Given this is an operations and cross-project issue, I'd like to bring it up again for the Vancouver summit if there is still interest (which I'm assuming there is from operators). I'd definitely support having a cross-project session on this. There is a blueprint specifically for the tenant deletion case but it's targeted at only Horizon [4]. Is anyone still working on this? Is there sufficient interest in a cross-project session at the L summit? Thinking out loud, even if nova doesn't listen to events from keystone, we could at least have a periodic task that looks for instances where the tenant no longer exists in keystone and then take some action (log a warning, shutdown/archive/, reap, etc). There is also a spec for L to transfer instance ownership [5] which could maybe come into play, but I wouldn't depend on it. [1] http://lists.openstack.org/pipermail/openstack-operators/2014-June/004559.html [2] https://bugs.launchpad.net/nova/+bug/967832 [3] https://blueprints.launchpad.net/keystone/+spec/notifications [4] https://blueprints.launchpad.net/horizon/+spec/tenant-deletion [5] https://review.openstack.org/#/c/105367/ -Matt Treinish pgp0Mz2keiApM.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Resources owned by a project/tenant are not cleaned up after that project is deleted from keystone
This came up in the operators mailing list back in June [1] but given the subject probably didn't get much attention. Basically there is a really old bug [2] from Grizzly that is still a problem and affects multiple projects. A tenant can be deleted in Keystone even though other resources in other projects are under that project, and those resources aren't cleaned up. Keystone implemented event notifications back in Havana [3] but the other projects aren't listening on them to know when a project has been deleted and act accordingly. The bug has several people saying we should talk about this at the summit for several summits, but I can't find any discussion or summit sessions related back to the bug. Given this is an operations and cross-project issue, I'd like to bring it up again for the Vancouver summit if there is still interest (which I'm assuming there is from operators). There is a blueprint specifically for the tenant deletion case but it's targeted at only Horizon [4]. Is anyone still working on this? Is there sufficient interest in a cross-project session at the L summit? Thinking out loud, even if nova doesn't listen to events from keystone, we could at least have a periodic task that looks for instances where the tenant no longer exists in keystone and then take some action (log a warning, shutdown/archive/, reap, etc). There is also a spec for L to transfer instance ownership [5] which could maybe come into play, but I wouldn't depend on it. [1] http://lists.openstack.org/pipermail/openstack-operators/2014-June/004559.html [2] https://bugs.launchpad.net/nova/+bug/967832 [3] https://blueprints.launchpad.net/keystone/+spec/notifications [4] https://blueprints.launchpad.net/horizon/+spec/tenant-deletion [5] https://review.openstack.org/#/c/105367/ -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Resources owned by a project/tenant are not cleaned up after that project is deleted from keystone
On 2/2/2015 11:46 AM, Matt Riedemann wrote: This came up in the operators mailing list back in June [1] but given the subject probably didn't get much attention. Basically there is a really old bug [2] from Grizzly that is still a problem and affects multiple projects. A tenant can be deleted in Keystone even though other resources in other projects are under that project, and those resources aren't cleaned up. Keystone implemented event notifications back in Havana [3] but the other projects aren't listening on them to know when a project has been deleted and act accordingly. The bug has several people saying we should talk about this at the summit for several summits, but I can't find any discussion or summit sessions related back to the bug. Given this is an operations and cross-project issue, I'd like to bring it up again for the Vancouver summit if there is still interest (which I'm assuming there is from operators). There is a blueprint specifically for the tenant deletion case but it's targeted at only Horizon [4]. Is anyone still working on this? Is there sufficient interest in a cross-project session at the L summit? Thinking out loud, even if nova doesn't listen to events from keystone, we could at least have a periodic task that looks for instances where the tenant no longer exists in keystone and then take some action (log a warning, shutdown/archive/, reap, etc). There is also a spec for L to transfer instance ownership [5] which could maybe come into play, but I wouldn't depend on it. [1] http://lists.openstack.org/pipermail/openstack-operators/2014-June/004559.html [2] https://bugs.launchpad.net/nova/+bug/967832 [3] https://blueprints.launchpad.net/keystone/+spec/notifications [4] https://blueprints.launchpad.net/horizon/+spec/tenant-deletion [5] https://review.openstack.org/#/c/105367/ I will apologize ahead of time for saying 'projects' for services like nova, glance, cinder, etc, while also talking about projects/tenants in keystone, I realize this is confusing. :) -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
