[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Milestone: essex-rc1 = 2012.1 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova (Ubuntu) Status: Confirmed = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
fwiw following recent fix might have changed the issue: commit 22b484a6d0b65c2a41fd4c730a5ebddf98c70c84 Author: Vishvananda Ishaya vishvana...@gmail.com Date: Wed Mar 14 09:26:40 2012 -0700 Allow errored volumes to be deleted * Allow volumes with no host set to be deleted from db * Allow volumes in state error to be deleted * Replicates code from nova-manage command * Fixes bug 953594 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
we discussed on irc. This is a separate bug ** Changed in: nova Importance: Undecided = Medium ** Tags added: essec-rc-potential -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
Bogged down with other things ATM, but spent some time looking at this
last night.
nova.compute.manager._shutdown_instance() raises an exception if the
instance is already in POWEROFF state. It looks like this conditional
has existed forever:
if current_power_state == power_state.SHUTOFF:
self.db.instance_destroy(context, instance_id)
_msg = _('trying to destroy already destroyed instance: %s')
raise exception.Invalid(_msg % instance_uuid)
It currently does nothing to cleanup bdms and inform nova-volume that the
volume is free. We can certainly do that from the compute manager when the
condition is met, volumes are freed up to be used elsewhere. The problem
there is the iSCSI sessions are never cleaned up from the compute host.
Reattaching the volume to another instance on the same compute node works okay
since https://review.openstack.org/#change,4611, but having dangling iSCSI
sessions hanging around seems dirty.
Looking at the libvirt compute driver, it appears the
l_shutdown_instance()'s later call to driver.destroy() handles
terminating an already SHUTOFF'd instance just fine, and also properly
cleans up its iscsi connections, among other things. It would appear
that, in teh case of libvirt, the condition raised above is obsolete.
But I'm unsure if this is true for other compute drivers and hesitant to
propose dropping it without confirmation.
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to nova in Ubuntu.
https://bugs.launchpad.net/bugs/954692
Title:
cannot detach volume from terminated instance
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/954692/+subscriptions
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Milestone: None = essex-rc1 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
I think removing that block of code makes sense. However, a few lines below, I would add a try/except around the call to self.driver.destroy(). It looks like it could raise if the instance is not created at the driver level, which could be the case for an instance that's in ERROR state. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
Chris- I'm a little uncomfortable wrapping self.driver.destroy in a try/except block until there is a common exception language virt drivers can use to raise specific, fatal exceptions. I'm thinking of a scenario where libvirt/xs cannot shutdown instance, raises, and compute continues on with volume disassociation. The stuck instance may still be powered on and scribbling all over its disk, meanwhile the volume has been attached to another instance. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
Makes sense. Thanks. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Assignee: Adam Gandelman (gandelman-a) = Vish Ishaya (vishvananda) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Assignee: Vish Ishaya (vishvananda) = Adam Gandelman (gandelman-a) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
Reviewed: https://review.openstack.org/5437 Committed: http://github.com/openstack/nova/commit/6657f70ee3e792b39e45a2a96fb5d4b380f0ae91 Submitter: Jenkins Branch:master commit 6657f70ee3e792b39e45a2a96fb5d4b380f0ae91 Author: Adam Gandelman ad...@canonical.com Date: Thu Mar 15 18:08:35 2012 -0700 Allow proper instance cleanup if state == SHUTOFF Removes an obsolete check for instance's power_state on shutdown_instance(). With it in place, volume detachment and disassociation never takes place. Compute should instead rely on virt drivers to handle this case and raise accordingly. libvirt's destroy() currently handles powered off instances fine, and properly detaches any existing volume connections. Fixes bug 954692 Change-Id: I200d5b2073e5b52a9733d8324d016b14bdc96067 ** Changed in: nova Status: In Progress = Fix Committed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova (Ubuntu) Importance: Undecided = Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
Steps to reproduce: 1. create volume 2. boot an instance (tested with cirros) 3. attach volume 4. in instance: 'sudo poweroff' 5. after kvm machine has stopped on compute node, terminate instance. results in traceback on compute.log: (nova.rpc.amqp): TRACE: File /usr/lib/python2.7/dist-packages/nova/compute/manager.py, line 719, in _delete_instance (nova.rpc.amqp): TRACE: self._shutdown_instance(context, instance, 'Terminating') (nova.rpc.amqp): TRACE: File /usr/lib/python2.7/dist-packages/nova/compute/manager.py, line 681, in _shutdown_instance (nova.rpc.amqp): TRACE: raise exception.Invalid(_msg % instance_uuid) (nova.rpc.amqp): TRACE: Invalid: trying to destroy already destroyed instance: b929bf81-65ee-46e0-8c07-5aae49a0213c 6. instance is destroyed, volume is still 'in-use' 7. detach volume, results in traceback in original bug report ** Also affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 954692] Re: cannot detach volume from terminated instance
** Changed in: nova Status: New = In Progress ** Changed in: nova Assignee: (unassigned) = Adam Gandelman (gandelman-a) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/954692 Title: cannot detach volume from terminated instance To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/954692/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
