From: Vishvananda Ishaya []
Sent: Wednesday, February 11, 2015 5:07 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [cinder] Fixing stuck volumes - part II

On Feb 11, 2015, at 3:45 PM, D'Angelo, Scott 
<<>> wrote:

At the cinder mid-cycle it was decided that the best way to fix volumes stuck 
in 'attaching' or 'detaching' was NOT to fix the broken reset-state command. 
The doc string and help message for reset-state have been modified to warn the 
user that the tool only affects Cinder DB and can cause problems. But, 
ultimately, a separate command to 'force-detach' would be better. I've 
abandoned the original BP/spec for reset-state involving the driver.

I have looked at the existing function 'force-detach' in Cinder and it seems to 
work...except that Nova must be involved. Nova uses the BlockDeviceMapping 
table to keep track of attached volumes and, if Nova is not involved, a 
force-detach'ed volume will not be capable of being re-attached.
So, my plan is to submit a blueprint + spec for Novaclient to add a 
'force-detach' command. This is technically fairly simple and only involves 
stripping away the checks for proper state in Nova, and calling Cinder 
force-detach. I don't plan on asking for an exception to feature freeze, unless 
there is optimism from the community that this could possible get in for L.
The existing Cinder force-detach calls terminate_connection() and 
detach_volume().  I assume detach_volume() is covered by the "Volume Detach" 
minimum feature? I see many drivers have terminate_connection(), but not all. I 
believe this will not be a minimum feature, but others may disagree.

If you are going to add a force-detach command to nova, I think it would be 
good to make it detach even if the cinder request fails. Currently if you try 
to detach a volume (or terminate an instance with an attached volume), if 
cinder is down or the volume node where the volume resides is down, nova 
refuses to continue, which is pretty bad user experience.


The only problem with that is, what happens when cinder comes back up? You have 
an user/admin who think that the volume should be available, but cinder DB 
still lists as attaching | detaching and the backend may still be exporting the 
volume to the Nova compute host.
We could expose 'force-detach' through the cinderclient to fix this, but the 
admin/user might think that this is the first place to start, and leave Nova 
out, which results in a volume that cannot be re-attached.
I think that you are right about user experience, but I'm cautious about other 
problems that might result.


OpenStack Development Mailing List (not for usage questions)

OpenStack Development Mailing List (not for usage questions)

Reply via email to