Public bug reported:

Description:
Failed to evacuate instances booted from volume

Steps to reproduce:
1.Create an instance from volume
nova boot --boot-volume <volume_id> --flavor <flavor> --nic <net-id=net-uuid> 
test

2.Force nova-compute service to down
nova service-force-down <hostname> nova-compute

3.Evacuate server from failed host
nova evacuate test

Result:
instance "test" fails to evacuate:
InvalidSharedStorage: Invalid state of instance files on shared storage

Environment:
version: Liberty
nova image backend: Ceph RBD

Error Log:
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher Traceback 
(most recent call last):
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, 
in _dispatch_and_reply
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback))
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, 
in _dispatch
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, 
in _do_dispatch
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     result = 
func(ctxt, **new_args)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in 
inner
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     return 
func(*args, **kwargs)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 353, in 
decorated_function
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 326, in 
decorated_function
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 403, in 
decorated_function
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 381, in 
decorated_function
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 369, in 
decorated_function
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2712, in 
rebuild_instance
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2751, in 
_do_rebuild_instance_with_claim
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2776, in 
_do_rebuild_instance
2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher 
InvalidSharedStorage: Invalid state of instance files on shared storage

** Affects: nova
     Importance: Undecided
         Status: New

** Description changed:

- Description
- ===========
+ Description:
  Failed to evacuate instances booted from volume
  
- 
- Steps to reproduce
- ==================
+ Steps to reproduce:
  1.Create an instance from volume
  nova boot --boot-volume <volume_id> --flavor <flavor> --nic <net-id=net-uuid> 
test
  
  2.Force nova-compute service to down
  nova service-force-down <hostname> nova-compute
  
  3.Evacuate server from failed host
  nova evacuate test
  
- 
- Result
- ======
+ Result:
  instance "test" fails to evacuate
  
  
- Environment
- ===========
+ Environment:
  version: Liberty
  nova image backend: Ceph RBD
  
  
  Error Log:
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher Traceback 
(most recent call last):
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, 
in _dispatch_and_reply
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback))
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, 
in _dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, 
in _do_dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     result 
= func(ctxt, **new_args)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in 
inner
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     return 
func(*args, **kwargs)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 353, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 326, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 403, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 381, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 369, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2712, in 
rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2751, in 
_do_rebuild_instance_with_claim
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2776, in 
_do_rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher 
InvalidSharedStorage: Invalid state of instance files on shared storage

** Description changed:

  Description:
  Failed to evacuate instances booted from volume
  
  Steps to reproduce:
  1.Create an instance from volume
  nova boot --boot-volume <volume_id> --flavor <flavor> --nic <net-id=net-uuid> 
test
  
  2.Force nova-compute service to down
  nova service-force-down <hostname> nova-compute
  
  3.Evacuate server from failed host
  nova evacuate test
  
  Result:
- instance "test" fails to evacuate
- 
+ instance "test" fails to evacuate:
+ InvalidSharedStorage: Invalid state of instance files on shared storage
  
  Environment:
  version: Liberty
  nova image backend: Ceph RBD
- 
  
  Error Log:
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher Traceback 
(most recent call last):
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, 
in _dispatch_and_reply
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback))
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, 
in _dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, 
in _do_dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     result 
= func(ctxt, **new_args)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in 
inner
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     return 
func(*args, **kwargs)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 353, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 326, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 403, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 381, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 369, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2712, in 
rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2751, in 
_do_rebuild_instance_with_claim
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2776, in 
_do_rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher 
InvalidSharedStorage: Invalid state of instance files on shared storage

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1624797

Title:
  Failed to evacuate instances booted from volume

Status in OpenStack Compute (nova):
  New

Bug description:
  Description:
  Failed to evacuate instances booted from volume

  Steps to reproduce:
  1.Create an instance from volume
  nova boot --boot-volume <volume_id> --flavor <flavor> --nic <net-id=net-uuid> 
test

  2.Force nova-compute service to down
  nova service-force-down <hostname> nova-compute

  3.Evacuate server from failed host
  nova evacuate test

  Result:
  instance "test" fails to evacuate:
  InvalidSharedStorage: Invalid state of instance files on shared storage

  Environment:
  version: Liberty
  nova image backend: Ceph RBD

  Error Log:
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher Traceback 
(most recent call last):
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, 
in _dispatch_and_reply
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback))
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, 
in _dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
executor_callback)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, 
in _do_dispatch
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     result 
= func(ctxt, **new_args)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 142, in 
inner
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     return 
func(*args, **kwargs)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 353, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 326, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 403, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 381, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in __exit__
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher     
six.reraise(self.type_, self.value, self.tb)
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 369, in 
decorated_function
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2712, in 
rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2751, in 
_do_rebuild_instance_with_claim
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2776, in 
_do_rebuild_instance
  2016-08-14 17:37:40.257 930041 ERROR oslo_messaging.rpc.dispatcher 
InvalidSharedStorage: Invalid state of instance files on shared storage

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1624797/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to