Re: [openstack-dev] [Nova][libvirt]when deleting instance which is in migrating state, instance files can be stay in destination node forever

2013-12-16 Thread Parthipan, Loganathan
Isn’t just handling the exception instance_not_found enough? By this time 
source would’ve been cleaned up. Destination VM resources will get cleaned up 
by the periodic task since the VM is not associated with this host. Am I 
missing something here?


From: 王宏 [mailto:w.wangho...@gmail.com]
Sent: 16 December 2013 11:32
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Nova][libvirt]when deleting instance which is in 
migrating state, instance files can be stay in destination node forever

Hi all.
When I try to fix a bug:https://bugs.launchpad.net/nova/+bug/1242961,
I get a trouble.

To reproduce the bug is very easy. Live migrate a vm in block_migration mode,
and then delelte the vm immediately.

The reason of this bug is as follow:
1. Because live migrate costs more time, so the vm will be deleted sucessfully
   before live migrate complete. And then, we will get an exception while live
   migrating.
2. After live migrate failed, we start to rollback. But, in the rollback method
   we will get or modify the info of vm from db. Because the vm has been deleted
   already, so we will get instance_not_found exception and rollback will be
   faild too.

I have two ways to fix the bug:
i)Add check in nova-api. When try to delete a vm, we return an error message if
the vm_state is LIVE_MIGRATING. This way is very simple, but need to carefully
consider. I have found a related discussion:
http://lists.openstack.org/pipermail/openstack-dev/2013-October/017454.html, but
it has no result in the discussion.
ii)Before live migrate we get all the data needed by rollback method, and add a
new rollback method. The new method will clean up resources at destination based
on the above data(The resouces at source has been already cleaned up by
deleting).

I have no idea whitch one I should choose. Or, any other ideas?:)

Regards,
wanghong
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][libvirt]when deleting instance which is in migrating state, instance files can be stay in destination node forever

2013-12-16 Thread Vladik Romanovsky
I would block it in the API or have the API cancelling the migration first. 
I don't see a reason why to start an operation that is meant to fail, which 
also has a complex chain of event, following it failure.

Regardless of the above, I think that the suggested exception handling is 
needed in any case.


Vladik

- Original Message -
 From: Loganathan Parthipan parthi...@hp.com
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Sent: Monday, 16 December, 2013 8:25:09 AM
 Subject: Re: [openstack-dev] [Nova][libvirt]when deleting instance which is 
 in migrating state, instance files can be
 stay in destination node forever
 
 
 
 Isn’t just handling the exception instance_not_found enough? By this time
 source would’ve been cleaned up. Destination VM resources will get cleaned
 up by the periodic task since the VM is not associated with this host. Am I
 missing something here?
 
 
 
 
 
 
 From: 王宏 [mailto:w.wangho...@gmail.com]
 Sent: 16 December 2013 11:32
 To: openstack-dev@lists.openstack.org
 Subject: [openstack-dev] [Nova][libvirt]when deleting instance which is in
 migrating state, instance files can be stay in destination node forever
 
 
 
 
 
 Hi all.
 
 
 When I try to fix a bug: https://bugs.launchpad.net/nova/+bug/1242961 ,
 
 
 I get a trouble.
 
 
 
 
 
 To reproduce the bug is very easy. Live migrate a vm in block_migration mode,
 
 
 and then delelte the vm immediately.
 
 
 
 
 
 The reason of this bug is as follow:
 
 
 1. Because live migrate costs more time, so the vm will be deleted
 sucessfully
 
 
 before live migrate complete. And then, we will get an exception while live
 
 
 migrating.
 
 
 2. After live migrate failed, we start to rollback. But, in the rollback
 method
 
 
 we will get or modify the info of vm from db. Because the vm has been deleted
 
 
 already, so we will get instance_not_found exception and rollback will be
 
 
 faild too.
 
 
 
 
 
 I have two ways to fix the bug:
 
 
 i)Add check in nova-api. When try to delete a vm, we return an error message
 if
 
 
 the vm_state is LIVE_MIGRATING. This way is very simple, but need to
 carefully
 
 
 consider. I have found a related discussion:
 
 
 http://lists.openstack.org/pipermail/openstack-dev/2013-October/017454.html ,
 but
 
 
 it has no result in the discussion.
 
 
 ii)Before live migrate we get all the data needed by rollback method, and add
 a
 
 
 new rollback method. The new method will clean up resources at destination
 based
 
 
 on the above data(The resouces at source has been already cleaned up by
 
 
 deleting).
 
 
 
 
 
 I have no idea whitch one I should choose. Or, any other ideas?:)
 
 
 
 
 
 Regards,
 
 
 wanghong
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][libvirt]when deleting instance which is in migrating state, instance files can be stay in destination node forever

2013-12-16 Thread Yaguang Tang
could we use Taskflow https://wiki.openstack.org/wiki/TaskFlow to manage
task state and resource for this kind of tasks in Nova? Cinder has been an
pilot to use Taskflow for volume backup tasks. anyone interested in this
suggestion or has done some research to improve the live migration
workflow?


2013/12/17 Vladik Romanovsky vladik.romanov...@enovance.com

 I would block it in the API or have the API cancelling the migration first.
 I don't see a reason why to start an operation that is meant to fail,
 which also has a complex chain of event, following it failure.

 Regardless of the above, I think that the suggested exception handling is
 needed in any case.


 Vladik

 - Original Message -
  From: Loganathan Parthipan parthi...@hp.com
  To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
  Sent: Monday, 16 December, 2013 8:25:09 AM
  Subject: Re: [openstack-dev] [Nova][libvirt]when deleting instance which
 is in migrating state, instance files can be
  stay in destination node forever
 
 
 
  Isn’t just handling the exception instance_not_found enough? By this time
  source would’ve been cleaned up. Destination VM resources will get
 cleaned
  up by the periodic task since the VM is not associated with this host.
 Am I
  missing something here?
 
 
 
 
 
 
  From: 王宏 [mailto:w.wangho...@gmail.com]
  Sent: 16 December 2013 11:32
  To: openstack-dev@lists.openstack.org
  Subject: [openstack-dev] [Nova][libvirt]when deleting instance which is
 in
  migrating state, instance files can be stay in destination node forever
 
 
 
 
 
  Hi all.
 
 
  When I try to fix a bug: https://bugs.launchpad.net/nova/+bug/1242961 ,
 
 
  I get a trouble.
 
 
 
 
 
  To reproduce the bug is very easy. Live migrate a vm in block_migration
 mode,
 
 
  and then delelte the vm immediately.
 
 
 
 
 
  The reason of this bug is as follow:
 
 
  1. Because live migrate costs more time, so the vm will be deleted
  sucessfully
 
 
  before live migrate complete. And then, we will get an exception while
 live
 
 
  migrating.
 
 
  2. After live migrate failed, we start to rollback. But, in the rollback
  method
 
 
  we will get or modify the info of vm from db. Because the vm has been
 deleted
 
 
  already, so we will get instance_not_found exception and rollback will be
 
 
  faild too.
 
 
 
 
 
  I have two ways to fix the bug:
 
 
  i)Add check in nova-api. When try to delete a vm, we return an error
 message
  if
 
 
  the vm_state is LIVE_MIGRATING. This way is very simple, but need to
  carefully
 
 
  consider. I have found a related discussion:
 
 
 
 http://lists.openstack.org/pipermail/openstack-dev/2013-October/017454.html,
  but
 
 
  it has no result in the discussion.
 
 
  ii)Before live migrate we get all the data needed by rollback method,
 and add
  a
 
 
  new rollback method. The new method will clean up resources at
 destination
  based
 
 
  on the above data(The resouces at source has been already cleaned up by
 
 
  deleting).
 
 
 
 
 
  I have no idea whitch one I should choose. Or, any other ideas?:)
 
 
 
 
 
  Regards,
 
 
  wanghong
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Tang Yaguang

Canonical Ltd. | www.ubuntu.com | www.canonical.com
Mobile:  +86 152 1094 6968
gpg key: 0x187F664F
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][libvirt]when deleting instance which is in migrating state, instance files can be stay in destination node forever

2013-12-16 Thread haruka tanizawa
Hi!

Actually, I have already filed cancel of LiveMigration with using taskflow
[0].
But my approach to blueprint was not so good at that time.
So, I rewrote this blueprint from also point of russelb.
I want to repush and wanted to approved.

If you have any suggestions, ideas etc...
I appreciate it :)

Sincerely, Haruka Tanizawa


[0] https://blueprints.launchpad.net/nova/+spec/feature-of-cancel


2013/12/17 Yaguang Tang yaguang.t...@canonical.com

 could we use Taskflow https://wiki.openstack.org/wiki/TaskFlow to
 manage task state and resource for this kind of tasks in Nova? Cinder has
 been an pilot to use Taskflow for volume backup tasks. anyone interested in
 this suggestion or has done some research to improve the live migration
 workflow?


 2013/12/17 Vladik Romanovsky vladik.romanov...@enovance.com

 I would block it in the API or have the API cancelling the migration
 first.
 I don't see a reason why to start an operation that is meant to fail,
 which also has a complex chain of event, following it failure.

 Regardless of the above, I think that the suggested exception handling is
 needed in any case.


 Vladik

 - Original Message -
  From: Loganathan Parthipan parthi...@hp.com
  To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
  Sent: Monday, 16 December, 2013 8:25:09 AM
  Subject: Re: [openstack-dev] [Nova][libvirt]when deleting instance
 which is in migrating state, instance files can be
  stay in destination node forever
 
 
 
  Isn’t just handling the exception instance_not_found enough? By this
 time
  source would’ve been cleaned up. Destination VM resources will get
 cleaned
  up by the periodic task since the VM is not associated with this host.
 Am I
  missing something here?
 
 
 
 
 
 
  From: 王宏 [mailto:w.wangho...@gmail.com]
  Sent: 16 December 2013 11:32
  To: openstack-dev@lists.openstack.org
  Subject: [openstack-dev] [Nova][libvirt]when deleting instance which is
 in
  migrating state, instance files can be stay in destination node forever
 
 
 
 
 
  Hi all.
 
 
  When I try to fix a bug: https://bugs.launchpad.net/nova/+bug/1242961 ,
 
 
  I get a trouble.
 
 
 
 
 
  To reproduce the bug is very easy. Live migrate a vm in block_migration
 mode,
 
 
  and then delelte the vm immediately.
 
 
 
 
 
  The reason of this bug is as follow:
 
 
  1. Because live migrate costs more time, so the vm will be deleted
  sucessfully
 
 
  before live migrate complete. And then, we will get an exception while
 live
 
 
  migrating.
 
 
  2. After live migrate failed, we start to rollback. But, in the rollback
  method
 
 
  we will get or modify the info of vm from db. Because the vm has been
 deleted
 
 
  already, so we will get instance_not_found exception and rollback will
 be
 
 
  faild too.
 
 
 
 
 
  I have two ways to fix the bug:
 
 
  i)Add check in nova-api. When try to delete a vm, we return an error
 message
  if
 
 
  the vm_state is LIVE_MIGRATING. This way is very simple, but need to
  carefully
 
 
  consider. I have found a related discussion:
 
 
 
 http://lists.openstack.org/pipermail/openstack-dev/2013-October/017454.html,
  but
 
 
  it has no result in the discussion.
 
 
  ii)Before live migrate we get all the data needed by rollback method,
 and add
  a
 
 
  new rollback method. The new method will clean up resources at
 destination
  based
 
 
  on the above data(The resouces at source has been already cleaned up by
 
 
  deleting).
 
 
 
 
 
  I have no idea whitch one I should choose. Or, any other ideas?:)
 
 
 
 
 
  Regards,
 
 
  wanghong
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Tang Yaguang

 Canonical Ltd. | www.ubuntu.com | www.canonical.com
 Mobile:  +86 152 1094 6968
 gpg key: 0x187F664F


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev