[Bug 1012551] Re: Cannot delete instance in ERROR status
It's also happening in Grizzly release (nova).: root@control01:/home/user1# nova delete vm1-acc68897-1155-45c8-b8e8-37e27a9899bd HTTPConnectionPool(host='192.168.220.40', port=8774): Max retries exceeded with url: /v2/e54c7f63e9284138ad207559ca4a40e4/servers/acc68897-1155-45c8-b8e8-37e27a9899bd and the VM is stuck in the error state or just stays in Deleting state... where is this fixed? in the nova-client code or in Horizon? Thank you, Abdi -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
It's also happening in Grizzly release (nova).: root@control01:/home/user1# nova delete vm1-acc68897-1155-45c8-b8e8-37e27a9899bd HTTPConnectionPool(host='192.168.220.40', port=8774): Max retries exceeded with url: /v2/e54c7f63e9284138ad207559ca4a40e4/servers/acc68897-1155-45c8-b8e8-37e27a9899bd and the VM is stuck in the error state or just stays in Deleting state... where is this fixed? in the nova-client code or in Horizon? Thank you, Abdi -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Changed in: nova Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Changed in: nova Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
I am not able to reproduce this at all on Grizzly. ** Changed in: nova Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
I am not able to reproduce this at all on Grizzly. ** Changed in: nova (Ubuntu) Status: Triaged = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
I am not able to reproduce this at all on Grizzly. ** Changed in: nova Status: Confirmed = Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
I am not able to reproduce this at all on Grizzly. ** Changed in: nova (Ubuntu) Status: Triaged = Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Tags added: canonistack -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Tags added: canonistack -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Changed in: nova (Ubuntu) Status: New = Triaged ** Changed in: nova (Ubuntu) Importance: Undecided = Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
** Changed in: nova (Ubuntu) Status: New = Triaged ** Changed in: nova (Ubuntu) Importance: Undecided = Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
You can reproduce this problem very quickly on a freshly installed Ubuntu 12.04 system with Folsom by following the user manual line for line due to bug #1064749 ** Also affects: nova (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
Here's the debug info from Folsom. $ nova --debug delete 3935bacf-63c5-4b19-9375-f067d75d56e6 REQ: curl -i http://192.168.200.135:35357/v2.0/tokens -X POST -H Content-Type: application/json -H Accept: application/json -H User- Agent: python-novaclient -d '{auth: {tenantName: dev, passwordCredentials: {username: admin, password: dev}}}' connect: (192.168.200.135, 35357) send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 192.168.200.135:35357\r\nContent-Length: 102\r\ncontent-type: application/json\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n{auth: {tenantName: dev, passwordCredentials: {username: admin, password: dev}}}' reply: 'HTTP/1.1 200 OK\r\n' header: Vary: X-Auth-Token header: Content-Type: application/json header: Date: Tue, 09 Oct 2012 23:34:02 GMT header: Transfer-Encoding: chunked RESP:{'date': 'Tue, 09 Oct 2012 23:34:02 GMT', 'transfer-encoding': 'chunked', 'status': '200', 'content-type': 'application/json', 'vary': 'X-Auth-Token'} {access: {token: {expires: 2012-10-10T23:34:01Z, id: 9545eac2bd254984805be5c6f65a943a, tenant: {enabled: true, description: EA development, name: dev, id: 5aa21f92483b441f89b844421b639f9c}}, serviceCatalog: [{endpoints: [{adminURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c;, region: Developers, internalURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c;, id: da16351759e0471c928e313ecf1541cd, publicURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: compute, name: nova}, {endpoints: [{adminURL: http://192.168.200.135:9292/v1;, region: Developers, internalURL: http://192.168.200.135:9292/v1;, id: 618b8e6304c1460fa0c71db565975420, publicURL: http://192.168.200.135:9292/v1}], endpoints_links: [], type: image, name: glance}, {endpoints: [{adminURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c;, region: Developers, internalURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c;, id: 1889f0b8fefa4049852427e379b2e189, publicURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: volume, name: volume}, {endpoints: [{adminURL: http://192.168.200.135:8773/services/Admin;, region: Developers, internalURL: http://192.168.200.135:8773/services/Cloud;, id: 171b68c50bbf4077869aa36a1daeda6b, publicURL: http://192.168.200.135:8773/services/Cloud}], endpoints_links: [], type: ec2, name: ec2}, {endpoints: [{adminURL: http://192.168.200.135:/v1;, region: Developers, internalURL: http://192.168.200.135:/v1/AUTH_5aa21f92483b441f89b844421b639f9c;, id: 4e74115c6d004c9e8e70c0624a1b498a, publicURL: http://192.168.200.135:/v1/AUTH_5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: object-store, name: swift}, {endpoints: [{adminURL: http://192.168.200.135:35357/v2.0;, region: Developers, internalURL: http://192.168.200.135:5000/v2.0;, id: f26bb12c650747efb3e3838b6da0bb6d, publicURL: http://192.168.200.135:5000/v2.0}], endpoints_links: [], type: identity, name: keystone}], user: {username: admin, roles_links: [], id: ace5f6119a6d43778da4c2181a3ef306, roles: [{name: admin}], name: admin}, metadata: {is_admin: 0, roles: [00b8179d4c674fbfb9df5b821aaf3f83]}}} REQ: curl -i http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6 -X GET -H X-Auth-Project-Id: dev -H User-Agent: python-novaclient -H Accept: application/json -H X-Auth-Token: 9545eac2bd254984805be5c6f65a943a connect: (192.168.200.135, 8774) send: u'GET /v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6 HTTP/1.1\r\nHost: 192.168.200.135:8774\r\nx-auth-project-id: dev\r\nx-auth-token: 9545eac2bd254984805be5c6f65a943a\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-Compute-Request-Id: req-854e4a01-36f2-4800-899b-dc37bb13fa10 header: Content-Type: application/json header: Content-Length: 1476 header: Date: Tue, 09 Oct 2012 23:34:02 GMT RESP:{'status': '200', 'content-length': '1476', 'content-location': u'http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6', 'x-compute-request-id': 'req-854e4a01-36f2-4800-899b-dc37bb13fa10', 'date': 'Tue, 09 Oct 2012 23:34:02 GMT', 'content-type': 'application/json'} {server: {status: ERROR, updated: 2012-10-09T23:13:39Z, hostId: c5a32de0acc09c5e6a74b1c7679b85d2a418a3c6ea58147888ba8fe5, OS-EXT-SRV-ATTR:host: sloth, addresses: {private: []}, links: [{href: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6;, rel: self}, {href: http://192.168.200.135:8774/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6;, rel: bookmark}], key_name: doug-pikewerks, image: {id:
[Bug 1012551] Re: Cannot delete instance in ERROR status
As pointed out to me by ev0ldave in #openstack the way to fix this is to figure out exactly what the instance ID is. $ mysql -u root -p mysql USE nova; mysql SELECT id, host, hostname FROM instances; ... mysql UPDATE instances SET deleted=1, deleted_at=2012-10-10 14:56:30, vm_state=deleted WHERE id = 240; ... mysql quit; $ sudo rm -rf /var/lib/nova/instances/instance-0240 Solves the issue assuming your bad instance is ID 240. Rinse and repeat for each instance. It'd be nice to programatically fix this. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to nova in Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
You can reproduce this problem very quickly on a freshly installed Ubuntu 12.04 system with Folsom by following the user manual line for line due to bug #1064749 ** Also affects: nova (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1012551] Re: Cannot delete instance in ERROR status
Here's the debug info from Folsom. $ nova --debug delete 3935bacf-63c5-4b19-9375-f067d75d56e6 REQ: curl -i http://192.168.200.135:35357/v2.0/tokens -X POST -H Content-Type: application/json -H Accept: application/json -H User- Agent: python-novaclient -d '{auth: {tenantName: dev, passwordCredentials: {username: admin, password: dev}}}' connect: (192.168.200.135, 35357) send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 192.168.200.135:35357\r\nContent-Length: 102\r\ncontent-type: application/json\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n{auth: {tenantName: dev, passwordCredentials: {username: admin, password: dev}}}' reply: 'HTTP/1.1 200 OK\r\n' header: Vary: X-Auth-Token header: Content-Type: application/json header: Date: Tue, 09 Oct 2012 23:34:02 GMT header: Transfer-Encoding: chunked RESP:{'date': 'Tue, 09 Oct 2012 23:34:02 GMT', 'transfer-encoding': 'chunked', 'status': '200', 'content-type': 'application/json', 'vary': 'X-Auth-Token'} {access: {token: {expires: 2012-10-10T23:34:01Z, id: 9545eac2bd254984805be5c6f65a943a, tenant: {enabled: true, description: EA development, name: dev, id: 5aa21f92483b441f89b844421b639f9c}}, serviceCatalog: [{endpoints: [{adminURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c;, region: Developers, internalURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c;, id: da16351759e0471c928e313ecf1541cd, publicURL: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: compute, name: nova}, {endpoints: [{adminURL: http://192.168.200.135:9292/v1;, region: Developers, internalURL: http://192.168.200.135:9292/v1;, id: 618b8e6304c1460fa0c71db565975420, publicURL: http://192.168.200.135:9292/v1}], endpoints_links: [], type: image, name: glance}, {endpoints: [{adminURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c;, region: Developers, internalURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c;, id: 1889f0b8fefa4049852427e379b2e189, publicURL: http://192.168.200.135:8776/v1/5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: volume, name: volume}, {endpoints: [{adminURL: http://192.168.200.135:8773/services/Admin;, region: Developers, internalURL: http://192.168.200.135:8773/services/Cloud;, id: 171b68c50bbf4077869aa36a1daeda6b, publicURL: http://192.168.200.135:8773/services/Cloud}], endpoints_links: [], type: ec2, name: ec2}, {endpoints: [{adminURL: http://192.168.200.135:/v1;, region: Developers, internalURL: http://192.168.200.135:/v1/AUTH_5aa21f92483b441f89b844421b639f9c;, id: 4e74115c6d004c9e8e70c0624a1b498a, publicURL: http://192.168.200.135:/v1/AUTH_5aa21f92483b441f89b844421b639f9c}], endpoints_links: [], type: object-store, name: swift}, {endpoints: [{adminURL: http://192.168.200.135:35357/v2.0;, region: Developers, internalURL: http://192.168.200.135:5000/v2.0;, id: f26bb12c650747efb3e3838b6da0bb6d, publicURL: http://192.168.200.135:5000/v2.0}], endpoints_links: [], type: identity, name: keystone}], user: {username: admin, roles_links: [], id: ace5f6119a6d43778da4c2181a3ef306, roles: [{name: admin}], name: admin}, metadata: {is_admin: 0, roles: [00b8179d4c674fbfb9df5b821aaf3f83]}}} REQ: curl -i http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6 -X GET -H X-Auth-Project-Id: dev -H User-Agent: python-novaclient -H Accept: application/json -H X-Auth-Token: 9545eac2bd254984805be5c6f65a943a connect: (192.168.200.135, 8774) send: u'GET /v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6 HTTP/1.1\r\nHost: 192.168.200.135:8774\r\nx-auth-project-id: dev\r\nx-auth-token: 9545eac2bd254984805be5c6f65a943a\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: python-novaclient\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-Compute-Request-Id: req-854e4a01-36f2-4800-899b-dc37bb13fa10 header: Content-Type: application/json header: Content-Length: 1476 header: Date: Tue, 09 Oct 2012 23:34:02 GMT RESP:{'status': '200', 'content-length': '1476', 'content-location': u'http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6', 'x-compute-request-id': 'req-854e4a01-36f2-4800-899b-dc37bb13fa10', 'date': 'Tue, 09 Oct 2012 23:34:02 GMT', 'content-type': 'application/json'} {server: {status: ERROR, updated: 2012-10-09T23:13:39Z, hostId: c5a32de0acc09c5e6a74b1c7679b85d2a418a3c6ea58147888ba8fe5, OS-EXT-SRV-ATTR:host: sloth, addresses: {private: []}, links: [{href: http://192.168.200.135:8774/v2/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6;, rel: self}, {href: http://192.168.200.135:8774/5aa21f92483b441f89b844421b639f9c/servers/3935bacf-63c5-4b19-9375-f067d75d56e6;, rel: bookmark}], key_name: doug-pikewerks, image: {id:
[Bug 1012551] Re: Cannot delete instance in ERROR status
As pointed out to me by ev0ldave in #openstack the way to fix this is to figure out exactly what the instance ID is. $ mysql -u root -p mysql USE nova; mysql SELECT id, host, hostname FROM instances; ... mysql UPDATE instances SET deleted=1, deleted_at=2012-10-10 14:56:30, vm_state=deleted WHERE id = 240; ... mysql quit; $ sudo rm -rf /var/lib/nova/instances/instance-0240 Solves the issue assuming your bad instance is ID 240. Rinse and repeat for each instance. It'd be nice to programatically fix this. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1012551 Title: Cannot delete instance in ERROR status To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1012551/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs