Hi Felix,
Below is my setup
1. compute node : kvm, neutron plugins, nova-compute ( 3 ethernet card as I
mentioned earlier)
2. Neutron Node: nova services, neutron services, neutron-server ( 3 ethernet
card as I mentioned earlier)
3. keystone node : remaining services
As per the update given by you I added vif_plugging_is_fatal: true" in my
nova.conf of compute node and neutron node .
But I got below error
==nova-compute.log ======================
2014-07-09 18:15:23.222 5632 WARNING nova.virt.libvirt.driver
[req-2cfbe49e-99bb-4e0e-8d94-a0265f8fb1c6 b9520b3afe3b44ddb438392b132d8836
6d5937db638a4ef3ab4b40a4515ec557] Timeout waiting for vif plugging callback for
instance 4f8ff29f-06a8-4918-b416-9d6551c4d15e
2014-07-09 18:15:24.064 5632 INFO nova.virt.libvirt.driver
[req-2cfbe49e-99bb-4e0e-8d94-a0265f8fb1c6 b9520b3afe3b44ddb438392b132d8836
6d5937db638a4ef3ab4b40a4515ec557] [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] Deleting instance files
/var/lib/nova/instances/4f8ff29f-06a8-4918-b416-9d6551c4d15e
2014-07-09 18:15:24.065 5632 INFO nova.virt.libvirt.driver
[req-2cfbe49e-99bb-4e0e-8d94-a0265f8fb1c6 b9520b3afe3b44ddb438392b132d8836
6d5937db638a4ef3ab4b40a4515ec557] [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] Deletion of
/var/lib/nova/instances/4f8ff29f-06a8-4918-b416-9d6551c4d15e complete
2014-07-09 18:15:24.114 5632 ERROR nova.compute.manager
[req-2cfbe49e-99bb-4e0e-8d94-a0265f8fb1c6 b9520b3afe3b44ddb438392b132d8836
6d5937db638a4ef3ab4b40a4515ec557] [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] Instance failed to spawn
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] Traceback (most recent call last):
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1720, in _spawn
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] block_device_info)
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2253, in
spawn
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] block_device_info)
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3663, in
_create_domain_and_network
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] raise
exception.VirtualInterfaceCreateException()
2014-07-09 18:15:24.114 5632 TRACE nova.compute.manager [instance:
4f8ff29f-06a8-4918-b416-9d6551c4d15e] VirtualInterfaceCreateException: Virtual
Interface creation failed
====neutron-server.log================
2014-07-09 18:15:16.015 1289 ERROR neutron.notifiers.nova [-] Failed to notify
nova on events: [{'status': 'completed', 'tag':
u'0d91fed9-cea9-4e69-b080-b959791d9682', 'name': 'network-vif-plugged',
'server_uuid': u'4f8ff29f-06a8-4918-b416-9d6551c4d15e'}]
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova Traceback (most
recent call last):
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py", line 187, in
send_events
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova batched_events)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/v1_1/contrib/server_external_events.py",
line 39, in create
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova return_raw=True)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/base.py", line 152, in _create
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova _resp, body =
self.api.client.post(url, body=body)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 286, in post
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova return
self._cs_request(url, 'POST', **kwargs)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 249, in
_cs_request
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova
self.authenticate()
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 382, in
authenticate
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova auth_url =
self._v2_auth(auth_url)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 469, in _v2_auth
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova return
self._authenticate(url, body)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 482, in
_authenticate
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova **kwargs)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 242, in
_time_request
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova resp, body =
self.request(url, method, **kwargs)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova File
"/usr/lib/python2.7/dist-packages/novaclient/client.py", line 236, in request
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova raise
exceptions.from_response(resp, body, url, method)
2014-07-09 18:15:16.015 1289 TRACE neutron.notifiers.nova Unauthorized: The
request you have made requires authentication. (HTTP 401)
Let me know if I need to add anything ekse in configuration files .
Regards,
Malleshi CN
-----Original Message-----
From: Channappa Negalur, M.
Sent: Wednesday, July 09, 2014 4:24 PM
To: 'Felix Lee'
Subject: RE: [Openstack] Icehouse : vm spawn failed
Thanks Felix,
Let me check with value : "vif_plugging_is_fatal: true"
Regards,
Malleshi CN
-----Original Message-----
From: Felix Lee [mailto:[email protected]]
Sent: Wednesday, July 09, 2014 3:21 PM
To: Channappa Negalur, M.; [email protected]
Subject: Re: [Openstack] Icehouse : vm spawn failed
Hi, Malleshi,
I am not sure if you also use neutron with openvswitch plugin, well, we once
met similar problem on our testbed before, but it's because our Neutron network
node was also compute node, so, when virtual machine was launched to neutron
network node, after it injected vif into openvswitch configuration, somehow it
will cause both hypervisor and VM network to be unstable(I didn't look into
neutron code, presumingly, there is conflict in openvswitch datapath
configuration between VM and hypervisor. ), I am not sure if that is also your
case, but if you put neutron network node and nova compute together, you might
want to separate neutron network from nova compute.
BTW, if you configure notify_nova_on_port_status_changes and
notify_nova_on_port_data_changes properly in nova.conf, you don't need
"vif_plugging_is_fatal: true"
Best regards,
Felix Lee ~
On 2014年07月09日 09:34, [email protected] wrote:
> Hello Felix,
>
> 1. Which network service do you use?
> Ans: Neutron
>
> 2. do you have separate network interfaces for VM instances and hypervisor?
> Ans: Yes I do have 3 interfaces on compute node and network node .
> 1.eth0 : Management network
> 2.eth1: external network
> 3.eth2: for vm communication
>
>
> Hypervisor used in nova-compute is KVM
>
> Regards,
> Malleshi CN
>
> -----Original Message-----
> From: Felix Lee [mailto:[email protected]]
> Sent: Wednesday, July 09, 2014 12:56 PM
> To: Channappa Negalur, M.; [email protected]
> Subject: Re: [Openstack] Icehouse : vm spawn failed
>
> Hi, Malleshi,
> Which network service do you use?
> Neutron or Nova network?
> And, do you have separate network interfaces for VM instances and hypervisor?
>
>
> Best regards,
> Felix Lee ~
>
> On 2014撟 07 07 13:48, [email protected] wrote:
>> Hello all,
>>
>> I am installing icehouse setup on 3 node. My entire set up is on
>> virtual environment.
>>
>> ======Nova-compute.log ================
>>
>> *WARNING nova.virt.libvirt.driver [-] Periodic task is updating the
>> host stat, it is trying to get disk instance-0000000b, but disk file
>> was removed by concurrent operations such as resize*
>>
>> **
>>
>> *WARNING nova.virt.disk.vfs.guestfs
>> [req-727a676a-ca15-4cb5-8fc4-73dbf307a14f
>> 4f783fbf23304d1682e820740b99f954 7e2d68b079be44048bedd223b3683f19]
>> Failed to close augeas aug_close: call launch before using this
>> function*
>>
>> *(in guestfish, don't forget to use the 'run' command)*
>>
>> 2014-07-07 17:02:23.852 2916 WARNING nova.virt.libvirt.driver
>> [req-df5b6fb1-5304-4f65-bfcf-fbff0ec7298f
>> 4f783fbf23304d1682e820740b99f954 7e2d68b079be44048bedd223b3683f19]
>> *Timeout waiting for vif plugging callback for instance*
>> 75515a86-ba63-4c95-8065-1add9da1f314
>>
>> 2014-07-07 17:02:24.693 2916 INFO nova.virt.libvirt.driver
>> [req-df5b6fb1-5304-4f65-bfcf-fbff0ec7298f
>> 4f783fbf23304d1682e820740b99f954 7e2d68b079be44048bedd223b3683f19]
>> [instance: 75515a86-ba63-4c95-8065-1add9da1f314] Deleting instance
>> files
>> /var/lib/nova/instances/75515a86-ba63-4c95-8065-1add9da1f314
>>
>> 2014-07-07 17:02:24.693 2916 INFO nova.virt.libvirt.driver
>> [req-df5b6fb1-5304-4f65-bfcf-fbff0ec7298f
>> 4f783fbf23304d1682e820740b99f954 7e2d68b079be44048bedd223b3683f19]
>> [instance: 75515a86-ba63-4c95-8065-1add9da1f314] Deletion of
>> /var/lib/nova/instances/75515a86-ba63-4c95-8065-1add9da1f314 complete
>>
>> 2014-07-07 17:02:24.771 2916 ERROR nova.compute.manager
>> [req-df5b6fb1-5304-4f65-bfcf-fbff0ec7298f
>> 4f783fbf23304d1682e820740b99f954 7e2d68b079be44048bedd223b3683f19]
>> [instance: 75515a86-ba63-4c95-8065-1add9da1f314] *Instance failed to
>> spawn*
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] Traceback (most recent call last):
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] File
>> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
>> 1720, in _spawn
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] block_device_info)
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] File
>> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
>> 2253, in spawn
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] block_device_info)
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] File
>> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
>> 3663, in _create_domain_and_network
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] raise
>> exception.VirtualInterfaceCreateException()
>>
>> 2014-07-07 17:02:24.771 2916 TRACE nova.compute.manager [instance:
>> 75515a86-ba63-4c95-8065-1add9da1f314] VirtualInterfaceCreateException:
>> Virtual Interface creation failed
>>
>> I got the below error from horizon
>>
>> *virtual interface creation failed *.
>>
>> As per the link
>> https://ask.openstack.org/en/question/26985/icehouse-virtual-interfac
>> e-creation-failed/
>>
>> If I add below entries in my nova.conf , I am able to launch a
>> instance but I ll be disconnected from my host machine and I will not be
>> able to
>> connect via ssh aslo again I need to reboot my host machine to
>> connect ..
>>
>> *vif_plugging_is_fatal: false *
>>
>> *vif_plugging_timeout: 0*
>>
>> is there method to fix this ..? Please let me know ,
>>
>> Regards,
>>
>> Malleshi CN
>>
>>
>> ---------------------------------------------------------------------
>> ---
>>
>> This message is for the designated recipient only and may contain
>> privileged, proprietary, or otherwise confidential information. If
>> you have received it in error, please notify the sender immediately
>> and delete the original. Any other use of the e-mail by you is prohibited.
>> Where allowed by local law, electronic communications with Accenture
>> and its affiliates, including e-mail and instant messaging (including
>> content), may be scanned by our systems for the purposes of
>> information security and assessment of internal compliance with Accenture
>> policy.
>> _____________________________________________________________________
>> _________________
>>
>> www.accenture.com
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : [email protected]
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
>
> --
> Felix H.T Lee Academia Sinica Grid & Cloud.
> Tel: +886-2-27898308
> Office: Room P111, Institute of Physics, 128 Academia Road, Section 2,
> Nankang, Taipei 115, Taiwan
>
> ________________________________
>
> This message is for the designated recipient only and may contain privileged,
> proprietary, or otherwise confidential information. If you have received it
> in error, please notify the sender immediately and delete the original. Any
> other use of the e-mail by you is prohibited. Where allowed by local law,
> electronic communications with Accenture and its affiliates, including e-mail
> and instant messaging (including content), may be scanned by our systems for
> the purposes of information security and assessment of internal compliance
> with Accenture policy.
> ______________________________________________________________________
> ________________
>
> www.accenture.com
>
--
Felix H.T Lee Academia Sinica Grid & Cloud.
Tel: +886-2-27898308
Office: Room P111, Institute of Physics, 128 Academia Road, Section 2, Nankang,
Taipei 115, Taiwan
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : [email protected]
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
