Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hi there, I finally figured it out that the problem is caused by the mismatch of the VLAN range between my openstack installation and the physical switch setting. Thank you all for your kindly help. Best regards, Lily Xing On Mon, Jul 6, 2015 at 2:02 PM, Lily.Sing lily.s...@gmail.com wrote: Hi Claudiu, Thank you very much for the info. I'm going through the steps to have a try. I will give the feedback later if it works. Best regards, Lily Xing(邢莉莉) On Fri, Jun 26, 2015 at 2:06 AM, Claudiu Belu cb...@cloudbasesolutions.com wrote: Hello, ml2 conf file looks fine. nova logs look fine. neutron logs also seem fine, but this worries me a bit: 2015-06-24 20:45:18.556 4116 DEBUG hyperv.neutron.security_groups_driver [req-3786da36-6b03-433d-941e-00327839603c ] Creating port 3 rules prepare_port_filter C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\security_groups_driver.py:54 Can you run this in powershell? # if you have multiple instances spawned. Get-VMNetworkAdapterExtendedAcl -VMName instance-... | ? Action -eq Allow # only one instance Get-VMNetworkAdapterExtendedAcl | ? Action -eq Allow Can you provide the output into a pastebin as well? Now, since you have security groups enabled, there should be a rule that allow DHCP. It should look like this: ParentAdapter : Microsoft.HyperV.PowerShell.VMNetworkAdapter Direction : Inbound Action : Allow LocalIPAddress : ANY RemoteIPAddress: 10.0.0.2 (this might be different for you) LocalPort : 68-68 RemotePort : ANY Protocol : udp Weight : 65480 Stateful : True IdleSessionTimeout : 0 IsolationID: 0 ToRemove : False All the security group rules the Hyper-V Neutron Agent are received from Neutron. This DHCP rule should be amoung them as well by default. If it is not, well, there the issue lies elsewhere, in neutron. Worst case scenario, you can just add the rule: neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol udp --port-range-min 68 --port-range-max 68 --remote-ip-prefix 10.0.0.2 sg_id Introducing that rule rule should allow DHCP traffic. If that still doesn't solve the issue, the problem might not be the security groups. You could try restarting the neutron-hyperv-agent with enable_security_group=false in the neutron_hyperv_agent.conf file and check if the instances are able to receive an IP. I assume you've checked the troubleshooting section of the page I've linked last time, but just to make sure.. can you check if DHCP is enabled in the subnet the instance was created in? neutron subnet-show subnet_id Then, considering that you went with the 3 NIC Controller and 2 NIC compute node like this: Controller: eth0 - mangement eth1 - vm-data eth2 - external Compute Node: eth0 - management eth1 - vSwitch - vm-data Can you confirm that the Controller eth1 and the Compute Node vSwitch (eth1) are in the same network? Also, Controller eth1, is it promiscuous mode? At this point, we will have to get our hands dirty and do some networking troubleshooting. :) On the Hyper-V Node, run: # $INSTANCE_NAME will be instance- Get-VM -VMName $INSTANCE_NAME | Get-VMConsole In the VM Console, login, and: ifconfig # no assinged IP? then assign it manually (value from OpenStack Controller): sudo ifconfig eth0 $ASSIGNED_IP netmask $NETWORK_NETMASK up ping $DHCP_IP # let it run. On the Controller, run: # both ICMP echo request and ICMP echo reply must be visible, for all commands. sudo tcpdump -vv -eni eth1 icmp sudo tcpdump -vv -eni br-int icmp #get the tap device name sudo ip netns exec qdhcp-$NET_ID ifconfig sudo ip netns exec qdhcp-$NET_ID tcpdump -vv -ni $TAP_NAME If on the first tcpdump you do not see any ping echo request, the traffic is not getting to the Controller. If you see a ping echo request but no ping echo reply, it means that the traffic gets to the Controller, but there is reply sent back. The next commands should reveal where the reply traffic stops. The last tcpdump is the DHCP network namespace. Ideally, there will be both the request and reply messages. Hope this helps find the issue! Best regards, Claudiu Belu -- *From:* Lily.Sing [lily.s...@gmail.com] *Sent:* Thursday, June 25, 2015 8:02 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Alessandro and Claudiu, Thank you for your quick reply. The version I am running is kilo. Yes I use networking-hyperv. And the windows version is Windows Server 2012 R2. Below are the output for the commands mentioned: Get-VMSwitch Name SwitchType NetAdapterInterfaceDescription
Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hi Claudiu, Thank you very much for the info. I'm going through the steps to have a try. I will give the feedback later if it works. Best regards, Lily Xing(邢莉莉) On Fri, Jun 26, 2015 at 2:06 AM, Claudiu Belu cb...@cloudbasesolutions.com wrote: Hello, ml2 conf file looks fine. nova logs look fine. neutron logs also seem fine, but this worries me a bit: 2015-06-24 20:45:18.556 4116 DEBUG hyperv.neutron.security_groups_driver [req-3786da36-6b03-433d-941e-00327839603c ] Creating port 3 rules prepare_port_filter C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\security_groups_driver.py:54 Can you run this in powershell? # if you have multiple instances spawned. Get-VMNetworkAdapterExtendedAcl -VMName instance-... | ? Action -eq Allow # only one instance Get-VMNetworkAdapterExtendedAcl | ? Action -eq Allow Can you provide the output into a pastebin as well? Now, since you have security groups enabled, there should be a rule that allow DHCP. It should look like this: ParentAdapter : Microsoft.HyperV.PowerShell.VMNetworkAdapter Direction : Inbound Action : Allow LocalIPAddress : ANY RemoteIPAddress: 10.0.0.2 (this might be different for you) LocalPort : 68-68 RemotePort : ANY Protocol : udp Weight : 65480 Stateful : True IdleSessionTimeout : 0 IsolationID: 0 ToRemove : False All the security group rules the Hyper-V Neutron Agent are received from Neutron. This DHCP rule should be amoung them as well by default. If it is not, well, there the issue lies elsewhere, in neutron. Worst case scenario, you can just add the rule: neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol udp --port-range-min 68 --port-range-max 68 --remote-ip-prefix 10.0.0.2 sg_id Introducing that rule rule should allow DHCP traffic. If that still doesn't solve the issue, the problem might not be the security groups. You could try restarting the neutron-hyperv-agent with enable_security_group=false in the neutron_hyperv_agent.conf file and check if the instances are able to receive an IP. I assume you've checked the troubleshooting section of the page I've linked last time, but just to make sure.. can you check if DHCP is enabled in the subnet the instance was created in? neutron subnet-show subnet_id Then, considering that you went with the 3 NIC Controller and 2 NIC compute node like this: Controller: eth0 - mangement eth1 - vm-data eth2 - external Compute Node: eth0 - management eth1 - vSwitch - vm-data Can you confirm that the Controller eth1 and the Compute Node vSwitch (eth1) are in the same network? Also, Controller eth1, is it promiscuous mode? At this point, we will have to get our hands dirty and do some networking troubleshooting. :) On the Hyper-V Node, run: # $INSTANCE_NAME will be instance- Get-VM -VMName $INSTANCE_NAME | Get-VMConsole In the VM Console, login, and: ifconfig # no assinged IP? then assign it manually (value from OpenStack Controller): sudo ifconfig eth0 $ASSIGNED_IP netmask $NETWORK_NETMASK up ping $DHCP_IP # let it run. On the Controller, run: # both ICMP echo request and ICMP echo reply must be visible, for all commands. sudo tcpdump -vv -eni eth1 icmp sudo tcpdump -vv -eni br-int icmp #get the tap device name sudo ip netns exec qdhcp-$NET_ID ifconfig sudo ip netns exec qdhcp-$NET_ID tcpdump -vv -ni $TAP_NAME If on the first tcpdump you do not see any ping echo request, the traffic is not getting to the Controller. If you see a ping echo request but no ping echo reply, it means that the traffic gets to the Controller, but there is reply sent back. The next commands should reveal where the reply traffic stops. The last tcpdump is the DHCP network namespace. Ideally, there will be both the request and reply messages. Hope this helps find the issue! Best regards, Claudiu Belu -- *From:* Lily.Sing [lily.s...@gmail.com] *Sent:* Thursday, June 25, 2015 8:02 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Alessandro and Claudiu, Thank you for your quick reply. The version I am running is kilo. Yes I use networking-hyperv. And the windows version is Windows Server 2012 R2. Below are the output for the commands mentioned: Get-VMSwitch Name SwitchType NetAdapterInterfaceDescription -- -- Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch External Intel(R) Ethernet Controller X540-AT2 #3 Get-VM | Get-VMNetworkAdapter Name IsManagementOs VMName
Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hello, ml2 conf file looks fine. nova logs look fine. neutron logs also seem fine, but this worries me a bit: 2015-06-24 20:45:18.556 4116 DEBUG hyperv.neutron.security_groups_driver [req-3786da36-6b03-433d-941e-00327839603c ] Creating port 3 rules prepare_port_filter C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\security_groups_driver.py:54 Can you run this in powershell? # if you have multiple instances spawned. Get-VMNetworkAdapterExtendedAcl -VMName instance-... | ? Action -eq Allow # only one instance Get-VMNetworkAdapterExtendedAcl | ? Action -eq Allow Can you provide the output into a pastebin as well? Now, since you have security groups enabled, there should be a rule that allow DHCP. It should look like this: ParentAdapter : Microsoft.HyperV.PowerShell.VMNetworkAdapter Direction : Inbound Action : Allow LocalIPAddress : ANY RemoteIPAddress: 10.0.0.2 (this might be different for you) LocalPort : 68-68 RemotePort : ANY Protocol : udp Weight : 65480 Stateful : True IdleSessionTimeout : 0 IsolationID: 0 ToRemove : False All the security group rules the Hyper-V Neutron Agent are received from Neutron. This DHCP rule should be amoung them as well by default. If it is not, well, there the issue lies elsewhere, in neutron. Worst case scenario, you can just add the rule: neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol udp --port-range-min 68 --port-range-max 68 --remote-ip-prefix 10.0.0.2 sg_id Introducing that rule rule should allow DHCP traffic. If that still doesn't solve the issue, the problem might not be the security groups. You could try restarting the neutron-hyperv-agent with enable_security_group=false in the neutron_hyperv_agent.conf file and check if the instances are able to receive an IP. I assume you've checked the troubleshooting section of the page I've linked last time, but just to make sure.. can you check if DHCP is enabled in the subnet the instance was created in? neutron subnet-show subnet_id Then, considering that you went with the 3 NIC Controller and 2 NIC compute node like this: Controller: eth0 - mangement eth1 - vm-data eth2 - external Compute Node: eth0 - management eth1 - vSwitch - vm-data Can you confirm that the Controller eth1 and the Compute Node vSwitch (eth1) are in the same network? Also, Controller eth1, is it promiscuous mode? At this point, we will have to get our hands dirty and do some networking troubleshooting. :) On the Hyper-V Node, run: # $INSTANCE_NAME will be instance- Get-VM -VMName $INSTANCE_NAME | Get-VMConsole In the VM Console, login, and: ifconfig # no assinged IP? then assign it manually (value from OpenStack Controller): sudo ifconfig eth0 $ASSIGNED_IP netmask $NETWORK_NETMASK up ping $DHCP_IP # let it run. On the Controller, run: # both ICMP echo request and ICMP echo reply must be visible, for all commands. sudo tcpdump -vv -eni eth1 icmp sudo tcpdump -vv -eni br-int icmp #get the tap device name sudo ip netns exec qdhcp-$NET_ID ifconfig sudo ip netns exec qdhcp-$NET_ID tcpdump -vv -ni $TAP_NAME If on the first tcpdump you do not see any ping echo request, the traffic is not getting to the Controller. If you see a ping echo request but no ping echo reply, it means that the traffic gets to the Controller, but there is reply sent back. The next commands should reveal where the reply traffic stops. The last tcpdump is the DHCP network namespace. Ideally, there will be both the request and reply messages. Hope this helps find the issue! Best regards, Claudiu Belu From: Lily.Sing [lily.s...@gmail.com] Sent: Thursday, June 25, 2015 8:02 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Alessandro and Claudiu, Thank you for your quick reply. The version I am running is kilo. Yes I use networking-hyperv. And the windows version is Windows Server 2012 R2. Below are the output for the commands mentioned: Get-VMSwitch Name SwitchType NetAdapterInterfaceDescription -- -- Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch External Intel(R) Ethernet Controller X540-AT2 #3 Get-VM | Get-VMNetworkAdapter Name IsManagementOs VMNameSwitchName -- ---- f7d8c327-606b-49cd-b740-ccaef468d535 False instance-000a Intel(R) Ethernet Controller X540-AT2 #3 - Vir... And nova-compute.log and neutron-hyperv-agent.log are pasted as below: http://pastebin.com/idXuhs8a nova
[openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hi,
I setup an openstack env as one controller + network node on OL7.1 and two
compute node on windows 2012 server with cloudbase hyper-v compute node
driver. Every compute node has two nics. I created a vswitch on the second
one and use it to connect to instances. Below is my
neutron_hyper_agent.conf:
[DEFAULT]
verbose=true
control_exchange=neutron
policy_file=C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\etc\policy.json
rpc_backend=neutron.openstack.common.rpc.impl_kombu
logdir=C:\OpenStack\Log\
logfile=neutron-hyperv-agent.log
[AGENT]
polling_interval=2
physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller X540-AT2
#3 - Virtual Switch
enable_metrics_collection=false
[SECURITYGROUP]
firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver
enable_security_group=true
[oslo_messaging_rabbit]
rabbit_host=rabbit_host
rabbit_port=5672
rabbit_userid=stackrabbit
rabbit_password=admin
After launching an instance, I can check from OpenStack UI that a fixed IP
is given, but when connecting the instance from hyper-v manager, no fixed
ip is bound to any port. And below is the error message I got from
neutron-hyper-agent.log:
2015-06-23 22:59:15.954 3552 INFO hyperv.neutron.hyperv_neutron_agent
[req-a0577427-04e9-481d-94f0-027cc57eb26a ] Provisioning network
6d5ee4aa-19e2-404e-9523-cc501b20f081
2015-06-23 22:59:22.088 3552 ERROR hyperv.neutron.hyperv_neutron_agent
[req-a0577427-04e9-481d-94f0-027cc57eb26a ] Error in agent event loop
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
Traceback (most recent call last):
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py,
line 356, in daemon_loop
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
sync = self._process_network_ports(port_info)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py,
line 332, in _process_network_ports
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
resync_a = self._treat_devices_added(port_info['added'])
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py,
line 296, in _treat_devices_added
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
device_details['admin_state_up'])
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py,
line 264, in _treat_vif_port
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
physical_network, segmentation_id)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py,
line 195, in _port_bound
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
self._utils.connect_vnic_to_vswitch(map['vswitch_name'], port_id)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py,
line 86, in connect_vnic_to_vswitch
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
self._add_virt_resource(vm, port)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py,
line 100, in _add_virt_resource
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
self._check_job_status(ret_val, job_path)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
File C:\Program Files (x86)\Cloudbase
Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utils.py,
line 137, in _check_job_status
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
raise HyperVException(msg=_('Job failed with error %d') % ret_val)
2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent
HyperVException: Hyper-V Exception: Job failed with error 32775
Seems it's failed to add a port to a VM, but I don't know how to fix it.
Any idea? Thanks.
Best regards,
Lily Xing
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hi Lily, What version are you running, Icehouse, Juno, Kilo or master? A full copy of the nova-compute.log and neutron_hyperv_agent.log on a pastebin might be helpful. Additionally, your neutron.conf from the neutron server could help along with your neutron network configuration. Finally, the output of Get-VMSwitch on the Hyper-V node could provide some insight on your visrtual switch configuration. This question is deployment related and at this stage not a development topic, so IMO a more suitable location to continue the discussion would be: https://ask.openstack.org Thanks, Alessandro On 24 Jun 2015, at 10:02, Lily.Sing lily.s...@gmail.com wrote: Hi, I setup an openstack env as one controller + network node on OL7.1 and two compute node on windows 2012 server with cloudbase hyper-v compute node driver. Every compute node has two nics. I created a vswitch on the second one and use it to connect to instances. Below is my neutron_hyper_agent.conf: [DEFAULT] verbose=true control_exchange=neutron policy_file=C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\etc\policy.json rpc_backend=neutron.openstack.common.rpc.impl_kombu logdir=C:\OpenStack\Log\ logfile=neutron-hyperv-agent.log [AGENT] polling_interval=2 physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch enable_metrics_collection=false [SECURITYGROUP] firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver enable_security_group=true [oslo_messaging_rabbit] rabbit_host=rabbit_host rabbit_port=5672 rabbit_userid=stackrabbit rabbit_password=admin After launching an instance, I can check from OpenStack UI that a fixed IP is given, but when connecting the instance from hyper-v manager, no fixed ip is bound to any port. And below is the error message I got from neutron-hyper-agent.log: 2015-06-23 22:59:15.954 3552 INFO hyperv.neutron.hyperv_neutron_agent [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Provisioning network 6d5ee4aa-19e2-404e-9523-cc501b20f081 2015-06-23 22:59:22.088 3552 ERROR hyperv.neutron.hyperv_neutron_agent [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Error in agent event loop 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent Traceback (most recent call last): 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 356, in daemon_loop 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent sync = self._process_network_ports(port_info) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 332, in _process_network_ports 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent resync_a = self._treat_devices_added(port_info['added']) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 296, in _treat_devices_added 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent device_details['admin_state_up']) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 264, in _treat_vif_port 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent physical_network, segmentation_id) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 195, in _port_bound 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent self._utils.connect_vnic_to_vswitch(map['vswitch_name'], port_id) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py, line 86, in connect_vnic_to_vswitch 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent self._add_virt_resource(vm, port) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py, line 100, in _add_virt_resource 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent self._check_job_status(ret_val, job_path) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase
Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hello, From what I can see in the trace you included, it seems to be either Kilo or master (it uses networking-hyperv, which was introduced in Kilo). Also, it might be useful to know if you are using Windows Server 2012 or Windows Server 2012 R2. Running this in Powershell should yield the OS version: [environment]::OSVersion.Version 6.2 is Windows Server 2012 6.3 is Windows Server 2012 R2 As for port binding, neutron-hyperv-agent, if it fails to bind a port, it will retry to bind it. Which means that even if you see that trace, it doesn't mean that the binding failed at all. If you run Get-VM | Get-VMNetworkAdapter in Powershell, you can see that ports have been bound to the vSwitch. Also, I don't know if your environment has been properly configured, but I can tell that you have set the hyperv Mechanism Driver on your Controller node in the /etc/neutron/plugins/ml2/ml2_conf.ini and you are using a network type compatible with Hyper-V. Otherwise, neutron would flag the port with something like port binding failed and neutron-hyperv-agent wouldn't attempt to bind it. A good start would be taking a look at: http://www.cloudbase.it/quantum-hyper-v-plugin/ The linked page contains all the configuration you would need plus some troubleshooting steps that can help. Also, as Alessandro said, copies of C:\OpenStack\Log\nova-compute.log and C:\OpenStack\Log\neutron_hyperv_agent.log and the neutron server's neutron.conf file can be useful. Also, setting debug=True in the Hyper-V's neutron_hyperv_agent.conf can be useful until the issue has been solved. [DEFAULT] debug = True Let us know how it goes. Best regards, Claudiu Belu From: Alessandro Pilotti Sent: Wednesday, June 24, 2015 5:20 PM To: OpenStack Development Mailing List (not for usage questions) Cc: Claudiu Belu Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Lily, What version are you running, Icehouse, Juno, Kilo or master? A full copy of the nova-compute.log and neutron_hyperv_agent.log on a pastebin might be helpful. Additionally, your neutron.conf from the neutron server could help along with your neutron network configuration. Finally, the output of Get-VMSwitch on the Hyper-V node could provide some insight on your visrtual switch configuration. This question is deployment related and at this stage not a development topic, so IMO a more suitable location to continue the discussion would be: https://ask.openstack.org Thanks, Alessandro On 24 Jun 2015, at 10:02, Lily.Sing lily.s...@gmail.com wrote: Hi, I setup an openstack env as one controller + network node on OL7.1 and two compute node on windows 2012 server with cloudbase hyper-v compute node driver. Every compute node has two nics. I created a vswitch on the second one and use it to connect to instances. Below is my neutron_hyper_agent.conf: [DEFAULT] verbose=true control_exchange=neutron policy_file=C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\etc\policy.json rpc_backend=neutron.openstack.common.rpc.impl_kombu logdir=C:\OpenStack\Log\ logfile=neutron-hyperv-agent.log [AGENT] polling_interval=2 physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch enable_metrics_collection=false [SECURITYGROUP] firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver enable_security_group=true [oslo_messaging_rabbit] rabbit_host=rabbit_host rabbit_port=5672 rabbit_userid=stackrabbit rabbit_password=admin After launching an instance, I can check from OpenStack UI that a fixed IP is given, but when connecting the instance from hyper-v manager, no fixed ip is bound to any port. And below is the error message I got from neutron-hyper-agent.log: 2015-06-23 22:59:15.954 3552 INFO hyperv.neutron.hyperv_neutron_agent [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Provisioning network 6d5ee4aa-19e2-404e-9523-cc501b20f081 2015-06-23 22:59:22.088 3552 ERROR hyperv.neutron.hyperv_neutron_agent [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Error in agent event loop 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent Traceback (most recent call last): 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 356, in daemon_loop 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent sync = self._process_network_ports(port_info) 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py, line 332, in _process_network_ports 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent resync_a = self
Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node
Hi Alessandro and Claudiu, Thank you for your quick reply. The version I am running is kilo. Yes I use networking-hyperv. And the windows version is Windows Server 2012 R2. Below are the output for the commands mentioned: Get-VMSwitch Name SwitchType NetAdapterInterfaceDescription -- -- Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch External Intel(R) Ethernet Controller X540-AT2 #3 Get-VM | Get-VMNetworkAdapter Name IsManagementOs VMName SwitchName -- -- -- f7d8c327-606b-49cd-b740-ccaef468d535 False instance-000a Intel(R) Ethernet Controller X540-AT2 #3 - Vir... And nova-compute.log and neutron-hyperv-agent.log are pasted as below: http://pastebin.com/idXuhs8a nova-compute.log http://pastebin.com/XNUVyGCC neutron-hyperv-agent.log My ml2_conf.ini file is pasted here: http://pastebin.com/jLykFDbh Best regards, Lily Xing Best regards, Lily Xing(邢莉莉) On Wed, Jun 24, 2015 at 11:20 PM, Claudiu Belu cb...@cloudbasesolutions.com wrote: Hello, From what I can see in the trace you included, it seems to be either Kilo or master (it uses networking-hyperv, which was introduced in Kilo). Also, it might be useful to know if you are using Windows Server 2012 or Windows Server 2012 R2. Running this in Powershell should yield the OS version: [environment]::OSVersion.Version 6.2 is Windows Server 2012 6.3 is Windows Server 2012 R2 As for port binding, neutron-hyperv-agent, if it fails to bind a port, it will retry to bind it. Which means that even if you see that trace, it doesn't mean that the binding failed at all. If you run Get-VM | Get-VMNetworkAdapter in Powershell, you can see that ports have been bound to the vSwitch. Also, I don't know if your environment has been properly configured, but I can tell that you have set the hyperv Mechanism Driver on your Controller node in the /etc/neutron/plugins/ml2/ml2_conf.ini and you are using a network type compatible with Hyper-V. Otherwise, neutron would flag the port with something like port binding failed and neutron-hyperv-agent wouldn't attempt to bind it. A good start would be taking a look at: http://www.cloudbase.it/quantum-hyper-v-plugin/ The linked page contains all the configuration you would need plus some troubleshooting steps that can help. Also, as Alessandro said, copies of C:\OpenStack\Log\nova-compute.log and C:\OpenStack\Log\neutron_hyperv_agent.log and the neutron server's neutron.conf file can be useful. Also, setting debug=True in the Hyper-V's neutron_hyperv_agent.conf can be useful until the issue has been solved. [DEFAULT] debug = True Let us know how it goes. Best regards, Claudiu Belu From: Alessandro Pilotti Sent: Wednesday, June 24, 2015 5:20 PM To: OpenStack Development Mailing List (not for usage questions) Cc: Claudiu Belu Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Lily, What version are you running, Icehouse, Juno, Kilo or master? A full copy of the nova-compute.log and neutron_hyperv_agent.log on a pastebin might be helpful. Additionally, your neutron.conf from the neutron server could help along with your neutron network configuration. Finally, the output of Get-VMSwitch on the Hyper-V node could provide some insight on your visrtual switch configuration. This question is deployment related and at this stage not a development topic, so IMO a more suitable location to continue the discussion would be: https://ask.openstack.org Thanks, Alessandro On 24 Jun 2015, at 10:02, Lily.Sing lily.s...@gmail.com wrote: Hi, I setup an openstack env as one controller + network node on OL7.1 and two compute node on windows 2012 server with cloudbase hyper-v compute node driver. Every compute node has two nics. I created a vswitch on the second one and use it to connect to instances. Below is my neutron_hyper_agent.conf: [DEFAULT] verbose=true control_exchange=neutron policy_file=C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\etc\policy.json rpc_backend=neutron.openstack.common.rpc.impl_kombu logdir=C:\OpenStack\Log\ logfile=neutron-hyperv-agent.log [AGENT] polling_interval=2 physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch enable_metrics_collection=false [SECURITYGROUP] firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver enable_security_group=true [oslo_messaging_rabbit] rabbit_host=rabbit_host rabbit_port=5672 rabbit_userid=stackrabbit rabbit_password=admin After launching an instance, I can check from OpenStack UI that a fixed IP is given
