Hello, ml2 conf file looks fine.
nova logs look fine. neutron logs also seem fine, but this worries me a bit: 2015-06-24 20:45:18.556 4116 DEBUG hyperv.neutron.security_groups_driver [req-3786da36-6b03-433d-941e-00327839603c ] Creating port 3 rules prepare_port_filter C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\security_groups_driver.py:54 Can you run this in powershell? # if you have multiple instances spawned. Get-VMNetworkAdapterExtendedAcl -VMName instance-... | ? Action -eq Allow # only one instance Get-VMNetworkAdapterExtendedAcl | ? Action -eq Allow Can you provide the output into a pastebin as well? Now, since you have security groups enabled, there should be a rule that allow DHCP. It should look like this: ParentAdapter : Microsoft.HyperV.PowerShell.VMNetworkAdapter Direction : Inbound Action : Allow LocalIPAddress : ANY RemoteIPAddress : 10.0.0.2 (this might be different for you) LocalPort : 68-68 RemotePort : ANY Protocol : udp Weight : 65480 Stateful : True IdleSessionTimeout : 0 IsolationID : 0 ToRemove : False All the security group rules the Hyper-V Neutron Agent are received from Neutron. This DHCP rule should be amoung them as well by default. If it is not, well, there the issue lies elsewhere, in neutron. Worst case scenario, you can just add the rule: neutron security-group-rule-create --direction ingress --ethertype IPv4 --protocol udp --port-range-min 68 --port-range-max 68 --remote-ip-prefix 10.0.0.2 <sg_id> Introducing that rule rule should allow DHCP traffic. If that still doesn't solve the issue, the problem might not be the security groups. You could try restarting the neutron-hyperv-agent with enable_security_group=false in the neutron_hyperv_agent.conf file and check if the instances are able to receive an IP. I assume you've checked the troubleshooting section of the page I've linked last time, but just to make sure.. can you check if DHCP is enabled in the subnet the instance was created in? neutron subnet-show <subnet_id> Then, considering that you went with the 3 NIC Controller and 2 NIC compute node like this: Controller: eth0 - mangement eth1 - vm-data eth2 - external Compute Node: "eth0" - management "eth1" -> vSwitch - vm-data Can you confirm that the Controller eth1 and the Compute Node vSwitch (eth1) are in the same network? Also, Controller eth1, is it promiscuous mode? At this point, we will have to get our hands dirty and do some networking troubleshooting. :) On the Hyper-V Node, run: # $INSTANCE_NAME will be instance-xxxxxxxx Get-VM -VMName $INSTANCE_NAME | Get-VMConsole In the VM Console, login, and: ifconfig # no assinged IP? then assign it manually (value from OpenStack Controller): sudo ifconfig eth0 $ASSIGNED_IP netmask $NETWORK_NETMASK up ping $DHCP_IP # let it run. On the Controller, run: # both ICMP echo request and ICMP echo reply must be visible, for all commands. sudo tcpdump -vv -eni eth1 icmp sudo tcpdump -vv -eni br-int icmp #get the tap device name sudo ip netns exec qdhcp-$NET_ID ifconfig sudo ip netns exec qdhcp-$NET_ID tcpdump -vv -ni $TAP_NAME If on the first tcpdump you do not see any ping echo request, the traffic is not getting to the Controller. If you see a ping echo request but no ping echo reply, it means that the traffic gets to the Controller, but there is reply sent back. The next commands should reveal where the reply traffic stops. The last tcpdump is the DHCP network namespace. Ideally, there will be both the request and reply messages. Hope this helps find the issue! Best regards, Claudiu Belu ________________________________ From: Lily.Sing [[email protected]] Sent: Thursday, June 25, 2015 8:02 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Alessandro and Claudiu, Thank you for your quick reply. The version I am running is kilo. Yes I use networking-hyperv. And the windows version is Windows Server 2012 R2. Below are the output for the commands mentioned: >Get-VMSwitch Name SwitchType NetAdapterInterfaceDescription ---- ---------- ------------------------------ Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch External Intel(R) Ethernet Controller X540-AT2 #3 >Get-VM | Get-VMNetworkAdapter Name IsManagementOs VMName SwitchName ---- -------------- ------ ---------- f7d8c327-606b-49cd-b740-ccaef468d535 False instance-0000000a Intel(R) Ethernet Controller X540-AT2 #3 - Vir... And nova-compute.log and neutron-hyperv-agent.log are pasted as below: http://pastebin.com/idXuhs8a nova-compute.log http://pastebin.com/XNUVyGCC neutron-hyperv-agent.log My ml2_conf.ini file is pasted here: http://pastebin.com/jLykFDbh Best regards, Lily Xing Best regards, Lily Xing(邢莉莉) On Wed, Jun 24, 2015 at 11:20 PM, Claudiu Belu <[email protected]<mailto:[email protected]>> wrote: Hello, From what I can see in the trace you included, it seems to be either Kilo or master (it uses networking-hyperv, which was introduced in Kilo). Also, it might be useful to know if you are using Windows Server 2012 or Windows Server 2012 R2. Running this in Powershell should yield the OS version: [environment]::OSVersion.Version 6.2 is Windows Server 2012 6.3 is Windows Server 2012 R2 As for port binding, neutron-hyperv-agent, if it fails to bind a port, it will retry to bind it. Which means that even if you see that trace, it doesn't mean that the binding failed at all. If you run "Get-VM | Get-VMNetworkAdapter" in Powershell, you can see that ports have been bound to the vSwitch. Also, I don't know if your environment has been properly configured, but I can tell that you have set the "hyperv" Mechanism Driver on your Controller node in the /etc/neutron/plugins/ml2/ml2_conf.ini and you are using a network type compatible with Hyper-V. Otherwise, neutron would flag the port with something like "port binding failed" and neutron-hyperv-agent wouldn't attempt to bind it. A good start would be taking a look at: http://www.cloudbase.it/quantum-hyper-v-plugin/ The linked page contains all the configuration you would need plus some troubleshooting steps that can help. Also, as Alessandro said, copies of C:\OpenStack\Log\nova-compute.log and C:\OpenStack\Log\neutron_hyperv_agent.log and the neutron server's neutron.conf file can be useful. Also, setting debug=True in the Hyper-V's neutron_hyperv_agent.conf can be useful until the issue has been solved. [DEFAULT] debug = True Let us know how it goes. Best regards, Claudiu Belu ________________________________________ From: Alessandro Pilotti Sent: Wednesday, June 24, 2015 5:20 PM To: OpenStack Development Mailing List (not for usage questions) Cc: Claudiu Belu Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed ip on hyper-v compute node Hi Lily, What version are you running, Icehouse, Juno, Kilo or master? A full copy of the nova-compute.log and neutron_hyperv_agent.log on a pastebin might be helpful. Additionally, your neutron.conf from the neutron server could help along with your neutron network configuration. Finally, the output of "Get-VMSwitch" on the Hyper-V node could provide some insight on your visrtual switch configuration. This question is deployment related and at this stage not a development topic, so IMO a more suitable location to continue the discussion would be: https://ask.openstack.org Thanks, Alessandro > On 24 Jun 2015, at 10:02, Lily.Sing > <[email protected]<mailto:[email protected]>> wrote: > > Hi, > > I setup an openstack env as one controller + network node on OL7.1 and two > compute node on windows 2012 server with cloudbase hyper-v compute node > driver. Every compute node has two nics. I created a vswitch on the second > one and use it to connect to instances. Below is my neutron_hyper_agent.conf: > > [DEFAULT] > verbose=true > control_exchange=neutron > policy_file=C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\etc\policy.json > rpc_backend=neutron.openstack.common.rpc.impl_kombu > logdir=C:\OpenStack\Log\ > logfile=neutron-hyperv-agent.log > [AGENT] > polling_interval=2 > physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller X540-AT2 #3 > - Virtual Switch > enable_metrics_collection=false > [SECURITYGROUP] > firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver > enable_security_group=true > [oslo_messaging_rabbit] > rabbit_host=<rabbit_host> > rabbit_port=5672 > rabbit_userid=stackrabbit > rabbit_password=admin > > > After launching an instance, I can check from OpenStack UI that a fixed IP is > given, but when connecting the instance from hyper-v manager, no fixed ip is > bound to any port. And below is the error message I got from > neutron-hyper-agent.log: > 2015-06-23 22:59:15.954 3552 INFO hyperv.neutron.hyperv_neutron_agent > [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Provisioning network > 6d5ee4aa-19e2-404e-9523-cc501b20f081 > 2015-06-23 22:59:22.088 3552 ERROR hyperv.neutron.hyperv_neutron_agent > [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Error in agent event loop > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > Traceback (most recent call last): > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", > line 356, in daemon_loop > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > sync = self._process_network_ports(port_info) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", > line 332, in _process_network_ports > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > resync_a = self._treat_devices_added(port_info['added']) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", > line 296, in _treat_devices_added > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > device_details['admin_state_up']) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", > line 264, in _treat_vif_port > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > physical_network, segmentation_id) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", > line 195, in _port_bound > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > self._utils.connect_vnic_to_vswitch(map['vswitch_name'], port_id) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py", > line 86, in connect_vnic_to_vswitch > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > self._add_virt_resource(vm, port) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py", > line 100, in _add_virt_resource > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > self._check_job_status(ret_val, job_path) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent File > "C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utils.py", > line 137, in _check_job_status > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > raise HyperVException(msg=_('Job failed with error %d') % ret_val) > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent > HyperVException: Hyper-V Exception: Job failed with error 32775 > > Seems it's failed to add a port to a VM, but I don't know how to fix it. Any > idea? Thanks. > > Best regards, > Lily Xing > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > [email protected]?subject:unsubscribe<http://[email protected]?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe<http://[email protected]?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
