Hi Claudiu, Thank you very much for the info. I'm going through the steps to have a try. I will give the feedback later if it works.
Best regards, Lily Xing(邢莉莉) On Fri, Jun 26, 2015 at 2:06 AM, Claudiu Belu <cb...@cloudbasesolutions.com> wrote: > Hello, > > ml2 conf file looks fine. > > nova logs look fine. > > neutron logs also seem fine, but this worries me a bit: > > 2015-06-24 20:45:18.556 4116 DEBUG hyperv.neutron.security_groups_driver > [req-3786da36-6b03-433d-941e-00327839603c ] Creating port 3 rules > prepare_port_filter C:\Program Files (x86)\Cloudbase > Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\security_groups_driver.py:54 > > Can you run this in powershell? > > # if you have multiple instances spawned. > Get-VMNetworkAdapterExtendedAcl -VMName instance-... | ? Action -eq Allow > > # only one instance > Get-VMNetworkAdapterExtendedAcl | ? Action -eq Allow > > Can you provide the output into a pastebin as well? > > Now, since you have security groups enabled, there should be a rule that > allow DHCP. It should look like this: > > ParentAdapter : Microsoft.HyperV.PowerShell.VMNetworkAdapter > Direction : Inbound > Action : Allow > LocalIPAddress : ANY > RemoteIPAddress : 10.0.0.2 (this might be different for you) > LocalPort : 68-68 > RemotePort : ANY > Protocol : udp > Weight : 65480 > Stateful : True > IdleSessionTimeout : 0 > IsolationID : 0 > ToRemove : False > > All the security group rules the Hyper-V Neutron Agent are received from > Neutron. This DHCP rule should be amoung them as well by default. If it is > not, well, there the issue lies elsewhere, in neutron. Worst case scenario, > you can just add the rule: > > neutron security-group-rule-create --direction ingress --ethertype IPv4 > --protocol udp --port-range-min 68 --port-range-max 68 --remote-ip-prefix > 10.0.0.2 <sg_id> > > Introducing that rule rule should allow DHCP traffic. If that still > doesn't solve the issue, the problem might not be the security groups. You > could try restarting the neutron-hyperv-agent with > enable_security_group=false in the neutron_hyperv_agent.conf file and check > if the instances are able to receive an IP. > > I assume you've checked the troubleshooting section of the page I've > linked last time, but just to make sure.. can you check if DHCP is enabled > in the subnet the instance was created in? > > neutron subnet-show <subnet_id> > > Then, considering that you went with the 3 NIC Controller and 2 NIC > compute node like this: > Controller: > eth0 - mangement > eth1 - vm-data > eth2 - external > > Compute Node: > "eth0" - management > "eth1" -> vSwitch - vm-data > > Can you confirm that the Controller eth1 and the Compute Node vSwitch > (eth1) are in the same network? Also, Controller eth1, is it promiscuous > mode? > > At this point, we will have to get our hands dirty and do some networking > troubleshooting. :) > > On the Hyper-V Node, run: > > # $INSTANCE_NAME will be instance-xxxxxxxx > Get-VM -VMName $INSTANCE_NAME | Get-VMConsole > > In the VM Console, login, and: > > ifconfig > > # no assinged IP? then assign it manually (value from OpenStack > Controller): > sudo ifconfig eth0 $ASSIGNED_IP netmask $NETWORK_NETMASK up > > ping $DHCP_IP > # let it run. > > On the Controller, run: > > # both ICMP echo request and ICMP echo reply must be visible, for all > commands. > sudo tcpdump -vv -eni eth1 icmp > sudo tcpdump -vv -eni br-int icmp > > #get the tap device name > sudo ip netns exec qdhcp-$NET_ID ifconfig > > sudo ip netns exec qdhcp-$NET_ID tcpdump -vv -ni $TAP_NAME > > If on the first tcpdump you do not see any ping echo request, the traffic > is not getting to the Controller. If you see a ping echo request but no > ping echo reply, it means that the traffic gets to the Controller, but > there is reply sent back. The next commands should reveal where the reply > traffic stops. The last tcpdump is the DHCP network namespace. Ideally, > there will be both the request and reply messages. > > > Hope this helps find the issue! > > Best regards, > Claudiu Belu > > ------------------------------ > *From:* Lily.Sing [lily.s...@gmail.com] > *Sent:* Thursday, June 25, 2015 8:02 AM > *To:* OpenStack Development Mailing List (not for usage questions) > *Subject:* Re: [openstack-dev] [neutron][hyper-v] Instance can't get > fixed ip on hyper-v compute node > > Hi Alessandro and Claudiu, > > Thank you for your quick reply. > > The version I am running is kilo. Yes I use networking-hyperv. And the > windows version is Windows Server 2012 R2. Below are the output for the > commands mentioned: > > >Get-VMSwitch > > Name SwitchType > NetAdapterInterfaceDescription > ---- ---------- > ------------------------------ > Intel(R) Ethernet Controller X540-AT2 #3 - Virtual Switch External > Intel(R) Ethernet Controller X540-AT2 #3 > > >Get-VM | Get-VMNetworkAdapter > > Name IsManagementOs VMName > SwitchName > ---- -------------- ------ > ---------- > f7d8c327-606b-49cd-b740-ccaef468d535 False instance-0000000a > Intel(R) Ethernet Controller X540-AT2 #3 - Vir... > > And nova-compute.log and neutron-hyperv-agent.log are pasted as below: > > http://pastebin.com/idXuhs8a nova-compute.log > http://pastebin.com/XNUVyGCC neutron-hyperv-agent.log > > > My ml2_conf.ini file is pasted here: http://pastebin.com/jLykFDbh > > Best regards, > Lily Xing > > > Best regards, > Lily Xing(邢莉莉) > > On Wed, Jun 24, 2015 at 11:20 PM, Claudiu Belu < > cb...@cloudbasesolutions.com> wrote: > >> Hello, >> >> From what I can see in the trace you included, it seems to be either Kilo >> or master (it uses networking-hyperv, which was introduced in Kilo). >> >> Also, it might be useful to know if you are using Windows Server 2012 or >> Windows Server 2012 R2. Running this in Powershell should yield the OS >> version: >> >> [environment]::OSVersion.Version >> >> 6.2 is Windows Server 2012 >> 6.3 is Windows Server 2012 R2 >> >> As for port binding, neutron-hyperv-agent, if it fails to bind a port, it >> will retry to bind it. Which means that even if you see that trace, it >> doesn't mean that the binding failed at all. If you run "Get-VM | >> Get-VMNetworkAdapter" in Powershell, you can see that ports have been bound >> to the vSwitch. >> >> >> Also, I don't know if your environment has been properly configured, but >> I can tell that you have set the "hyperv" Mechanism Driver on your >> Controller node in the /etc/neutron/plugins/ml2/ml2_conf.ini and you are >> using a network type compatible with Hyper-V. Otherwise, neutron would flag >> the port with something like "port binding failed" and neutron-hyperv-agent >> wouldn't attempt to bind it. >> >> A good start would be taking a look at: >> http://www.cloudbase.it/quantum-hyper-v-plugin/ >> The linked page contains all the configuration you would need plus some >> troubleshooting steps that can help. >> >> Also, as Alessandro said, copies of C:\OpenStack\Log\nova-compute.log and >> C:\OpenStack\Log\neutron_hyperv_agent.log and the neutron server's >> neutron.conf file can be useful. >> Also, setting debug=True in the Hyper-V's neutron_hyperv_agent.conf can >> be useful until the issue has been solved. >> >> [DEFAULT] >> debug = True >> >> Let us know how it goes. >> >> Best regards, >> Claudiu Belu >> ________________________________________ >> From: Alessandro Pilotti >> Sent: Wednesday, June 24, 2015 5:20 PM >> To: OpenStack Development Mailing List (not for usage questions) >> Cc: Claudiu Belu >> Subject: Re: [openstack-dev] [neutron][hyper-v] Instance can't get fixed >> ip on hyper-v compute node >> >> Hi Lily, >> >> What version are you running, Icehouse, Juno, Kilo or master? >> >> A full copy of the nova-compute.log and neutron_hyperv_agent.log on a >> pastebin >> might be helpful. Additionally, your neutron.conf from the neutron server >> could >> help along with your neutron network configuration. >> >> Finally, the output of "Get-VMSwitch" on the Hyper-V node could provide >> some >> insight on your visrtual switch configuration. >> >> This question is deployment related and at this stage not a development >> topic, >> so IMO a more suitable location to continue the discussion would be: >> https://ask.openstack.org >> >> Thanks, >> >> Alessandro >> >> >> > On 24 Jun 2015, at 10:02, Lily.Sing <lily.s...@gmail.com> wrote: >> > >> > Hi, >> > >> > I setup an openstack env as one controller + network node on OL7.1 and >> two compute node on windows 2012 server with cloudbase hyper-v compute node >> driver. Every compute node has two nics. I created a vswitch on the second >> one and use it to connect to instances. Below is my >> neutron_hyper_agent.conf: >> > >> > [DEFAULT] >> > verbose=true >> > control_exchange=neutron >> > policy_file=C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\etc\policy.json >> > rpc_backend=neutron.openstack.common.rpc.impl_kombu >> > logdir=C:\OpenStack\Log\ >> > logfile=neutron-hyperv-agent.log >> > [AGENT] >> > polling_interval=2 >> > physical_network_vswitch_mappings=*:Intel(R) Ethernet Controller >> X540-AT2 #3 - Virtual Switch >> > enable_metrics_collection=false >> > [SECURITYGROUP] >> > >> firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver >> > enable_security_group=true >> > [oslo_messaging_rabbit] >> > rabbit_host=<rabbit_host> >> > rabbit_port=5672 >> > rabbit_userid=stackrabbit >> > rabbit_password=admin >> > >> > >> > After launching an instance, I can check from OpenStack UI that a fixed >> IP is given, but when connecting the instance from hyper-v manager, no >> fixed ip is bound to any port. And below is the error message I got from >> neutron-hyper-agent.log: >> > 2015-06-23 22:59:15.954 3552 INFO hyperv.neutron.hyperv_neutron_agent >> [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Provisioning network >> 6d5ee4aa-19e2-404e-9523-cc501b20f081 >> > 2015-06-23 22:59:22.088 3552 ERROR hyperv.neutron.hyperv_neutron_agent >> [req-a0577427-04e9-481d-94f0-027cc57eb26a ] Error in agent event loop >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> Traceback (most recent call last): >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", >> line 356, in daemon_loop >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> sync = self._process_network_ports(port_info) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", >> line 332, in _process_network_ports >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> resync_a = self._treat_devices_added(port_info['added']) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", >> line 296, in _treat_devices_added >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> device_details['admin_state_up']) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", >> line 264, in _treat_vif_port >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> physical_network, segmentation_id) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\hyperv_neutron_agent.py", >> line 195, in _port_bound >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> self._utils.connect_vnic_to_vswitch(map['vswitch_name'], port_id) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py", >> line 86, in connect_vnic_to_vswitch >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> self._add_virt_resource(vm, port) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utilsv2.py", >> line 100, in _add_virt_resource >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> self._check_job_status(ret_val, job_path) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> File "C:\Program Files (x86)\Cloudbase >> Solutions\OpenStack\Nova\Python27\lib\site-packages\hyperv\neutron\utils.py", >> line 137, in _check_job_status >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> raise HyperVException(msg=_('Job failed with error %d') % ret_val) >> > 2015-06-23 22:59:22.088 3552 TRACE hyperv.neutron.hyperv_neutron_agent >> HyperVException: Hyper-V Exception: Job failed with error 32775 >> > >> > Seems it's failed to add a port to a VM, but I don't know how to fix >> it. Any idea? Thanks. >> > >> > Best regards, >> > Lily Xing >> > >> __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev