Hi Chiradeep and Wei, thanks for your reply. Wei, here's the result you requested:
root@r-4155-VM:/var/log2# cat /etc/dnsmasq.conf |grep -v "^#" |grep -v "^$" domain-needed bogus-priv resolv-file=/etc/dnsmasq-resolv.conf local=/cs1cloud.internal/ except-interface=eth1 except-interface=eth2 except-interface=lo no-dhcp-interface=eth1 no-dhcp-interface=eth2 expand-hosts domain=cs1cloud.internal domain=cs1cloud.internal domain=cs1cloud.internal dhcp-range=X.Y.202.1,static dhcp-hostsfile=/etc/dhcphosts.txt dhcp-ignore=tag:!known dhcp-option=15,"cs1cloud.internal" dhcp-option=vendor:MSFT,2,1i dhcp-boot=pxelinux.0 enable-tftp tftp-root=/opt/tftpboot dhcp-lease-max=2100 domain=cs1cloud.internal log-dhcp log-facility=/var/log2/dnsmasq.log conf-dir=/etc/dnsmasq.d dhcp-optsfile=/etc/dhcpopts.txt dhcp-option=option:router,X.Y.202.1 dhcp-option=6,X.Y.202.2,8.8.8.8,8.8.4.4 dhcp-client-update We actually have 3 class-C subnets assigned to the network. X.Y.202.0/24 X.Y.203.0/24 Z.A.107.0/24 After enabling log-dhcp as per Chiradeep Vittal, I can see that the available DHCP subnet is only the first subnet. Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet: X.Y.202.2/255.255.255.0 Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet: X.Y.202.1/255.255.255.0 Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 client provides name: Debian-81-64b Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 DHCPDISCOVER(eth0) 06:c5:38:01:13:40 ignored Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet: X.Y.202.2/255.255.255.0 Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet: X.Y.202.1/255.255.255.0 Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 client provides name: WIN-H4INMOBBRJA Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 vendor class: MSFT 5.0 Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 DHCPDISCOVER(eth0) 06:31:ac:01:13:f0 ignored Coincidentally, all affected VMs are coming from the second subnet: X.Y.203.0/24, while the third subnet is rarely used. Do we need to specifically include the second and third subnets into the dnsmasq.conf file? I tried adding below line: dhcp-range=X.Y.203.1,static so it will become: dhcp-range=X.Y.202.1,static dhcp-range=X.Y.203.1,static but it doesn't seem to work. Any advice is highly appreciated. Thank you. On Tue, Nov 8, 2016 at 4:19 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote: > can you paste the result of following command? > > cat /etc/dnsmasq.conf |grep -v "^#" |grep -v "^$" > > -Wei > > > 2016-11-07 20:27 GMT+01:00 Cloud List <cloud-l...@sg.or.id>: > > > Hi Wei, > > > > In addition, > > > > The VR is serving a shared not isolated network, meaning the IP it serves > > is 'guest' not 'public' IP. Will that make a difference on the iptables > > command we need to execute? > > > > Looking forward to your reply, thank you. > > > > Cheers. > > > > > > On Tue, Nov 8, 2016 at 3:19 AM, Cloud List <cloud-l...@sg.or.id> wrote: > > > > > Hi Wei and Ozhan, > > > > > > Thanks for your reply. > > > > > > The problem doesn't affect only Debian-based guest VMs, but also > affected > > > some Windows and Ubuntu-based VMs as well. I have executed the command > on > > > the VR and reset the NIC of the guest VM, but unfortunately the issue > > still > > > persists. > > > > > > iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM > > > --checksum-fill > > > > > > After issuing the above command on VR and reset the NIC on guest vm > > > (ifdown eth0, ifup eth0): > > > > > > On VR's /var/log/dnsmasq.log: > > > > > > Nov 7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > 06:b1:22:01:13:17 > > > ignored > > > Nov 7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > 06:b1:22:01:13:17 > > > ignored > > > Nov 7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > 06:b1:22:01:13:17 > > > ignored > > > Nov 7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > 06:b1:22:01:13:17 > > > ignored > > > > > > On the guest VM: > > > > > > root@vm:~# ifup eth0 > > > Internet Systems Consortium DHCP Client 4.2.2 > > > Copyright 2004-2011 Internet Systems Consortium. > > > All rights reserved. > > > For info, please visit https://www.isc.org/software/dhcp > > > > > > Listening on LPF/eth0/06:b1:22:01:13:17 > > > Sending on LPF/eth0/06:b1:22:01:13:17 > > > Sending on Socket/fallback > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4 > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5 > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 64DHCPDISCOVER > > on > > > eth0 to 255.255.255.255 port 67 interval 14 > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9 > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10 > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13 > > > No DHCOFFERS received. > > > No working leases in persistent database - sleeping. > > > > > > I also tried to execute similar hotfix as mentioned on the bug report: > > > > > > iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM > > > --checksum-fill > > > > > > The problem still persists. Any further advice on this is highly > > > appreciated. > > > > > > Looking forward to your reply, thank you. > > > > > > Cheers. > > > > > > > > > On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU <ustcweiz...@gmail.com> > wrote: > > > > > >> GOOD point. > > >> > > >> @CloudList, can you try again after executing the following command in > > VR > > >> ? > > >> > > >> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM > > >> --checksum-fill > > >> > > >> -Wei > > >> > > >> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman < > > oruzgarkara...@gmail.com > > >> >: > > >> > > >> > Hi; > > >> > Whats your problematic vm's type is it Debian? Maybe you are > affected > > >> from > > >> > the bug CLOUDSTACK-8326? I do not know if this bug has effected on > ACS > > >> 4.2 > > >> > release but i know that it effects release 4.8.x 4.9.x > > >> > > > >> > Thanks > > >> > Özhan > > >> > > > >> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List <cloud-l...@sg.or.id> > > wrote: > > >> > > > >> > > Hi Wei, > > >> > > > > >> > > Thanks for your reply. > > >> > > > > >> > > I checked and I can confirm that the mac address is listed on > > >> > > /etc/dhcphosts.txt in VR. > > >> > > > > >> > > For example: > > >> > > > > >> > > Nov 7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > >> 06:31:ac:01:13:YY > > >> > > ignored > > >> > > Nov 7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > >> 06:31:ac:01:13:YY > > >> > > ignored > > >> > > Nov 7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > >> 06:31:ac:01:13:YY > > >> > > ignored > > >> > > > > >> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 > /etc/dhcphosts.txt > > >> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite > > >> > > > > >> > > YY - last two digits of the mac address > > >> > > X.X.X.X - ip address which is supposed to be allocated to the VM > > >> > > > > >> > > Any advice how can I troubleshoot this further? > > >> > > > > >> > > Looking forward to your reply, thank you. > > >> > > > > >> > > Cheers. > > >> > > > > >> > > > > >> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU <ustcweiz...@gmail.com> > > >> wrote: > > >> > > > > >> > > > If the mac address is not listed in /etc/dhcphosts.txt in VR, > the > > >> > request > > >> > > > will be ignored. > > >> > > > > > >> > > > Can you give more details so we can reproduce it and fix it ? > > >> > > > > > >> > > > -Wei > > >> > > > > > >> > > > 2016-11-07 13:44 GMT+01:00 Cloud List <cloud-l...@sg.or.id>: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > After our upgrade from CloudStack 4.2 to 4.8.1.1, we noted > that > > >> some > > >> > > (but > > >> > > > > not all) of our VMs are not able to get DHCP from the VR. This > > >> gives > > >> > > > > problem when the VM is restarted and cannot get up and running > > >> > because > > >> > > > > unable to get IP. > > >> > > > > > > >> > > > > I logged in to the VR and found below messages showing that > the > > >> DHCP > > >> > > > server > > >> > > > > is ignoring the request from the VM. > > >> > > > > > > >> > > > > Nov 7 12:19:43 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:19:52 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:05:64:01:13:d3 > > >> > > > > ignored > > >> > > > > Nov 7 12:19:53 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:20:04 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:20:11 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:05:64:01:13:d3 > > >> > > > > ignored > > >> > > > > Nov 7 12:20:22 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:20:30 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:20:42 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:21:02 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > Nov 7 12:21:19 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) > > >> > > 06:44:da:01:13:98 > > >> > > > > ignored > > >> > > > > > > >> > > > > Any reason why the dnsmasq service ignored the request from > the > > VM > > >> > and > > >> > > > how > > >> > > > > can we fix that? > > >> > > > > > > >> > > > > At the moment, the only workaround we can do is to configure > the > > >> IP > > >> > > > address > > >> > > > > statically to the servelet, which is not practical. > > >> > > > > > > >> > > > > Any advice is greatly appreciated. > > >> > > > > > > >> > > > > Thank you. > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
