Hi Wei, Any documentation on how to implement your suggestion -- adding a new NIC for new IP range, while leaving the shared network untagged? Does this mean that the different IP subnets can still share the same VLAN and broadcast domain?
Looking forward to your reply, thank you. Cheers. -ip- On Tue, Nov 8, 2016 at 9:52 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote: > Hi, > > I do not know if it works. however, I would prefer to add a new nic for new > ip range, like we do in vpc vrs, this need more work. > > The shared networks can be untagged. > > -Wei > > 2016-11-08 9:58 GMT+01:00 Cloud List <cloud-l...@sg.or.id>: > > > Hi Wei, > > > > Adding new subnet on a a new shared network, meaning a new VLAN? But that > > means the IPs will not be interchange-able between networks? > > > > I have managed to find the workaround to the problem. On the VR, I only > see > > the IP address of the first subnet on the eth0 interface. The remaining > > subnets are not there. This was not the case on ACS 4.2 where each of the > > subnet's IP has "representation" on the VR's /etc/network/interface as > > eth0's subinterfaces (eth0:0, eth0:1, etc). > > > > This is what I did: > > > > - Add the rest of the multiple subnets as eth0 subinterfaces on > > /etc/network/interfaces: > > > > iface eth0:0 inet static > > address X.X.203.2 > > netmask 255.255.255.0 > > > > iface eth0:1 inet static > > address X.X.152.2 > > netmask 255.255.255.0 > > > > iface eth0:2 inet static > > address X.X.153.2 > > netmask 255.255.255.0 > > > > - Bring up the sub-interfaces: > > > > ifup eth0:0 > > ifup eth0:1 > > ifup eth0:2 > > > > - Add below lines on /etc/dnsmasq.conf: > > > > dhcp-range=X.X.203.1,static > > dhcp-range=X.X.152.1,static > > dhcp-range=X.X.153.1,static > > > > - Add below files on /etc/dnsmasq.d folder: > > > > cloud203.conf > > > > dhcp-range=interface:eth0:0,set:interface-eth0:0,X.X.203.2,static > > dhcp-option=tag:interface-eth0:0,15,cs1cloud.internal > > dhcp-option=tag:interface-eth0:0,6,8.8.8.8,8.8.4.4 > > dhcp-option=tag:interface-eth0:0,3,X,X.203.1 > > dhcp-option=tag:interface-eth0:0,1,255.255.255.0 > > > > cloud152.conf, cloud153.conf etc > > > > - Restart dnsmasq: > > > > service dnsmasq restart > > > > Do you think the above workaround will work and there will not be any > side > > effect? We understand that it means we have to re-apply the configuration > > again in the event the VR needs to be re-created. > > > > Looking forward to your reply, thank you. > > > > Cheers. > > > > -ip- > > > > > > > > On Tue, Nov 8, 2016 at 4:17 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote: > > > > > Hi, > > > > > > I think it is related to multiple ip ranges in a shared network. > > > If I change the ip in /etc/dhcphosts.txt to another ip in other range, > I > > > got the same error log in dnsmasq.log > > > > > > We do not use multiple vlan or ip ranges in a shared network. I believe > > few > > > of us have the same use case. > > > If possible, it would be better to add the new ip range as a new shared > > > network. > > > > > > -Wei > > > > > > > > > 2016-11-07 21:35 GMT+01:00 Cloud List <cloud-l...@sg.or.id>: > > > > > > > Hi Chiradeep and Wei, thanks for your reply. > > > > > > > > Wei, here's the result you requested: > > > > > > > > root@r-4155-VM:/var/log2# cat /etc/dnsmasq.conf |grep -v "^#" |grep > -v > > > > "^$" > > > > domain-needed > > > > bogus-priv > > > > resolv-file=/etc/dnsmasq-resolv.conf > > > > local=/cs1cloud.internal/ > > > > except-interface=eth1 > > > > except-interface=eth2 > > > > except-interface=lo > > > > no-dhcp-interface=eth1 > > > > no-dhcp-interface=eth2 > > > > expand-hosts > > > > domain=cs1cloud.internal > > > > domain=cs1cloud.internal > > > > domain=cs1cloud.internal > > > > dhcp-range=X.Y.202.1,static > > > > dhcp-hostsfile=/etc/dhcphosts.txt > > > > dhcp-ignore=tag:!known > > > > dhcp-option=15,"cs1cloud.internal" > > > > dhcp-option=vendor:MSFT,2,1i > > > > dhcp-boot=pxelinux.0 > > > > enable-tftp > > > > tftp-root=/opt/tftpboot > > > > dhcp-lease-max=2100 > > > > domain=cs1cloud.internal > > > > log-dhcp > > > > log-facility=/var/log2/dnsmasq.log > > > > conf-dir=/etc/dnsmasq.d > > > > dhcp-optsfile=/etc/dhcpopts.txt > > > > dhcp-option=option:router,X.Y.202.1 > > > > dhcp-option=6,X.Y.202.2,8.8.8.8,8.8.4.4 > > > > dhcp-client-update > > > > > > > > We actually have 3 class-C subnets assigned to the network. > > > > > > > > X.Y.202.0/24 > > > > X.Y.203.0/24 > > > > Z.A.107.0/24 > > > > > > > > After enabling log-dhcp as per Chiradeep Vittal, I can see that the > > > > available DHCP subnet is only the first subnet. > > > > > > > > Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP > subnet: > > > > X.Y.202.2/255.255.255.0 > > > > Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP > subnet: > > > > X.Y.202.1/255.255.255.0 > > > > Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 client provides name: > > > > Debian-81-64b > > > > Nov 7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 DHCPDISCOVER(eth0) > > > > 06:c5:38:01:13:40 ignored > > > > Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP > subnet: > > > > X.Y.202.2/255.255.255.0 > > > > Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP > subnet: > > > > X.Y.202.1/255.255.255.0 > > > > Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 client provides name: > > > > WIN-H4INMOBBRJA > > > > Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 vendor class: MSFT > 5.0 > > > > Nov 7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 DHCPDISCOVER(eth0) > > > > 06:31:ac:01:13:f0 ignored > > > > > > > > Coincidentally, all affected VMs are coming from the second subnet: > > > > X.Y.203.0/24, while the third subnet is rarely used. > > > > > > > > Do we need to specifically include the second and third subnets into > > the > > > > dnsmasq.conf file? I tried adding below line: > > > > > > > > dhcp-range=X.Y.203.1,static > > > > > > > > so it will become: > > > > > > > > dhcp-range=X.Y.202.1,static > > > > dhcp-range=X.Y.203.1,static > > > > > > > > but it doesn't seem to work. > > > > > > > > Any advice is highly appreciated. > > > > > > > > Thank you. > > > > > > > > On Tue, Nov 8, 2016 at 4:19 AM, Wei ZHOU <ustcweiz...@gmail.com> > > wrote: > > > > > > > > > can you paste the result of following command? > > > > > > > > > > cat /etc/dnsmasq.conf |grep -v "^#" |grep -v "^$" > > > > > > > > > > -Wei > > > > > > > > > > > > > > > 2016-11-07 20:27 GMT+01:00 Cloud List <cloud-l...@sg.or.id>: > > > > > > > > > > > Hi Wei, > > > > > > > > > > > > In addition, > > > > > > > > > > > > The VR is serving a shared not isolated network, meaning the IP > it > > > > serves > > > > > > is 'guest' not 'public' IP. Will that make a difference on the > > > iptables > > > > > > command we need to execute? > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > On Tue, Nov 8, 2016 at 3:19 AM, Cloud List <cloud-l...@sg.or.id> > > > > wrote: > > > > > > > > > > > > > Hi Wei and Ozhan, > > > > > > > > > > > > > > Thanks for your reply. > > > > > > > > > > > > > > The problem doesn't affect only Debian-based guest VMs, but > also > > > > > affected > > > > > > > some Windows and Ubuntu-based VMs as well. I have executed the > > > > command > > > > > on > > > > > > > the VR and reset the NIC of the guest VM, but unfortunately the > > > issue > > > > > > still > > > > > > > persists. > > > > > > > > > > > > > > iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j > > > > CHECKSUM > > > > > > > --checksum-fill > > > > > > > > > > > > > > After issuing the above command on VR and reset the NIC on > guest > > vm > > > > > > > (ifdown eth0, ifup eth0): > > > > > > > > > > > > > > On VR's /var/log/dnsmasq.log: > > > > > > > > > > > > > > Nov 7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > 06:b1:22:01:13:17 > > > > > > > ignored > > > > > > > Nov 7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > 06:b1:22:01:13:17 > > > > > > > ignored > > > > > > > Nov 7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > 06:b1:22:01:13:17 > > > > > > > ignored > > > > > > > Nov 7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > 06:b1:22:01:13:17 > > > > > > > ignored > > > > > > > > > > > > > > On the guest VM: > > > > > > > > > > > > > > root@vm:~# ifup eth0 > > > > > > > Internet Systems Consortium DHCP Client 4.2.2 > > > > > > > Copyright 2004-2011 Internet Systems Consortium. > > > > > > > All rights reserved. > > > > > > > For info, please visit https://www.isc.org/software/dhcp > > > > > > > > > > > > > > Listening on LPF/eth0/06:b1:22:01:13:17 > > > > > > > Sending on LPF/eth0/06:b1:22:01:13:17 > > > > > > > Sending on Socket/fallback > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4 > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5 > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval > > > > 64DHCPDISCOVER > > > > > > on > > > > > > > eth0 to 255.255.255.255 port 67 interval 14 > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9 > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10 > > > > > > > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13 > > > > > > > No DHCOFFERS received. > > > > > > > No working leases in persistent database - sleeping. > > > > > > > > > > > > > > I also tried to execute similar hotfix as mentioned on the bug > > > > report: > > > > > > > > > > > > > > iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j > > CHECKSUM > > > > > > > --checksum-fill > > > > > > > > > > > > > > The problem still persists. Any further advice on this is > highly > > > > > > > appreciated. > > > > > > > > > > > > > > Looking forward to your reply, thank you. > > > > > > > > > > > > > > Cheers. > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU < > ustcweiz...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > >> GOOD point. > > > > > > >> > > > > > > >> @CloudList, can you try again after executing the following > > > command > > > > in > > > > > > VR > > > > > > >> ? > > > > > > >> > > > > > > >> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j > > > > CHECKSUM > > > > > > >> --checksum-fill > > > > > > >> > > > > > > >> -Wei > > > > > > >> > > > > > > >> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman < > > > > > > oruzgarkara...@gmail.com > > > > > > >> >: > > > > > > >> > > > > > > >> > Hi; > > > > > > >> > Whats your problematic vm's type is it Debian? Maybe you are > > > > > affected > > > > > > >> from > > > > > > >> > the bug CLOUDSTACK-8326? I do not know if this bug has > > effected > > > on > > > > > ACS > > > > > > >> 4.2 > > > > > > >> > release but i know that it effects release 4.8.x 4.9.x > > > > > > >> > > > > > > > >> > Thanks > > > > > > >> > Özhan > > > > > > >> > > > > > > > >> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List < > > cloud-l...@sg.or.id > > > > > > > > > > wrote: > > > > > > >> > > > > > > > >> > > Hi Wei, > > > > > > >> > > > > > > > > >> > > Thanks for your reply. > > > > > > >> > > > > > > > > >> > > I checked and I can confirm that the mac address is listed > > on > > > > > > >> > > /etc/dhcphosts.txt in VR. > > > > > > >> > > > > > > > > >> > > For example: > > > > > > >> > > > > > > > > >> > > Nov 7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > > >> 06:31:ac:01:13:YY > > > > > > >> > > ignored > > > > > > >> > > Nov 7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > > >> 06:31:ac:01:13:YY > > > > > > >> > > ignored > > > > > > >> > > Nov 7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) > > > > > > >> 06:31:ac:01:13:YY > > > > > > >> > > ignored > > > > > > >> > > > > > > > > >> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 > > > > > /etc/dhcphosts.txt > > > > > > >> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite > > > > > > >> > > > > > > > > >> > > YY - last two digits of the mac address > > > > > > >> > > X.X.X.X - ip address which is supposed to be allocated to > > the > > > VM > > > > > > >> > > > > > > > > >> > > Any advice how can I troubleshoot this further? > > > > > > >> > > > > > > > > >> > > Looking forward to your reply, thank you. > > > > > > >> > > > > > > > > >> > > Cheers. > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU < > > > ustcweiz...@gmail.com > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > > > >> > > > If the mac address is not listed in /etc/dhcphosts.txt > in > > > VR, > > > > > the > > > > > > >> > request > > > > > > >> > > > will be ignored. > > > > > > >> > > > > > > > > > >> > > > Can you give more details so we can reproduce it and fix > > it > > > ? > > > > > > >> > > > > > > > > > >> > > > -Wei > > > > > > >> > > > > > > > > > >> > > > 2016-11-07 13:44 GMT+01:00 Cloud List < > > cloud-l...@sg.or.id > > > >: > > > > > > >> > > > > > > > > > >> > > > > Hi, > > > > > > >> > > > > > > > > > > >> > > > > After our upgrade from CloudStack 4.2 to 4.8.1.1, we > > noted > > > > > that > > > > > > >> some > > > > > > >> > > (but > > > > > > >> > > > > not all) of our VMs are not able to get DHCP from the > > VR. > > > > This > > > > > > >> gives > > > > > > >> > > > > problem when the VM is restarted and cannot get up and > > > > running > > > > > > >> > because > > > > > > >> > > > > unable to get IP. > > > > > > >> > > > > > > > > > > >> > > > > I logged in to the VR and found below messages showing > > > that > > > > > the > > > > > > >> DHCP > > > > > > >> > > > server > > > > > > >> > > > > is ignoring the request from the VM. > > > > > > >> > > > > > > > > > > >> > > > > Nov 7 12:19:43 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:19:52 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:05:64:01:13:d3 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:19:53 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:20:04 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:20:11 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:05:64:01:13:d3 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:20:22 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:20:30 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:20:42 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:21:02 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > Nov 7 12:21:19 dnsmasq-dhcp[29938]: > DHCPDISCOVER(eth0) > > > > > > >> > > 06:44:da:01:13:98 > > > > > > >> > > > > ignored > > > > > > >> > > > > > > > > > > >> > > > > Any reason why the dnsmasq service ignored the request > > > from > > > > > the > > > > > > VM > > > > > > >> > and > > > > > > >> > > > how > > > > > > >> > > > > can we fix that? > > > > > > >> > > > > > > > > > > >> > > > > At the moment, the only workaround we can do is to > > > configure > > > > > the > > > > > > >> IP > > > > > > >> > > > address > > > > > > >> > > > > statically to the servelet, which is not practical. > > > > > > >> > > > > > > > > > > >> > > > > Any advice is greatly appreciated. > > > > > > >> > > > > > > > > > > >> > > > > Thank you. > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
