Some considerations ... I don't know if nova remove rules when an instance is terminated. But it seems that some rules are missing. For example, we can se a lot of address in the NAT table, from .4 to .9 ... But, in the FILTER table, we can see rules just to hosts .7 and .18.
Chain nova-compute-local (1 references) target prot opt source destination nova-compute-inst-996 all -- 0.0.0.0/0 10.0.1.7 nova-compute-inst-1011 all -- 0.0.0.0/0 10.0.1.1 The packets are going but not returning. Another thing... the names that VMs query to DNS are wrong. See in tcpdump output: 10.0.1.4.52463 > 10.0.1.1.domain: 28545+ A? google.com.novalocal. (38) The normal would be google.com. and not google.com.novalocal. This is related to domain novalocal search novalocal entries in resolv.conf. I've had the same problem. If I change the resolv.conf it returns to this default after some time. Now, WE need help... lol ! :) On Oct 7, 2011, at 4:43 PM, Sharif Islam wrote: > > Thanks Jorge. > > On 10/07/2011 02:30 PM, Jorge Luiz Correa wrote: >> It seems that configs are OK. > > > Yes, that's what baffling me. I am pretty sure it was working before. I > applied some redhat update and rebooted the cluster couple weeks ago. > > > >> >> If you use dig from the controller, could resolv names? I'm asking >> because can be case that packets arrive from VMs to controller but >> couldn't go to Internet. > >> From the controller, it is fine: > > # dig @10.0.1.1 google.com > > ; <<>> DiG 9.7.3-P1-RedHat-9.7.3-2.el6_1.P1.1 <<>> @10.0.1.1 google.com > ; (1 server found) > ;; global options: +cmd > ;; Got answer: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18002 > ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 4, ADDITIONAL: 4 > > ;; QUESTION SECTION: > ;google.com. IN A > > ;; ANSWER SECTION: > google.com. 263 IN A 72.14.204.99 > google.com. 263 IN A 72.14.204.103 > google.com. 263 IN A 72.14.204.104 > google.com. 263 IN A 72.14.204.105 > google.com. 263 IN A 72.14.204.147 > > ;; AUTHORITY SECTION: > google.com. 84809 IN NS ns2.google.com. > google.com. 84809 IN NS ns3.google.com. > google.com. 84809 IN NS ns4.google.com. > google.com. 84809 IN NS ns1.google.com. > > ;; ADDITIONAL SECTION: > ns1.google.com. 160584 IN A 216.239.32.10 > ns2.google.com. 159501 IN A 216.239.34.10 > ns3.google.com. 159500 IN A 216.239.36.10 > ns4.google.com. 159497 IN A 216.239.38.10 > > ;; Query time: 1 msec > ;; SERVER: 10.0.1.1#53(10.0.1.1) > ;; WHEN: Fri Oct 7 14:44:10 2011 > ;; MSG SIZE rcvd: 244 > > > > > > >> >> Another thing you can check. Although the resolv.conf of VMs are set >> with 10.0.1.1, there are a lot of iptables rules. I was using Cactus and >> I noticed that. If you type nova-manage network list you will see the >> networks and you can see a DNS collumn. The default was 8.8.4.4 but when >> I started instances this values changed to 10.0.2.1 or something like >> that! My concern is about what address nova uses to create rules!! Maybe >> all services are OK but a wrong iptables rule is dropping packets! >> >> iptables -n -L > > http://paste.openstack.org/show/2646/ > > > >> iptables -n -L -t nat > > http://paste.openstack.org/show/2647/ > >> >> Check if you have some rule permitting udp 53 to be forward/accepted >> (ie, not dropped). > > Looks ok to me: > > ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:53 > ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 > > > >> >> As a debug option, you can run tcpdump on the controller interface and >> see what are happening with the packets. >> >> tcpdump -n -i <interface> port 53 > > # tcpdump -n -i eth0 port 53 > tcpdump: WARNING: eth0: no IPv4 address assigned > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes > 15:42:41.459072 IP 10.0.1.4.46200 > 10.0.1.1.domain: 46894+ A? > google.com. (28) > 15:42:41.459423 IP 10.0.1.4.49593 > 10.0.1.1.domain: 46894+ A? > google.com. (28) > 15:42:41.459748 IP 10.0.1.4.32779 > 10.0.1.1.domain: 28545+ A? > google.com.novalocal. (38) > 15:42:41.460029 IP 10.0.1.4.52463 > 10.0.1.1.domain: 28545+ A? > google.com.novalocal. (38) > > > This is when I pinged google.com from the vm. So iptables blocking > something? > > --sharif > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : [email protected] > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
