I have a testing cloudstack cluster, I destroyed it and rebuilding upgraded serveral times, each time I ran into the same problem, unable to access outside world from instances behind virtual router.
here is iptables before upgrade, Cloudstack 4.2 # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014 *mangle :PREROUTING ACCEPT [2317:1282555] :INPUT ACCEPT [409:147015] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [189:29312] :POSTROUTING ACCEPT [189:29312] :FIREWALL_176.23.23.192 - [0:0] :VPN_176.23.23.192 - [0:0] -A PREROUTING -d 176.23.23.192/32 -j VPN_176.23.23.192 -A PREROUTING -d 176.23.23.192/32 -j FIREWALL_176.23.23.192 -A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill -A FIREWALL_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FIREWALL_176.23.23.192 -j DROP -A VPN_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT -A VPN_176.23.23.192 -j RETURN COMMIT # Completed on Fri Apr 11 19:53:57 2014 # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014 *filter :INPUT DROP [204:117504] :FORWARD DROP [0:0] :OUTPUT ACCEPT [150:22404] :FW_OUTBOUND - [0:0] :NETWORK_STATS - [0:0] -A INPUT -j NETWORK_STATS -A INPUT -d 224.0.0.18/32 -j ACCEPT -A INPUT -d 225.0.0.50/32 -j ACCEPT -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -i eth0 -p udp -m udp --dport 67 -j ACCEPT -A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT -A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT -A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 3922 -j ACCEPT -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT -A INPUT -s 10.1.1.0/24 -i eth0 -p tcp -m state --state NEW -m tcp --dport 8080 -j ACCEPT -A FORWARD -j NETWORK_STATS -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth0 -o eth0 -m state --state NEW -j ACCEPT -A FORWARD -i eth0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth0 -o eth2 -j FW_OUTBOUND -A OUTPUT -j NETWORK_STATS -A FW_OUTBOUND -m state --state RELATED,ESTABLISHED -j ACCEPT -A NETWORK_STATS -i eth0 -o eth2 -A NETWORK_STATS -i eth2 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp COMMIT # Completed on Fri Apr 11 19:53:57 2014 # Generated by iptables-save v1.4.14 on Fri Apr 11 19:53:57 2014 *nat :PREROUTING ACCEPT [2078:1204416] :INPUT ACCEPT [10:964] :OUTPUT ACCEPT [1:338] :POSTROUTING ACCEPT [1:338] -A POSTROUTING -o eth2 -j SNAT --to-source 176.23.23.192 COMMIT # Completed on Fri Apr 11 19:53:57 2014 after upgrading to Cloudstack 4.3 :POSTROUTING ACCEPT [211:25828] :FIREWALL_176.23.23.192 - [0:0] :VPN_176.23.23.192 - [0:0] -A PREROUTING -d 176.23.23.192/32 -j VPN_176.23.23.192 -A PREROUTING -d 176.23.23.192/32 -j FIREWALL_176.23.23.192 -A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill -A FIREWALL_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FIREWALL_176.23.23.192 -j DROP -A VPN_176.23.23.192 -m state --state RELATED,ESTABLISHED -j ACCEPT -A VPN_176.23.23.192 -j RETURN COMMIT # Completed on Fri Apr 11 20:49:46 2014 # Generated by iptables-save v1.4.14 on Fri Apr 11 20:49:46 2014 *filter :INPUT DROP [68:32168] :FORWARD DROP [0:0] :OUTPUT ACCEPT [81:12516] :FW_EGRESS_RULES - [0:0] :FW_OUTBOUND - [0:0] :NETWORK_STATS - [0:0] -A INPUT -j NETWORK_STATS -A INPUT -d 224.0.0.18/32 -j ACCEPT -A INPUT -d 225.0.0.50/32 -j ACCEPT -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -i eth2 -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -i eth0 -p udp -m udp --dport 67 -j ACCEPT -A INPUT -i eth0 -p udp -m udp --dport 53 -j ACCEPT -A INPUT -i eth0 -p tcp -m tcp --dport 53 -j ACCEPT -A INPUT -i eth1 -p tcp -m state --state NEW -m tcp --dport 3922 -j ACCEPT -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT -j ACCEPT -A FORWARD -j NETWORK_STATS -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth2 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth0 -o eth0 -m state --state NEW -j ACCEPT -A FORWARD -i eth0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth0 -o eth2 -j FW_OUTBOUND -A FORWARD -i eth3 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT -A FORWARD -i eth0 -o eth3 -j FW_OUTBOUND -A OUTPUT -j NETWORK_STATS -A FW_EGRESS_RULES -p udp -m udp --dport 1:65535 -j ACCEPT -A FW_EGRESS_RULES -p icmp -m icmp --icmp-type 8/0 -j ACCEPT -A FW_EGRESS_RULES -p tcp -m tcp --dport 1:65535 -j ACCEPT -A FW_OUTBOUND -m state --state RELATED,ESTABLISHED -j ACCEPT -A FW_OUTBOUND -j FW_EGRESS_RULES -A NETWORK_STATS -i eth0 -o eth2 -A NETWORK_STATS -i eth2 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth2 -p tcp -A NETWORK_STATS -i eth2 ! -o eth0 -p tcp -A NETWORK_STATS -i eth0 -o eth3 -A NETWORK_STATS -i eth3 -o eth0 -A NETWORK_STATS ! -i eth0 -o eth3 -p tcp -A NETWORK_STATS -i eth3 ! -o eth0 -p tcp COMMIT # Completed on Fri Apr 11 20:49:46 2014 # Generated by iptables-save v1.4.14 on Fri Apr 11 20:49:46 2014 *nat :PREROUTING ACCEPT [685:441108] :INPUT ACCEPT [7:492] :OUTPUT ACCEPT [2:136] :POSTROUTING ACCEPT [3:412] -A POSTROUTING -o eth3 -j SNAT --to-source 199.96.39.192 COMMIT # Completed on Fri Apr 11 20:49:46 2014 Please help!! is a KVM cluster, I have two bridges, cloudbr0 and cloudbr1. bridge cloudbr0 is default public/guest and cloudbr1 is management. I am able to ping from virtual router, I personally think source NAT is the issue, Thanks, On Fri, Apr 11, 2014 at 8:14 AM, motty cruz <motty.c...@gmail.com> wrote: > looking at the VR it has three interfaces NIC1 10.1.1.1 NIC2 (type > control) Local Link NIC3(default NIC) public IP > > when I log in to the router and ran the following command: > ifconfig -a > I see 4 interfaces eth0 10.1.1.1 eth1 (local link)169.254.3.81 eth2 public > IP eth3 (same public IP as eth2). > > I believe this is my problem, but don't know how to fix it, > any ideas? > > > On Fri, Apr 11, 2014 at 8:04 AM, motty cruz <motty.c...@gmail.com> wrote: > >> Thanks Suresh, >> >> I tried your suggestion and I'm not able to access outside the VR router. >> I am stumped! >> >> please help! >> >> >> >> >> On Thu, Apr 10, 2014 at 7:21 AM, Suresh Sadhu <suresh.sa...@citrix.com>wrote: >> >>> Ok then work around is manually append rule to cloudbr1 . >>> >>> Take the backup of iptables rules >>> Manfully detach the eth interface from cloudbr0 and attach to cloudbr1 >>> Apply the all exiting firewall rules manually on the interface gain >>> >>> >>> After that your VMs will access the public network. >>> >>> >>> Regards >>> Sadhu >>> >>> >>> >>> -----Original Message----- >>> From: motty cruz [mailto:motty.c...@gmail.com] >>> Sent: 10 April 2014 19:40 >>> To: users@cloudstack.apache.org >>> Subject: Re: Cloudstack 4.3 instances can't access outside world >>> >>> yes, I'm am using traffic labels, everything was working fine before the >>> upgrade to 4.3. did not change anything on the cloudbr0 or cloudbr1. >>> >>> >>> On Thu, Apr 10, 2014 at 7:05 AM, Suresh Sadhu <suresh.sa...@citrix.com >>> >wrote: >>> >>> > Did you used traffic name labels? >>> > >>> > In 4.3 traffic labels are not considering ,by default its attaching to >>> > default traffic labels(eg:in KVM its cloudbr0 ...due to this unable >>> > to access public network i.r before upgrade if ieth2 attached cloudbr1 >>> > and after upgrade its attached to cloudbr0).maybe you are hitting this >>> issue. >>> > >>> > Regards >>> > sadhu >>> > >>> > >>> > -----Original Message----- >>> > From: motty cruz [mailto:motty.c...@gmail.com] >>> > Sent: 10 April 2014 19:28 >>> > To: users@cloudstack.apache.org >>> > Subject: Re: Cloudstack 4.3 instances can't access outside world >>> > >>> > yes I can ping VR, also after the upgrade VR has four insterfaces, >>> > eth0 subnet for Instances, eth1, eth2 for public IP and eth3 for >>> public IP. >>> > >>> > >>> > On Wed, Apr 9, 2014 at 10:35 PM, Erik Weber <terbol...@gmail.com> >>> wrote: >>> > >>> > > Can you ping the VR? Log on to the VR, and get the iptables rules. >>> > > How do they look? >>> > > >>> > > Erik Weber >>> > > 10. apr. 2014 00:21 skrev "motty cruz" <motty.c...@gmail.com> >>> følgende: >>> > > >>> > > > I did add egress rules, reboot network but no sucess, so I removed >>> > > > that rules and nothing. >>> > > > >>> > > > I am lost. >>> > > > >>> > > > >>> > > > On Wed, Apr 9, 2014 at 9:08 AM, Erik Weber <terbol...@gmail.com> >>> > wrote: >>> > > > >>> > > > > Did you remove the egress rule again? If not, try that. >>> > > > > >>> > > > > Erik >>> > > > > 9. apr. 2014 15:49 skrev "motty cruz" <motty.c...@gmail.com> >>> > følgende: >>> > > > > >>> > > > > > yes I try adding the rule, restart network and router but no >>> > success! >>> > > > > > >>> > > > > > >>> > > > > > On Tue, Apr 8, 2014 at 11:16 PM, Erik Weber >>> > > > > > <terbol...@gmail.com> >>> > > > wrote: >>> > > > > > >>> > > > > > > Try adding an egress rule, and removing it again. >>> > > > > > > >>> > > > > > > We experience the same, but has so far believed it was >>> > > > > > > because we >>> > > > > changed >>> > > > > > > the default rule from deny to allow after accounts were >>> made.. >>> > > > > > > >>> > > > > > > >>> > > > > > > On Tue, Apr 8, 2014 at 11:14 PM, motty cruz >>> > > > > > > <motty.c...@gmail.com> >>> > > > > > wrote: >>> > > > > > > >>> > > > > > > > I have two isolated network both virtual routers can ping >>> > > anywhere, >>> > > > > but >>> > > > > > > the >>> > > > > > > > Instances behind the virtual router can't ping or access >>> > > > > > > > the >>> > > > > internet. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > On Tue, Apr 8, 2014 at 10:38 AM, motty cruz < >>> > > motty.c...@gmail.com> >>> > > > > > > wrote: >>> > > > > > > > >>> > > > > > > > > Hello, >>> > > > > > > > > I'm having issues with VMs unable to access outside >>> world. >>> > > > > > > > > I >>> > > can >>> > > > > ping >>> > > > > > > > > gateway, also when I log in to virtual router, I am able >>> > > > > > > > > to >>> > > ping >>> > > > > > > > > google.com or anywhere. >>> > > > > > > > > in the Egress rules I am allowing all. reboot network >>> > > > > > > > > and >>> > > virtual >>> > > > > > > router >>> > > > > > > > > does not help. >>> > > > > > > > > >>> > > > > > > > > VMs were able to access outside before upgrading from >>> > > > > > > > > 4.2 to >>> > > 4.3. >>> > > > > > > > > >>> > > > > > > > > any ideas? >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> >> >> >
