Matthew, Your Bridge configs look OK, but we normally see the management IP on the bridge.
Can you move your clustered file system onto a dedicated nic, allowing management IP to be placed on the bridge, this may stop the self fencing issues. It seems too much of a coincidence that the management IP is not on the bridge and it's the management traffic which is failing. Regards Geoff Higginbottom CTO / Cloud Architect D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com> | www.shapeblue.com<htp://www.shapeblue.com/> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS<x-apple-data-detectors://37> On 8 Apr 2014, at 19:56, "Matthew Midgett" <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz>> wrote: Here are my interfaces and I am sure the labels are correct. Any other ideas? This is for my management ip, I couldn't put it on the bridge as my clustered file system would keep fencing it self. cat ifcfg-eth0 DEVICE="eth0" BOOTPROTO=static HWADDR="78:E7:D1:8E:2F:AE" NM_CONTROLLED="none" ONBOOT=yes TYPE="Ethernet" IPADDR=172.16.0.11 NETMASK=255.255.0.0 GATEWAY=172.16.0.1 This is where the physicals for my bridges start cat ifcfg-eth1 DEVICE=eth1 BOOTPROTO=none HWADDR=78:E7:D1:8E:2F:B0 ONBOOT=yes USERCTL=no NM_CONTROLLED=no BRIDGE=cloudbr0 cat ifcfg-cloudbr0 DEVICE=cloudbr0 NM_CONTROLLED=no ONBOOT=yes TYPE=Bridge NAME=cloudbr0 cat ifcfg-eth2 DEVICE=eth2 BOOTPROTO=none HWADDR=78:E7:D1:8E:2F:B2 ONBOOT=yes USERCTL=no BRIDGE=cloudbr1 NM_CONTROLLED=no cat ifcfg-cloudbr1 DEVICE=cloudbr1 NM_CONTROLLED=no ONBOOT=yes TYPE=Bridge NAME=cloudbr1 cat ifcfg-eth3 DEVICE=eth3 BOOTPROTO=none HWADDR=78:E7:D1:8E:2F:B4 ONBOOT=yes USERCTL=no BRIDGE=cloudbr2 NM_CONTROLLED=no cat ifcfg-cloudbr2 DEVICE=cloudbr2 NM_CONTROLLED=no ONBOOT=yes TYPE=Bridge BOOTPROTO=none NAME=cloudbr2 The interfaces match my layout and the physical network cloudbr0 - Management cloudbr1 - guest cloudbr2 - public brctl show bridge name bridge id STP enabled interfaces cloud0 8000.fe00a9fe01e4 no vnet0 vnet3 cloudbr0 8000.78e7d18e2fb0 no eth1 vnet1 vnet4 vnet6 cloudbr1 8000.78e7d18e2fb2 no eth2 cloudbr2 8000.78e7d18e2fb4 no eth3 vnet2 vnet5 virbr0 8000.5254007e4d34 yes virbr0-nic On 04/08/2014 02:28 PM, Geoff Higginbottom wrote: I would check the traffic labels for the management network match for your zone and hosts. Regards Geoff Higginbottom CTO / Cloud Architect D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbot...@shapeblue.com> | www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/> | Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS<x-apple-data-detectors://5> On 8 Apr 2014, at 17:44, "Matthew Midgett" <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>> wrote: My problem is that the ssvm will not ping anything on the management network, It will not do dns lookups as it will not ping past the default public gateway. The ssvm is up and I can ping the public ip from a different subnet. If i remove this I can use external dns and the ssvm can ping anything public 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 I change the values as Geoff had suggested and I rebooted the management service. Then i destroyed the ssvm so it would be recreated with the new changes. Here is the route as it is now and not working. root@s-9-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 To me it doesn't look like the use external dns setting to true didn't do anything. It sill wants to tell the dns to go across the management network. That would be fine if the routes to the management network worked. I'm willing to pay someone to help me at this point. On 04/08/2014 12:10 PM, Erik Weber wrote: You don't need gateway for network in the same subnet on an interface. You never told us what your real problem is (i think), so why not start by telling us what is not working :-) As i tried to explain, 8.8.8.8 and 8.8.4.4 will be routed over the mgmt interface if it's entered as the internal dns. If that is your problem, try changing the value Geoff suggested. Erik 8. apr. 2014 17:10 skrev "Matthew Midgett" <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>> følgende: Erik that would be fine but this is wrong, this means its going to route over the public address as 0.0.0.0 points to my public gateway. 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 On 04/08/2014 11:03 AM, Erik Weber wrote: If you use the same dns servers for both internal and external it will add a route over mgmt interface. Erik Weber 8. apr. 2014 16:07 skrev "Matthew Midgett" <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>> følgende: I destroyed the VM so it would create a new route. First off the 8.8.8.8 and 8.8.4.4 should have a default gateway of 0.0.0.0 if its to use the public address to get dns. If its supposed to use the management network then the 172.16.0.0 0.0.0.0 should be 172.16.0.0 172.16.0.1 This way it doesn't route properly root@s-4-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 8.8.4.4 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 8.8.8.8 172.16.0.1 255.255.255.255 UGH 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1 172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth3 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 This way works. root@s-4-VM:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 216.249.111.1 0.0.0.0 UG 0 0 0 eth2 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0 eth1 172.16.0.0 172.16.0.1 255.255.0.0 U 0 0 0 eth3 216.249.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 Still not sure where I should put this. I know its the way that Im connecting to it but what route should it take? I'm thinking it should be 172.16.0.0 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 On 04/08/2014 01:35 AM, Geoff Higginbottom wrote: Matthew, Can you give examples of the routes you are seeing and explain why they are wrong please. Regards Geoff Higginbottom CTO / Cloud Architect D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 0540<tel: +442036030540> | M: +447968161581<tel:+447968161581> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbottom@ shapeblue.com<http://shapeblue.com><http://shapeblue.com>> | www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com><htp://www.shapeblue.com/> | Twitter:@cloudstackguru < https://twitter.com/#!/cloudstackguru> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS<x-apple-data-detectors://5> On 7 Apr 2014, at 22:24, "Matthew Midgett" <supp...@trickhosting.biz<mailto:supp...@trickhosting.biz><mailto:supp...@trickhosting.biz>< mailto:supp...@trickhosting.biz>> wrote: My system vm's are being created with the wrong routes. Manually deleting and adding them fixes the problem. Where would I go to fix this permanently? Sent from my Galaxy S(r)III Need Enterprise Grade Support for Apache CloudStack? Our CloudStack Infrastructure Support<http://shapeblue.com/ cloudstack-infrastructure-support/> offers the best 24/7 SLA for CloudStack Environments. Apache CloudStack Bootcamp training courses **NEW!** CloudStack 4.2.1 training<http://shapeblue.com/ cloudstack-training/> 28th-29th May 2014, Bangalore. Classromm<http://shapeblue. com/cloudstack-training/> 16th-20th June 2014, Region A. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> 23rd-27th June 2014, Region B. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> 15th-20th September 2014, Region A. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> 22nd-27th September 2014, Region B. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> 1st-6th December 2014, Region A. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> 8th-12th December 2014, Region B. Instructor led, On-line< http://shapeblue.com/cloudstack-training/> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark. This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
