Re: [pfSense Support] Dual WAN - major problems

[Royce Mitchell III]

Title: Dual WAN - major problems

Alastair Stevens wrote: Hi again We're gradually working towards a fully redundant setup, with two pfSense boxes running with CARP and pfsync, and two WAN connections to the world. So far, with a single main WAN connection, everything is working beautifully.  We have seamless failover between the two boxes, everything is syncing nicely, and all is well. But when we try to bring up the secondary WAN connections, with load balancing, everything goes to pieces.  Of course, what we really want is direct WAN failover, but I know this isn't possible yet, so we're having a crack at load balancing for now. I've read the few tutorials that cover multi-WAN setups, and they seem to give widely differing views on setting up the outbound NAT and firewall side of things.  So far, I've achieved (briefly) some sort of functioning dual WAN, and traffic flows through both WAN interfaces, but it seems very unstable.  It also seems to break the routing, such that some of the internal LAN subnets get cutoff from each other.  This is no doubt a firewall problem, but I've tried everything and can't seem to fix it.  The traffic graph also shows *massive* symmetrical traffic on the WAN interfaces on both boxes (eg 30Mbps in both directions, all the time) - what's going on there? What I'd really value is a document/tutorial on a similar setup - ie 2 pfSense boxes with CARP/failover and 2 WANs in load-balanced mode.  Has anyone out there achieved such a setup? Regards Alastair I have a load balanced, dual wan, carp setup with an ipsec tunnel running off of wan2. Here is what I've had to do to get it to work. If there's enough interest I'll filter my config file and share it. First of all, you must setup two advanced outbound nat rules: 1st nat rule:     if: WAN     source: 192.168.0.0/24 ( lan subnet )     source port: *     dest: *     dest port: *     nat addr: wan carp ip     nat port: *     static port: no 2nd nat rule:     if: WAN2 ( OPT1 or whatever you relabel it to, I recommend you rename it to wan2 or something )     source: 192.168.0.0/24 ( lan subnet again - same as first rule )     source port: *     dest: *     dest port: *     nat addr: wan2 carp ip     nat port: *     static port: no It doesn't matter what order these rules are in. You need to go to static routes and create a static route to each of your wans' dns servers if you want DNS forwarder to work: iface: WAN network: wan1 dns ip/32 gateway: wan1 gateway iface: WAN2 network: wan2 dns ip/32 gateway: wan2 gateway You need to setup your LAN firewall rules. You will need a few rules to override the load balancing rule. I'm pretty sure I don't need all of these, but this is what I have: Proto Source          Port Destination    Port        Gateway     Description *     LAN net         *    (wan1 pub ips) *           (wan1 gw)   don't LB WAN1 Subnet traffic *     LAN net         *    (wan2 pub ips) *           (wan2 gw)   don't LB WAN2 Subnet traffic UDP   LAN net         *    LAN net        53(DNS)     *           don't LB local DNS traffic ICMP  LAN net         *    LAN net        *           *           don't LB local icmp traffic *     LAN net         *    10.31.0.0/16   *           *           don't LB my ipsec traffic TCP   LAN net         *    *              25(SMTP)    (wan1 gw)   don't LB SMTP *     192.168.0.96/31 *    ! LAN net      *           (wan2 gw)   don't LB this user TCP   LAN net         *    *              443(HTTPS)  *           don't LB HTTPS traffic *     LAN net         *    *              *           OutboundLB  ** DEFAULT RULE **