Re: Multihoming with carp possible? and ipsec failover?
Em 17-05-2014 16:26, Florenz Kley escreveu: On May 13, 2014, at 22:13, Giancarlo Razzolini grazzol...@gmail.com wrote: go there, this e-mail would be too big. If you want I can elaborate more. yes please! do elaborate a bit! fl Well, I always wanted to write an article and post it on a blog. But never really stopped everything and wrote it. I'll try to condense things on this e-mail, but I'll also try to write a lengthier article someday. First of all, you need to enable ip forwarding and, if you want your firewall machine itself, to have multiple gateways in it's routing table, you'll need to enable multipath routing. Both these are on sysctl.conf. The multipath part is really only needed if you'll run other services on your firewall itself. Such as a dns daemon, a proxy, etc. The firewall must be able to reach the internet at all times, or you'll have problems in this case. Now, you need to determine how you will be able to tell if a determined ISP link is down. This part is the one that gets more different ways to accomplish. Somebody always will tell you to simply ping an external IP address and get over with it. I really don't like it and use it only as last resort. Anyway, one nice way to determine if a link is down, is to use snmp. You can enable snmp on your ISP router/modem/whatever, do an snmpwalk on it, determine which interfaces it show. Then you trigger a disconnect and see which one of them are down. These will be the ones you'll check. You can write a simple script that does an snmpget on the interface oid and check if the state is up or down. If your router do not have snmp, but does have another form of remote administration, like a web interface or telnet, you can either write a script that connects on it via http and go to the page where it prints the internet connection status and check if it's connected. Or you can write a expect script to connect to it via telnet and type the commands that will show if the connection is up, or not. The external ip pinging part is the easiest to implement, but also easily to get wrong. You need to ping more than one IP address and only if all of them fail to reply, them you can (hopefully) consider the link down. I hope you got the idea. Almost forgot to mention, you need to script these to simply return 0 or 1 to the shell that runs it. This is the way that ifstated will be able to tell if the link is up or down. As you can see, there are several ways to accomplish this. The next part is to write the ifstated configuration. This is not as hard as it seems. It's a state machine. It is a: do this, if this fails, do that, but if it gets back, return to the previous state. Or do something entirely different. You have at least n² states. n being the number of internet links you have. So, if you have 2 links, you have at least 4 states. If you have 3, 6 states, and so on. The reason is that you need an state where all the links are on (the state where we hope it will be most of the time) and one where all the links are off (zombie apocalypse anyone?). Remember those scripts you wrote that monitor your links? You'll configure them to run in your ifstated every x seconds. The x here you'll define based on what you want to accomplish. Mine run every 20 seconds. So basically you'll have some lines in the beginning of your ifstated.conf that are something like this: link_isp = '( path to script every 20 )' The first state you define on ifstated will be the default state, unless you override with the default_state option. So, the first state will most likely be the state where all the links are up. You can run things when you enter a state. I use this to update dynamic dns records, to mail me when a link is down, reload/restart daemons that are bound to a determined isp connection, etc. But the most important thing to do is to change your pf rules when you change states. To do this, there are lots of ways too. I use anchors. So, I have n²-1 anchor files. n being the number of links you have. Because when all of them are down, there is no point in changing your rules. You could point your users to a internal web page telling them they have no internet, and, therefore, no facebook for them. But this is out of the scope. Anyway, I use ifstated to load anchors that change the pf rules to use the isp's that are working. So how you point your users to different gateways? You use the route-to option. This will point any machine using your firewall to the gateways you choose. Since this will be up entirely to you, I'll not go into details here. Just be sure to not confuse traffic that the machines behind your firewall generate, from traffic that your firewall itself generate. These need to be dealt differently, in the case of a link change. As I mentioned before, if you only have pf on the firewall, you can just use anchors and you are ok. If not, you'll need to change the routes on the firewall itself, with multipath. In my case, I
Re: Multihoming with carp possible? and ipsec failover?
On May 13, 2014, at 22:13, Giancarlo Razzolini grazzol...@gmail.com wrote: go there, this e-mail would be too big. If you want I can elaborate more. yes please! do elaborate a bit! fl
Multihoming with carp possible? and ipsec failover?
Hello Misc-Users, I'm looking in to the possibility to do multihoming (more than one isp) on a Carp setup. To do live failover if one isp goes down, the other takes over. Just as carp does if one of the routers goes down. I'm thinking that in combination with ifstated it might be possible, but have yet to find someone that has actually done it sofar. Next issue if the first one is possible. The proposed router in question is a IPSEC gateway, with several nodes connected to it. Fail over here with just the carp and one isp is no issues. But if the remote node, has only one isp, and it has no carp or such, its just a plain obsd box running a site-to-site tunnel, routing everything (0.0.0.0/0) over the tunnel. How would one manage to do a failover to the second isp of the above box, without loss of the tunnel during fail over. Regards, Magnus
Re: Multihoming with carp possible? and ipsec failover?
On Tue, May 13, 2014 at 4:58 AM, Magnus mag...@tokra.org wrote: Hello Misc-Users, I'm looking in to the possibility to do multihoming (more than one isp) on a Carp setup. To do live failover if one isp goes down, the other takes over. Just as carp does if one of the routers goes down. You can do this with OpenBGPd, but CARP can only fail out between two routers sharing the same IP (at its most basic setup, more complicated setups are possible, obviously). If both ISPs are routing to the same netblock, then you can fail in the way you want. Otherwise, take a look at ifstated(8), and modify your routing tables or do dynamic routing. I'm thinking that in combination with ifstated it might be possible, but have yet to find someone that has actually done it sofar. A solution I've used in the past is a controlled endpoint that represents the exit for the network. The ISPs acted as pure transit for the external network, the VPN carried to a common end point(s). Next issue if the first one is possible. The proposed router in question is a IPSEC gateway, with several nodes connected to it. Fail over here with just the carp and one isp is no issues. But if the remote node, has only one isp, and it has no carp or such, its just a plain obsd box running a site-to-site tunnel, routing everything (0.0.0.0/0) over the tunnel. How would one manage to do a failover to the second isp of the above box, without loss of the tunnel during fail over. Take a look at sasyncd(8).
Re: Multihoming with carp possible? and ipsec failover?
Em 13-05-2014 08:58, Magnus escreveu: Hello Misc-Users, I'm looking in to the possibility to do multihoming (more than one isp) on a Carp setup. To do live failover if one isp goes down, the other takes over. Just as carp does if one of the routers goes down. I'm thinking that in combination with ifstated it might be possible, but have yet to find someone that has actually done it sofar. Next issue if the first one is possible. The proposed router in question is a IPSEC gateway, with several nodes connected to it. Fail over here with just the carp and one isp is no issues. But if the remote node, has only one isp, and it has no carp or such, its just a plain obsd box running a site-to-site tunnel, routing everything (0.0.0.0/0) over the tunnel. How would one manage to do a failover to the second isp of the above box, without loss of the tunnel during fail over. Regards, Magnus Magnus, I used some years ago a carp setup with 4 ISP's on the same machine, no problem there. You are right about using ifstated. There are several ways to accomplish this. You can even use relayd instead of ifstated. The carp setup will probably not interfere with your isp failover part, if you carefully craft your ifstated.conf. I suggest you put on a piece of paper all the possible different states your setup can have and you can replicate them on the ifstated.conf. Nowadays I don't use carp, but I do have one OpenBSD firewall with 2 ISP's and it works very well. Your first concern is to determine how you'll be able to detect a link failure. If your IPSEC router has snmp, and it sends the appropriates MIB's when you snmpwalk it, them you might be able to detect a link failure by simply monitoring the interfaces changes in your router. If not, then you'll have to use a external method. My method is to use 3 consecutive ping's to 3 different ip addresses and, if all 3 of them fails, them I failover the link and change state in ifstated. And, I use the multipath option in OpenBSD, so I can have more than one gateway configured in the routing table. You can play with their priorities to accomplish the failover, or you can simply remove a route and add the other. As I mentioned there are lot's of ways to accomplish this. I even make my ifstated mail me when a link failure occurs and with it gets back. There is also the pf.conf part but, if I'd go there, this e-mail would be too big. If you want I can elaborate more. Cheers, -- Giancarlo Razzolini GPG: 4096R/77B981BC
