Giancarlo Razzolini wrote:
Aaron escreveu:
I am trying to configure ifstated on an i386 4.2 Stable pair of openbsd
firewalls but having some issues on how to determine connectivity of a
backup/secondary wan interface.
The carp states seem solid and preempt seems to work great. The only
thing I'm really worried about is an upstream link dying, carp staying
master and traffic getting blackholed. I want ifstated to simply change
the default route to the backup wan interface should connectivity out
the primary get interrupted and then switch back when primary
connectivity comes back. I'm just trying to get it figured out on one
machine first before I move to the second. I'm having trouble figuring
out if there is connectivity on the backup wan interface. I read some
posts that suggested using ping -I so that the pings go out the
appropriate interface, but this seems to not work, if i try to ping
anything other than the backup wan's gateway, it still goes out the
default route.. It is only able to ping the gateway address and with
the (-r) option the pinged host has to be on the directly connected
network.
snip...
Now this is what i call a lot of info. :) Well, let me drop my
experience with ifstated and ping, and multiple wan links. First of all,
avoid pinging external address like you would avoid devil himself. I say
this, because it's isn't as reliable as many people think. How to do
then? My advice is, use snmp. Almost all (if not all of them) network
devices, which are fcc compliant, must have support for snmp, at least
version 1 of it. Install net-snmp on openbsd, and do a snmpwalk on the
modem, router, etc. Most of them will come with snmp enabled and with
the default communities "public" and "private". As you won't be
changing anything, i recommend using the public comm. Try this:
snmpwalk -v 1 -c public <ip of your router>
This will give you a lot of info. There is a snmp MIB called IF. As you
might guess, it refers to the interfaces of the device. This is the mib
you will most certainly use. Take a look at the following output from
one of my adsl devices:
IF-MIB::ifDescr.1 = STRING: loopback (pseudo ethernet)
IF-MIB::ifDescr.2 = STRING: ti
IF-MIB::ifDescr.3 = STRING: Bridge
IF-MIB::ifDescr.4 = STRING: Ethernet
IF-MIB::ifDescr.5 = STRING: Ethernet over USB
IF-MIB::ifDescr.6 = STRING: ATM
IF-MIB::ifDescr.7 = STRING: RFC-2684B PPPoE Proxy
IF-MIB::ifDescr.8 = STRING: PPPoE
IF-MIB::ifType.1 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.2 = INTEGER: adsl(94)
IF-MIB::ifType.3 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.4 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.5 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.6 = INTEGER: atm(37)
IF-MIB::ifType.7 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.8 = INTEGER: ppp(23)
.
.
.
IF-MIB::ifOperStatus.1 = INTEGER: up(1)
IF-MIB::ifOperStatus.2 = INTEGER: up(1)
IF-MIB::ifOperStatus.3 = INTEGER: up(1)
IF-MIB::ifOperStatus.4 = INTEGER: up(1)
IF-MIB::ifOperStatus.5 = INTEGER: down(2)
IF-MIB::ifOperStatus.6 = INTEGER: up(1)
IF-MIB::ifOperStatus.7 = INTEGER: up(1)
IF-MIB::ifOperStatus.8 = INTEGER: up(1)
The ifDescr atributes tell you what kind of if is this. In this specific
device i monitor the adsl, atm, and ppp if's. Any one of those that goes
down, mean that your wan link is dead. The attribute that you will use
to check it is the ifOperStatus. If some of the 3 if's i mentioned
before is with it ifOperStatus down, instead of up, you certainly is
with your wan link down. Then you can easily create a shell script to
accomplish this task. As ifstated tests expect a 0 for success and 1 for
error, your script only need to return this. Then you can call it
directly from ifstated, beside this you can overcheck and se if there
physical ethernet link. I do 3 checks in my ifstated: First the snmp
check, to check for wan connectivity directly with the device, second i
do ping the router to see if i can reach it (i know what i said about
ping before, but this one is different, and can help you) and third i
check the ethernet interface for connectivity. This way you can deal
with the 3 cases: no wan connectivity, but if up and the router up (call
your provider), no wan connectivity, and the router is down, but the if
is up (most certainly your router is hang up, so take a look at it) and
the third case, the if is down, can have 2 meanings: the router is
totally down or the physical if is with problems. This way i can say you
are all covered up. :) I also send nice mails to myself to inform of the
three cases.
Now to the failover part. It's not a good thing to change the router of
the firewall itself. It will die the connections of clients instantly,
what isn't a good thing. Instead, change the route of then using the
route-to statement of pf, and let the new conn's migrate to the other
wan link. Do this to avoid the connections dying when the primary link
backs up. I had this problems, as i do have 3 wan connections. Well,
there is much more to do, but the principle is here. I can help you with
more examples if you want.
My regards,
--
Giancarlo Razzolini
Linux User 172199
Red Hat Certified Engineer no:804006389722501
Moleque Sem Conteudo Numero #002
Slackware Current
OpenBSD Stable
Ubuntu 6.10 Edgy Eft
Snike Tecnologia em Informatica
4386 2A6F FFD4 4D5F 5842 6EA0 7ABE BBAB 9C0E 6B85
[demime 1.01d removed an attachment of type application/pgp-signature which had
a name of signature.asc]
This sounds really great.. and I've seen people suggest snmp as an
option in the past, but my snmp knowledge is somewhat limited. I'd very
much appreciate some guidance in this arena (off list if more
appropriate). One thing about the machines I've put together is, they
have 0 non-base packages installed. I am not sure of the requirements
of snmp, but i'm trying like the plague to keep this box minimal, if
nothing else I would like to avoid running into the libexpat issue that
will require me to install x-base.
I'd just like to point out as well, I did manage to get the ping going
out the correct interface and to use the proper route, with the
following route-to statements in my pf.conf, but i'm not exactly sure
why it's working since it doesn't seem like passing out on fxp1 and then
routing to fxp0 and vice versa would have much effect since if the
packet was passed out on fxp0, seems as if it'd be past the point of
being "routed"
pass out on fxp1 route-to ( fxp0 192.168.3.94 ) inet proto icmp from
fxp0 to any
pass out on fxp0 route-to ( fxp1 192.168.2.190 ) inet proto icmp from
fxp1 to any
I'm particularly interested in not changing the default route and using
the route-to as you suggested, as I ran into the immediate death of my
connection when the route changed.. and of course it wouldn't pick up
with the current download after the route change. I'd be very grateful
too if you can point me in the right direction to how i can prevent this.
Here is what i have as my ifstated.conf (mainly a modification of the
default with the addition of routing changes) and please note I wasn't
finished with it, but thought maybe if i was doing something blatantly
wrong, someone could point them out or me in the right direction.
# whether we have connectivity. Make sure the hosts are always up, or
# test multiple ip's, 'or'-ing the tests.
pnet = '( "ping -q -c 1 -w 1 -I 192.168.3.66 192.168.3.94 > /dev/null"
every 10 && \
"ping -q -c 1 -w 1 -I 192.168.3.66 www.hotbot.com > /dev/null" every
10 || \
"ping -q -c 1 -w 1 -I 192.168.3.66 192.168.3.94 > /dev/null"
every 10 && \
"ping -q -c 1 -w 1 -I 192.168.3.66 www.yahoo.com > /dev/null"
every 10 )'
bnet = '( "ping -q -c 1 -w 1 -I 192.168.2.162 192.168.2.190 > /dev/null"
every 10 && \
"ping -q -c 1 -w 1 -I 192.168.2.162 www.hotbot.com > /dev/null" every
10 || \
"ping -q -c 1 -w 1 -I 192.168.2.162 192.168.2.190 > /dev/null"
every 10 && \
"ping -q -c 1 -w 1 -I 192.168.2.162 www.yahoo.com > /dev/null"
every 10 )'
# The peer addresses below are the real ip addresses of the OTHER firewall
peer = '( "ping -q -c 1 -w 1 -I 192.168.3.66 192.168.3.67 > /dev/null"
every 10 && \
"ping -q -c 1 -w 1 -I 192.168.2.162 192.168.2.163 > /dev/null" every 10)'
state auto {
if $carp_up && $pnet
set-state primary
if $carp_down || !$pnet
set-state backup
}
state primary {
init {
run "ifconfig carp0 advskew 10"
run "ifconfig carp1 advskew 10"
run "ifconfig carp2 advskew 10"
run "ifconfig carp3 advskew 10"
run "route delete default && route add default 192.168.3.94"
run 'echo "using pnet route" | mailx -s "switch route"
<email address>'
}
if (! $pnet || ! $carp_up) && $bnet
set-state demoted
}
state demoted {
init {
run "ifconfig carp0 advskew 254"
run "ifconfig carp1 advskew 254"
run "ifconfig carp2 advskew 254"
run "ifconfig carp3 advskew 254"
run "route delete default && route add default 192.168.2.190"
run 'echo "using bnet route" | mailx -s "switch route"
<email address>'
}
if $pnet && $bnet
set-state primary
if $carp_sync && ! $pnet
set-state broute
}
state promoted {
init {
run "ifconfig carp0 advskew 0"
run "ifconfig carp1 advskew 0"
run "ifconfig carp2 advskew 0"
run "ifconfig carp3 advskew 0"
run "route delete default && route add default 192.168.3.94"
run 'echo "using pnet route" | mailx -s "which route"
<email address>'
}
# if $peer || ! $net
if ! $pnet && $bnet
set-state backup
}
state backup {
init {
run "ifconfig carp0 advskew 100"
run "ifconfig carp1 advskew 100"
run "ifconfig carp2 advskew 100"
run "ifconfig carp3 advskew 100"
run "route delete default && route add default 192.168.2.190"
run 'echo "using bnet route" | mailx -s "which route"
<email address>'
}
# The "sleep 5" below is a hack to dampen the $carp_sync when we come
# out of promoted state. Thinking about the correct fix...
if ! $carp_sync && $pnet && "sleep 5" every 10
if ! $carp_sync && $pnet
set-state promoted
}
# if ! $pnet && $carp_up && $bnet
# set-state broute
state broute {
init {
run "route delete default && route add default
192.168.2.190"
run 'echo "using bnet route" | mailx -s "which route"
<email address>'
}
if $carp_up && $pnet
set-state proute
}
state proute{
init {
run "route delete default && route add default 192.168.3.94"
run 'echo "using pnet route" | mailx -s "which route"
<email address>'
}
if $carp_sync && ! $pnet
set-state broute
}
