Note: Our setup is a CentOS 6 box, running Shorewall 4.5.4 (from the
Prereqs:
First off, you should print out the MultiISP document from the Shorewall
website and start marking it up with notes:
http://shorewall.net/MultiISP.html
About 98% of the answers are in the MultiISP file or one of the other
shorewall documents. The other 2% are found on the mailing list.
A) I suggest renaming your ethernet devices to have more meaningful
names. In a CentOS environment, we do this in
/etc/sysconfig/network-scripts by:
- Renaming ifcfg-eth0 to ifcfg-newname
- Editing ifcfg-newname and using DEVICE=newname
We used "wancbl" (our cable modem, 25-30Mbps inbound, 2-3Mbps outbound,
DHCP address), "want1" (our T1 line, static IP), "lan1" and "lan2" for
the internal ethernet ports.
This made it much easier for us to see traffic when looking using "atop"
or other tools, rather then trying to remember eth0..eth3 and which was
what.
B) Turn off the NetworkManager service if you have it installed. Simply
doing "chkconfig NetworkManager off" takes care of it in CentOS/RHEL.
- The number one bugaboo when you have NetworkManager installed is that
it likes to overwrite your /etc/resolv.conf file (and does other things
not suitable for a firewall server).
C) We bonded our two LAN ethernet ports (lan1 and lan2) into a simple
active-backup (mode 1) bond called "bond0". So wherever we refer to
"bond0", that is our internal LAN address.
Setting up bonding is not hard, but beyond the scope of this.
D) We are still using IPv4 for everything. While our T1 provider is
talking about IPv6 rollout soon, our cable modem provider has no plans
at all.
E) Use a version control system for /etc, /usr/local and any other
directories where configuration files might be stored. My
recommendation has been FSVS (stores data into a SVN repository) for a
number of years. But you could also use git, hg, cvs, etc.
- It allows you to easily revert changes, without having to reach for
the backup tapes.
- You can leave notes to yourself in the commit messages (in addition to
leaving comments inside the configuration file). With FSVS, I can look
back at the entire history of the server and see when I installed and
configured a particular bit of software, what files I touched, and what
I changed.
- Doing a diff between the configuration right after you installed
something and after you mucked it up makes it easy to undo changes (or
see how you did something 2-3 years from now).
...
Files that we changed from the Shorewall 4.5.4 defaults:
A) Zones file (self-explanatory, we don't yet use a DMZ zone or anything
fancy like VLANs):
/etc/shorewall/zones
fw firewall
loc ipv4
net ipv4
B) Interfaces (options may not be 100% correct, but seems to work):
/etc/shorewall/interfaces
loc bond0 dhcp,logmartians,nosmurfs,required,tcpflags
net wancbl dhcp,logmartians,nosmurfs,optional,tcpflags
net want1 logmartians,nosmurfs,optional,tcpflags
In your case, since both WAN interfaces are DHCP, you'll need to use the
dhcp option on both. For us, only "wancbl" interface used a DHCP address.
Since DHCP is in use on the internal LAN, the bond0 interface also
needed the 'dhcp' option.
C) Policies (you should customize this based on how secure you want
things, letting the $FW talk to everything is probably not best):
/etc/shorewall/policies
loc net ACCEPT
net all DROP info
$FW loc ACCEPT
$FW net ACCEPT
all all REJECT info
D) Rules (you will be doing very custom things to this based on your
needs, we run SSH on multiple ports, only allow SSH from outside via
port 2222/tcp, plus we have NTP and BIND, plus allowing ICMP):
/etc/shorewall/rules
ACCEPT loc $FW tcp 22 # SSHD port
ACCEPT loc $FW tcp 2222 # 2nd SSHD port
ACCEPT net $FW tcp 2222 # only allow SSH over tcp/2222 from outside
ACCEPT $FW net udp ntp
ACCEPT loc $FW udp ntp
ACCEPT $FW loc udp ntp
REDIRECT loc ntp udp ntp # redirect NTP traffic to firewall NTP server
DNS/ACCEPT $FW net
DNS/ACCEPT loc $FW
DNS/ACCEPT $FW loc
DNS/ACCEPT loc net
ACCEPT loc $FW icmp
ACCEPT $FW loc icmp
ACCEPT $FW net icmp
ACCEPT net $FW icmp
E) Masq file (NAT for the LAN clients to the internet). The example
here uses 192.168.0.0/24 as our internal LAN address range (in reality
we use a 172.x.y.z range). The important thing is that you need to list
both WAN ethernet devices, with the same internal LAN address range for
both lines.:
/etc/shorewall/masq
wancbl 192.168.0.0/24
want1 192.168.0.0/24
F) Providers (I am still a bit fuzzy about things here). Note that we
use TRACK_PROVIDERS=Yes and USE_DEFAULT_RT=Yes in the shorewall.conf
file. This affects how you define your two providers:
/etc/shorewall/providers
provcbl 1 2 - wancbl detect balance=100
provt1 2 2 - want1 999.999.999.999 balance=1
- Replace 999.999.999.999 with "detect" if you are using DHCP, or enter
the static external IP address assigned to that interface.
- USE_DEFAULT_RT=Yes means that the DUPLICATE column should be left as
'-' in the providers file.
- TRACK_PROVIDERS=Yes automatically adds "track" to the OPTIONS column,
which is why I don't specify it as "track,balance=###".
- The 100:1 weight means that 99% of all traffic goes out over the
"wancbl" interface. Only if the cable modem is down, does it start to
route traffic over our T1 line.
- The provider names (1st column) will also show up in your routing
tables (see "shorewall show routing"). So we prefix our provider names
with "prov" to make it clearer.
- If you need specific traffic to go out over the secondary provider,
then you need to play with "rtrules" or "tcrules" (we don't use this
yet, so that is not a certainty). Get it working first without touching
those files.
- I have been unable to get "fallback" option working properly in
Shorewall 4.5.4. But the above works well enough for our purposes.
G) shorewall.conf (the main config file). I am only listing lines that
changed from the base install. You should *definitely* read the main
Shorewall documentation and make notes in the configuration file for why
you changed a particular value.
/etc/shorewall/shorewall.conf
VERBOSITY=2
NULL_ROUTE_RFC1918=Yes
RESTORE_DEFAULT_ROUTE=No
TRACK_PROVIDERS=Yes
USE_DEFAULT_RT=Yes
H) The "routestopped" file (for Shorewall v4.5.4), it has a different
name in later versions of Shorewall (stoppedrules). These are the
iptables rules that get generate whenever Shorewall shuts down.
bond0 - - icmp
bond0 - - tcp 22
bond0 - - udp 123
bond0 - - tcp 2222
want1 - source icmp
want1 - source tcp 2222
wancbl - source icmp
wancbl - source tcp 2222
...
That's the end of the *simple* stuff. Other then the providers file and
defining multiple WAN interfaces, it looks very similar to a single-ISP
setup.
...
LSM configuration notes:
- We used lsm-0.163-1.foo6.src.rpm for our LSM install on CentOS 6.
- The default LSM configuration is not quite correct for working with
Shorewall.
...
A) You need to change a few defaults in the lsm.conf file. These lines
were all added or changed inside the "defaults{}" block.
eventscript=/etc/lsm/script
notifyscript=
status=1
B) Before the "#EOF" line at the end of the /etc/lsm/lsm.conf file, you
need to add the following line:
include /var/lib/shorewall/lsm.conf
C) A sample /var/lib/shorewall/lsm.conf
connection {
name=T1
checkip=999.999.999.999
device=want1
ttl=2
}
connection {
name=Cablevision
checkip=999.999.999.999
device=wancbl
ttl=2
}
- You should replace "999.999.999.999" with the current DHCP IP address
- This file actually gets written out by Shorewall, but for testing LSM,
you will want to initially create it by hand.
D) You do *not* need to run LSM as a service. On CentOS 6, if
"chkconfig --list lsm" shows that LSM is running in runlevels 3-5, then
you should "chkconfig lsm off" to disable the service.
E) /etc/lsm/script - This is the script that LSM will run whenever the
status of a link changes (eventscript= argument in /etc/lsm/lsm.conf).
See attachment to this email for a copy of our LSM script (which is
slightly different then the Shorewall MultiISP example script).
It needs to perform the following tasks when a link goes "up":
- Remove the [interface].status file
- Execute the Shorewall "firewall enable {device}" script
When the link goes down, it needs to:
- Execute the Shorewall "firewall disable {device}" script
Note: The above only works for Shorewall 4.5.4 and later.
I have added some other logging statements to the script that shoves the
details and events into /var/log/lsm.log.
...
Now that LSM is setup, we can go back to setting up Shorewall to
interact with LSM.
...
A) /etc/shorewall/isusable
This *should* be already setup by default. It looks for
"[interface].status" files in the VARDIR, where LSM will write those
status files.
B) /etc/shorewall/params
You *could* define various things here instead of hardcoding them in the
other files. But we did not do so in our setup.
C) /etc/shorewall/started
This is the shell script commands that get run whenever shorewall starts
up. We decided to use the single solitary line in our "started" file
rather then check whether this was a "restart" command. The "start_lsm"
is a function defined in /etc/shorewall/lib.private
# [Re]Start LSM
start_lsm
D) /etc/shorewall/restored
This is covered in the multi-ISP document. It checks to see if LSM is
already running, otherwise it starts it.
# Start lsm if it isn't running
if [ -z "$(ps ax | grep 'lsm ' | grep -v 'grep ' )" ]; then
start_lsm
fi
E) /etc/shorewall/lib.private
The meat-and-potatoes for interacting with LSM. It defines the
"start_lsm()" shell function. See attached file.
- The checkip line can be written two ways. You can specify a static IP
address that is the "next hop out" for that particular interface. Or
you can use the "SW_(interfacename)_GATEWAY" environment variable for
DHCP setups and also specify a fallback address.
Static version:
checkip=999.999.999.999
DHCP version:
checkip=${SW_WANCBL_GATEWAY:-999.999.999.999}
- 999.999.999.999 in this file needs to be your "next-hop" gateway IP
address for that WAN interface, not the external IP address of your
firewall.
- If you need to go two+ hops out, then you will *also* need to add a
static "ip route". I'm not showing an example of that, and for cable
modem setups it is probably not needed.
- Our lib.private is somewhat custom, with some additional logging added
that is not in the MultiISP example document.
- start_lsm() is responsible for writing out the /var/shorewall/lsm.conf
file.
- We should be using more of Shorewall variables (such as VARDIR)
instead of hardcoding /var/lib/shorewall in places.
F) /etc/shorewall/findgw
This file may not exist by default, or may not be compatible with your
distro. Sample versions of the "findgw" script can be found at:
http://www.shorewall.net/pub/shorewall/contrib/findgw/
Without the proper "findgw" file, you will see that even though the
interface is up, there is no default route being added to that
provider's routing table (see "shorewall show routing").
...
I think that's everything that we changed when setting up our MultiISP
configuration. There are no warranties, but this should serve as a
trail of breadcrumbs to help you figure out what needs to be touched.
#!/bin/sh
#
# (C) 2009 Mika Ilmaranta <[email protected]>
# (C) 2009 Tom Eastep <[email protected]>
#
# License: GPLv2
#
# Beginning with Shorewall 4.4.23, it is not necessary to restart the
# firewall when an interface transitions between the usable and
# unusable states.
STATE=${1}
NAME=${2}
CHECKIP=${3}
DEVICE=${4}
WARN_EMAIL=${5}
REPLIED=${6}
WAITING=${7}
TIMEOUT=${8}
REPLY_LATE=${9}
CONS_RCVD=${10}
CONS_WAIT=${11}
CONS_MISS=${12}
AVG_RTT=${13}
if [ -f /usr/share/shorewall-lite/lib.base ]; then
VARDIR=/var/lib/shorewall-lite
STATEDIR=/etc/shorewall-lite
else
VARDIR=/var/lib/shorewall
STATEDIR=/etc/shorewall
fi
[ -f ${STATEDIR}/vardir ] && . ${STATEDIR}/vardir
cat <<EOM | mail -s "LSM (fw2-sec): ${NAME} ${STATE}, DEV ${DEVICE}"
${WARN_EMAIL}
Hi,
Connection ${NAME} is now ${STATE}.
Following parameters were passed:
newstate = ${STATE}
name = ${NAME}
checkip = ${CHECKIP}
device = ${DEVICE}
warn_email = ${WARN_EMAIL}
Packet counters:
replied = ${REPLIED} packets replied
waiting = ${WAITING} packets waiting for reply
timeout = ${TIMEOUT} packets that have timed out (= packet loss)
reply_late = ${REPLY_LATE} packets that received a reply after timeout
cons_rcvd = ${CONS_RCVD} consecutively received replies in sequence
cons_wait = ${CONS_WAIT} consecutive packets waiting for reply
cons_miss = ${CONS_MISS} consecutive packets that have timed out
avg_rtt = ${AVG_RTT} average rtt, notice that waiting and timed out
packets have rtt = 0 when calculating this
Your LSM Daemon
EOM
LSMLOG=/var/log/lsm.log
echo "--------------------------------------" >> $LSMLOG
if [ ${STATE} = up ]; then
# echo 0 > ${VARDIR}/${DEVICE}.status # Uncomment this line if you are running
Shorewall 4.4.x or earlier
rm -f ${VARDIR}/${DEVICE}.status
${VARDIR}/firewall enable ${DEVICE}
echo "EVENT: ${VARDIR}/firewall enable ${DEVICE} @ $(date)" >> $LSMLOG
else
# echo 1 > ${VARDIR}/${DEVICE}.status # Uncomment this line if you are running
Shorewall 4.4.x or earlier
${VARDIR}/firewall disable ${DEVICE}
echo "EVENT: ${VARDIR}/firewall disable ${DEVICE} @ $(date)" >> $LSMLOG
fi
/sbin/shorewall show routing >> $LSMLOG
echo "--------------------------------------" >> $LSMLOG
exit 0
#EOF
#
# Shorewall version 4 - lib.private File
#
# /etc/shorewall/lib.private
#
# Use this file to declare shell functions to be called in the other
# run-time extension scripts. The file will be copied into the generated
# firewall script.
#
# See http://shorewall.net/shorewall_extension_scripts.htm for additional
# information.
#
###############################################################################
start_lsm() {
LSMLOG=/var/log/lsm.log
echo "--------------------------------" >> $LSMLOG
echo "/etc/shorewall/lib.private -- start_lsm() @ $(date)" >> $LSMLOG
#
# Kill any existing lsm process(es)
#
echo "Kill any existing lsm processes" >> $LSMLOG
killall lsm 2> /dev/null
#
# Create the Shorewall-specific part of the LSM configuration. This file is
# included by /etc/lsm/lsm.conf
#
echo "Rewrite /var/lib/shorewall/lsm.conf" >> $LSMLOG
echo "$(/usr/bin/printenv)" >> $LSMLOG
cat <<EOF > /var/lib/shorewall/lsm.conf
connection {
name=T1
checkip=999.999.999.999
device=want1
ttl=2
}
connection {
name=Cablevision
checkip=${SW_WANCBL_GATEWAY:-999.999.999.999}
device=wancbl
ttl=2
}
EOF
#
# Since LSM assumes that interfaces start in the 'up' state, remove any
# existing status files that might have an interface in the down state
#
echo "Remove existing *.status files" >> $LSMLOG
rm -f /var/lib/shorewall/*.status
#
# Run LSM -- by default, it forks into the background
#
echo "START LSM" >> $LSMLOG
/usr/sbin/lsm /etc/lsm/lsm.conf >> $LSMLOG
echo "--------------------------------" >> $LSMLOG
}
------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Shorewall-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/shorewall-users
