Good day, Stephen,
First off: thanks for making a detailed report. That makes my job
a _lot_ easier.
I've forwarded your message on to the IP masquerading list in the
hopes that others might find the information useful either now or when
someone searches through the archives in the future. To protect your
privacy, I've hidden part of the real IP addresses you use below. I
sincerely hope you don't mind spreading the knowledge around...
On Mon, 27 Jul 1998, Stephen Briggs wrote:
> ..for a bit of (<offtopic>) help?
>
> I've just read through the webpage you have up for Mason.. I've had
> zero luck in tracking down someone who might know what is going
> wrong with my firewall setup, so I'm crossing my fingers and ask you
> (After convincing the Executive staff here that our company
> REALLY didnt want a WinNT backbone, I'm now eating my
> lunch). You do see to know what you are doing.
That depends on the day ;-), but thanks.
> I'm having problems, and I thought I'd write you before I ventured
> into the various Usenet groups. I do hope these types of requests for
> assistance are welcome. if not, let me know, and I'll seek help
> elsewhere.
Glad to help!
> My setup:
>
> Linux firewall, 2.0.34, compiled following your guidelines (though I
> added in masq, in case it is decided to use it later).
>
> eth0-ip address XXX.XXX.XXX.3
> eth1-ip address 192.168.2.1
>
> There is 1 test-workstation on the 192 network, 192.168.2.10. This
> station can ping both eth0 and eth1 and the firewall machine can ping
> 192.168.2.10 (the firewall can also ping any other host on the
> internet). But the test-workstation cannot ping ANYTHING else.
>
> My question lies here: Your HOWTO says "if you can (ping an outside
> address) you have not turned off IP forwarding. Make sure this is the
> way you want it.". Well, the linux kernel was compiled with
> CONFIG_IP_FORWARD=y. But, I have found no documentation anywhere to
> find out if it is merely installed, but not turned on, or installed and
> working.
(Side note: there _are_ firewall types where it is necessary to
turn off IP forwarding; the firewall we're looking at requires that IP
forwarding be turned ON.)
I'll bring this up now, because this is probably one of the
causes of the problem you're seeing.
Your suspicion about "installed but not turned on" is very likely
to be correct. Despite the fact that you compiled the kernel to have the
_ability_ to forward packets, that may need to be supplemented with a
specific request to tell it to _actually_ forward packets. I can see the
logic in the choice made by the kernel developers to turn this off by
default as most Linux systems are end nodes (workstations and servers)
rather than routers, but it still bugs me as it's by no means obvious that
this needs to be handled manually by router/firewall guys like ourselves.
It's one of those ulcer-causing things we learn <sigh>.
Check by:
cd /proc/sys/net/ipv4
ls -al ip_for*
You should see an entry like "ip_forward" or "ip_forwarding",
depending on your specific kernel revision. Try:
cat ip_forward
or
cat ip_forwarding
as appropriate. The command should return either 0 or 1; 0 means
the system will flatly refuse to forward any packets that it recieves
which are not destined for it, 1 means it will perform normal routing
tasks.
Some distributions handle this automatically at boot time; if so,
you should find out how it's enabled or disabled and use the appropriate
configuration file (cd /etc/rc.d ; grep 'ip_for' * */* */*/* | less).
If it doesn't seem to be specially handled, add the following to
/etc/rc.d/rc.local :
if [ -f /proc/sys/net/ipv4/ip_forward ]; then
echo 1 >/proc/sys/net/ipv4/ip_forward
echo Enabling IP forwarding
elif [ -f /proc/sys/net/ipv4/ip_forwarding ]; then
echo 1 >/proc/sys/net/ipv4/ip_forwarding
echo Enabling IP forwarding
else
echo Unable to enable IP forwarding. Missing forwarding sysctl.
echo Is IP forwarding enabled in this kernel?
fi
> What I am left with right now is a linux box that will accept pings and
> telnet requests from two different IP addresses, XXX.XXX.XXX.4 (real
> internet) and 192.168.2.1. The machine behind the firewall can do
> nothing except talk with the firewall machine. Once I telnet into the
> firewall, DNS look-ups, pings, etc., work just fine, so there is no
> problem in the cross between the 192.168.* network are the XXX.XXX.*
> network. But, it does not work from the local machine itself.
Keep in mind that when you telnet into your box from the 192
network, you have a TCP/IP connection that starts at .10 and ends at .1,
no matter what you do on the router. If you start up a ping or a telnet
to the outside world, it has _nothing at all_ to do with the original
telnet packets, that only need to bounce back and forth between two hosts
that know that each other are just across the internal wire. This is why
you can "get" to the outside world from your internal box when you telnet
to it.
> Here's the ipfwadm output:
> # ipfwadm -F -l
> IP firewall forward rules, default policy: accept
> # ipfwadm -I -l
> IP firewall input rules, default policy: accept
> # ipfwadm -O -l
> IP firewall output rules, default policy: accept
>
> Everything is accepted.
Not terribly secure <grin>, but perfect for troubleshooting.
> ifconfig:
>
> # ifconfig
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Bcast:127.255.255.255 Mask:255.0.0.0
> UP BROADCAST LOOPBACK RUNNING MTU:3584 Metric:1
> RX packets:8 errors:0 dropped:0 overruns:0
> TX packets:8 errors:0 dropped:0 overruns:0
>
> eth0 Link encap:Ethernet HWaddr 00:60:08:1C:76:D1
> inet addr:XXX.XXX.XXX.3 Bcast:XXX.XXX.XXX.127
> Mask:255.255.255.128
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:3903 errors:0 dropped:0 overruns:0
> TX packets:771 errors:0 dropped:0 overruns:0
> Interrupt:9 Base address:0xff00
>
> eth1 Link encap:Ethernet HWaddr 00:60:08:1C:76:DD
> inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0
>
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1692 errors:0 dropped:0 overruns:0
> TX packets:876 errors:0 dropped:0 overruns:0
> Interrupt:9 Base address:0xfe80
(Side note; I assume these are cards that don't mind sharing an
interrupt...)
> Looks good to me, and should pass, according to your printout (it is
> very close to yours.. the difference, I'm on a sub-netted class C on
> the internet).
>
> route:
> # route
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> XXX.XXX.XXX.0 * 255.255.255.128 U 0 0 6
> eth0
> 192.168.2.0 * 255.255.255.0 U 0 0 2
> eth1
> 127.0.0.0 * 255.0.0.0 U 0 0 1
> lo
> default XXX.XXX.XXX.1 0.0.0.0 UG 1 0 7
> eth0
>
> Again, according to your table, everything looks OK.. the big
> difference, I have a 0.0.0.0 in the default genmask, where your table
> had a '*'.
Not an issue, I think. You've already demonstrated that the
default route works as you can get to the Internet from the router.
> If I had to guess, I would think that it has something to do with the
> route, because it seems that the firewall doesn't know what to do with
> the packets from the 192.* network, it isn't routing them to the
(The ip_forward sysctl above should take care of not forwarding
packets.)
Routing decisions (with the exception of some very new options in
the 2.1 kernel series that don't apply here) are made _solely_ on the
basis of the packet's _destination_. If the kernel receives a packet that
isn't destined for an internal IP address (and forwarding is turned on),
it looks to the routing table and (simplifying a bit) looks for a routing
table entry that matches the _destination_ IP address, then either dumps
that packet onto the appropriate cable (if the destination address is
directly connected to the router) or hands off the packet to the machine
in the "gateway" field, in the hopes that said gateway will send it on
its merry way. At this specific stage of the linux' IP stack, it wouldn't
matter if the source address were 334567.-16.3+1/4.pi/2 (a mail server in
Bolivia, I think :-).
> outside. Plus, monitoring the firewall with 'tcpdump host
> XXX.XXX.XXX.3' confirms that only requests made by the firewall machine
> itself (i.e., from a shell) cause activity, while any action for the
> test-workstation brings about no output.
When you're using tcpdump on a machine with more than one Ethernet
card, make sure you specify wich interface to listen to:
On the test machine (192.168.2.10), try
ping 199.245.105.1 #Avoid DNS issues, just ping an IP I know exists.
While that's running (and failing, for reasons that we'll see in a
minute), try the following on the router one at a time:
tcpdump -i eth1 -q -t ip -n
This should show the ping packets coming from 192.168.2.10 to
199.245.105.1. That shows the packets are arriving on the ethernet wire
to the router. Now:
tcpdump -i eth0 -q -t ip -n
Same as above, except you're listening on the outbound wire,
showing that the packets are, indeed, getting forwarded to the appropriate
router. Before you did the above sysctl trick, this would have shown _no_
packets. With the sysctl in place, you should see packets flowing out,
ala (192.168.2.10 > 199.245.105.1), but no packets coming back, ala
(199.245.105.1 > 192.168.2.10). Why?
Please forgive me if you already know this, but it's worth
repeating as a general lesson in routing that will save you hours of
headache if you remember this one principle:
"For two hosts to communicate via TCP/IP, both must know where to
send the packets to get them one step closer to the other end, and every
router in the middle must know how to get one step closer to both
endpoints." In short, if _either_ host doesn't have it's default route
set, or _any_ router in the middle doesn't know how to get one step closer
to either host, _IP communication between those two hosts will fail_.
I won't bore a man with his neck on the Windows NT chopping block
with "stop here and check each machine's routing table to make sure each
machine involved can get one step closer to the other." You don't have
access to most of the routers anyways. Besides, the real problem lies in
your (correct, but relevant) choice of IP addresses for eth1.
The 192.168.X.Y addresses are recommended because Internic calls
them "reserved" addresses; reserved for internal office use. It's
intended that a building that does not connect to the Internet use one of
these address blocks. Obviously, thousands of buildings around the world
could be using the 192.168.2.X address block you've chosen. Let's say
that you and 30 other buildings do _exactly_ what you've done above, using
that same 192.168.2.X network block, and try to ping 199.245.105.161. The
packet leaves the reserved network, travels through a few routers (all of
which have no trouble nudging packets on to 199.245.105.1) and finally
arrive there. Remember, every machine involved uses the _destination
address_ (199.245.105.1) to decide where to send the packet next.
199.245.105.1 creates a ping reponse (with a destination address
of your box) and hands it off to its router. It might get a few more
steps before an Internet backbone router says "Huh? Where the heck do I
send this? This is a reserved address which could exist at any of
thousands of locations around the world. I'll just drop it."
I'll bet you didn't know that Cisco routers even have Soundblaster
Cards and voice emulation software... :-)
Your ping (half) succeeds! The machine at the other end of the
ping actually gets the packets; it's the routers in the middle that refuse
to complete the round trip.
> I don't really want to use masq-ing, due to the limitations, and the
> modules that need to be installed (the programmers here would gripe that
>
> not every one of their little toys works, etc..). So a filtering
> firewall is all that I want, but I cant seem to get just that working.
I'm sorry, but masquerading is _required_ in this case. The linux
router/firewall replaces the reserved IP address with its own
XXX.XXX.XXX.3. When you do that ping again with masquerading in place,
199.245.105.1 believes it is being pinged by the external IP address of
your router (XXX.XXX.XXX.3). Since both end points know how to get a step
closer to each other and the routers in the middle don't drop packets any
more, you can finally see the outside world from your lan.
BTW - if this fails and you're not sure if ICMP masquerading is
compiled in, try telnetting to some known IP address out in the real
world. Ping uses ICMP and thus requires ICMP masquerading to ping a host
on the Internet.
Doing this is not a big deal. You've already compiled a kernel
with IP masquerading (did you also turn on ICMP masquerading? - you
should). Simply find out what modules are available in your kernel (try:
cd /lib/modules/`uname -r`/ipv4 ; ls -a ip_masq* ), then insert them at
boot time with the following lines in /etc/rc.d/rc.local, probably near
the end (following output should be what you need for 2.0.34; other
kernels may have more or less modules):
modprobe ip_masq_cuseeme
modprobe ip_masq_ftp
modprobe ip_masq_irc
modprobe ip_masq_quake
modprobe ip_masq_raudio
modprobe ip_masq_vdolive
As for the programmers, you'll probably find that doing the above
makes them gripe a lot _less_ than if you didn't do the above. :-) The
only real warning is that they won't be able to accept _incoming_
connections of any type unless you want to get into ipautofw and the like.
Phew!
Allright, how to do this? Try the following (not tested, as I
don't have your network setup):
/sbin/ipfwadm -F -f #Remove any existing forwarding rules
/sbin/ipfwadm -F -a accept -m -S 192.168.2.0/24 -D 0/0
(in command, not grammatical, order:
-add a firewall rule to the
-forwarding list
-append it to the end (check it last, unless other rules added after me)
-accept the packet
-m masquerade it, replacing its source address with the IP I would use if
the packet were coming directly from me.
-make this rule apply only to packets coming from my IP block
-make this rule apply only to packets going to anywhere (not needed,
technically)
The kernel understands that it should automatically accept any
packets coming back in response to one of these masqueraded outbound
packets.
When you do this, you want to to avoid having people from the
outside world come back into your lan by lying about _their_ IP address;
to aboid this spoofing, put the following rule at the top of your incoming
rules:
/sbin/ipfwadm -I -i deny -W eth0 -S 192.168.2.0/24
In english, don't allow any packets arriving on our real world interface
(eth0) that claim to have a source address of a machine that should be
connected to eth1 - naughty, naughty!
You can still do packet filtering, but it needs to be done,
generally, with input and output rules. This certainly isn't a problem,
as you could put together a pretty good firewall with only input or only
output rules.
If you don't know how to set those up, see the .sig below. <grin>
> Even with total access setup:
>
> # ipfwadm -F -l
> IP firewall forward rules, default policy: accept
> type prot source destination ports
> acc tcp 192.168.0.0/16 anywhere any -> any
> acc udp 192.168.0.0/16 anywhere any -> any
> # ipfwadm -O -l
> IP firewall output rules, default policy: accept
> type prot source destination ports
> acc tcp 192.168.0.0/16 anywhere any -> any
> acc udp 192.168.0.0/16 anywhere any -> any
> # ipfwadm -I -l
> IP firewall input rules, default policy: accept
> type prot source destination ports
> acc udp 192.168.0.0/16 anywhere any -> any
> acc tcp 192.168.0.0/16 anywhere any -> any
>
>
> I get nothing when I try to ping or telnet.
Please accept the following as constructive criticism; you've got
the right start, but it needs some work.
1) You're using a class C network correctly in your ifconfig and
routing table, but you've chosen to use a '/16' (class B) in your rules.
This makes the rules apply to any packet between 192.168.0.0 and
192.168.255.255, rather than the 192.168.2.0 to 192.168.2.255 that I think
you intended. Use '192.168.2.0/24' instead to select the appropriate
class C.
2) In each of the three independant categories, you have two rules
that are checked. If the conditions match either rule, the packet is
allowed in, out or through, as appropriate. If neither rule matches, the
packet falls through to the default policy which allows it in, out or
through. In short, because you have no deny rules, this is the exact same
firewall you described in the beginning - everything goes.
The two simplest types of firewalls consist of either:
- a bunch of input and/or output "deny" rules for traffic you _don't_ want
with a default policy of allow (less work, but less secure), or
- a bunch of input and/or output "allow" rules for traffic you _do_ want
with a default policy of deny (more secure, but more work and more
likelihood that you'll block the Cobol guy's new netmeeting program).
3) You should also remember the icmp protocol when doing rules;
without it, a lot of necesary TCP/IP error messages won't get through and
many ping programs will fail to work.
4) If you adjust your above ruleset to have a default policy of
deny for input, output or forwarding, _no_ traffic will travel. Why?
Because (almost) every IP conversation consists of packets travling from A
to B and packets returning from B back to A. Your above rules cover the
outbound packets (A is a machine on your 192 network), but don't allow the
reponses back in. You would need a corresponding rule to allow return
traffic (hint: try '-D 192.168.2.0/24' rules _in addition to_ the '-S
192.168.2.0/24' rules you already have).
> Thank you for your time,
You're quite welcome. _Please_ let me know if any of this helped,
and how the project comes out in the end. I'd be happy to answer any more
questions you have.
A quick note; you mention above "my document", "my guidelines",
and "my HOWTO", but I think others deserve the credit for the documents to
which you refer. References to the documents I _have_ written usually
can't be reprinted in mixed company... ;-)
> Stephen Briggs
> www.ondemandinc.com
Cheers,
- Bill
---------------------------------------------------------------------------
Unix _is_ user friendly. It's just very selective about who its friends
are. And sometimes even best friends have fights.
William Stearns ([EMAIL PROTECTED])
Mason, buildkernel, and named2hosts are at: http://www.pobox.com/~wstearns
---------------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
For daily digest info, email [EMAIL PROTECTED]
