We recently installed IP Filter on a number of Sun (SPARC) systems, most
running Solaris 2.6, one running Solaris 8, to provide "personal firewalls"
for the servers. By that, I mean they're not protecting other systems and not
using NAT, just protecting the servers themselves. Traffic is arriving for
the system on which IP Filter is running (some of it on "virtual" interfaces
sharing the physical interface) and outbound traffic is all locally
originated, nothing to be routed onward.
The ipf settings that are configurable via /etc/system should be at their
default values - we don't currently have any overrides in place - and IP
Filter was compiled without any changes to the source code.
In general, that appeared to work fine using IP Filter 3.4.25 (compiled with
Sun's C compiler, version varying between systems), once we sorted out
sensible configurations. However, there are a few residual oddities about
which we'd appreciate advice, especially a puzzle affecting FTP.
(1) We are using the "keep state" facility extensively in our rules. While
we've seen the documentation about how to change the state table size,
we're unclear how to decide what size is appropriate or what the configured
numbers actually represent. A configuration muddle leading to excessive
saved state ("keep state" used without "flags s") showed that the IPSTATE_SIZE
and IPSTATE_MAX settings are clearly not directly limiting the amount of
saved state, as judged by ipfstat -s output which reported hundreds of
thouands of active state entries. [That muddle was quickly fixed.]
Are they just determining the size of a hash table via which the state data
can be found, so that if too small (whatever that means in reality) it hits
performance but wouldn't result in blocked/dropped connections?
Even on our (moderately busy) web server, the ipfstat -s output shows only
e.g. "1406 bkts in use" and "1567 active" which suggests the default state
table size settings should be OK - true? Are there any "rules of thumb" for
deciding when the state table should be enlarged and by how much?
(2) It's clear from the documentation that re-transmitted packets from
already-closed stateful TCP connections may result in "blocked" log entries
for connections which completed normally, but we're a little worried about
the frequency with which we're seeing that (for an Apache web server, an
Inktomi Search search engine, and an INN news server).
For example, the log entries for yesterday show 0.1% of HTTP connections to
the web server logged blocked RST packets and 0.6% blocked ACK packets (some
but only a minority ACK with FIN or ACK with PUSH). Is that typical and
reasonable for web traffic through IP Filter, or should we be worrying about
it?
We have not had any complaints about connections being dropped randomly and
haven't seen any such problems ourselves, but we know from other contexts
that very often people assume a problem will be known about and fixed in due
course, and not bother reporting it. Or assume it was a transient glitch if
clicking their browser's reload button fixes the prioblem.
(3) [This is the "big" problem]
The web server uses the "mirror" FTP mirroring script from
http://sunsite.org.uk/packages/mirror/ to copy some of the web content from
a local FTP server. As noted above, NAT is not involved - IP Filter is just
protecting the individual servers - and the FTP connections would be from
the web server's canonical address to the FTP server.
Our IP Filter configuration allows all outbound traffic, with saved state
for SYN packets, which we expect to allow back in the corresponding response
packest. We therefore expected changing mirror's configuration to use passive
FTP (previously used active) to "just work".
Reality differed from expectation, and more or less every hourly mirror run
reported a random handful of the several hundred groups of files being
mirrored as having a lot of files deleted on the FTP server (not true!)
hence also deleting them on the web server. The next run would then
reinstate them.
Looking at the ethernet traffic showed that in the problem cases, halfway
through the transfer of an FTP directory listing from the FTP server to
mirror on the web server, IP Filter was sending a RST on the data connection
(passive FTP, so connection initiated by mirror on the web server system).
mirror saw the resulting closed connection as end of directory listing, and
concluded that files it had seen previously mirrored (which would have been
in the remainder of the directory listing) had been deleted. I'd guess the
actual file transfers may also have been affected, though if so it went
unreported (like the temporary disappearance of groups of files from the web
sites due to the directory listing problem...).
The ipf rules appeared to be appropriate for what we intended, so we tried
upgrading to the newly released V3.4.27 as it appeared to include some fixes
to state management, but that made no difference. We've "fixed" the problem
temporarily by allowing all incoming traffic to port 1024 and higher from
the FTP server, but that's not ideal and we'd like to identify and fix the
problem.
I believe that (without the "temporary fix" rules) the only rules that
should be involved in handling the passive FTP requests from mirror (the
rest are for specific incoming ports or non-TCP packets) are:
pass out quick proto tcp all flags S keep state keep frags
and
block return-rst in log quick proto tcp all
In other words, all outbound traffic TCP should be allowed (including FTP
control and data connections), with state kept based on the SYN packets,
hence incoming response packets should be allowed in. Any TCP packets not
allowed due to saved state should get a RST. Some mirror data connections
got RSTs half-way through, but I can't see why that should happen unless
we've misunderstood how the saved state is meant to work and it's timing
out prematurely in some cases, or there's a relevant problem in the versions
of IP Filter that we've tried.
Any suggestions for why that does not work for mirror (using passive FTP),
or how to investigate in more detail, would be very much appreciated. I've
checked recent mail to the list, but the mentions of FTP problems there all
seem to involve NAT.
John Line
--
University of Cambridge WWW manager account (usually John Line)
Send general WWW-related enquiries to [EMAIL PROTECTED]
