On Thu, Dec 26, 2002 at 01:03:54AM -0500, Paul Chvostek wrote:
>
> Infrequently ... that is, perhaps once every few weeks, my mail server
> grinds to a halt. The load average climbs into the hundreds, processes
> start getting killed off, and all because something seems to want to
> launch as many instances of portmap as it possibly can. And for the
> life o' me, I can't figure out what.
>
> The box is currently running 4.7-STABLE, but it's been doing this off
> and on since 4.4-RELEASE. The box is not an NFS server, but it becomes
> a client from time to time, and portmap and nfsiod are launched at
> startup. There are no NFS devices listed in /etc/fstab, there is no
> /etc/exports and no other NFS-related daemons are running. If I catch
> things in time (and have a shell already open), I can usually recover
> with `killall portmap`. If I'm not around (which is usually the case),
> the box will either grind to a halt requiring a console reset, or merely
> spontaneously reboot.
The facts that you're only seeing this effect intermittently and that
it has persisted over 4 different FreeBSD releases and that it's only
you that seems to be affected suggest that it's something peculiar to
your environment that is triggering the problem. I've certainly run
FreeBSD boxes that make occasional NFS mounts via amd(8) during that
timeframe without any problem.
> I can see no other strange behaviour (or network traffic) going on with
> this box -- aside from this problem, it behaves perfectly.
>
> Does any of this sound familiar? Where do I look for the problem?
Hmmm... Did you ever happen to be monitoring network traffic starting
*before* portmap went crazy? If it could be possible to grab (using
'tcpdump -o filename') a chunk of network traffic spanning the onset
of crazyness it would be invaluable for debugging. However, that's
probably going to be pretty tricky to arrange...
Debugging this is going to be a tedious process of eliminating all the
possible causes until you can isolate what the problem is. There are
two angles of enquiry I'd start looking at.
i) Hardware problems. Are you being hit by intermittent network
flakyness? Look at the output of:
netstat -i
on the boxes in question (NFS server and client). You might even see
error counts on other asymptomatic machines around your network. If
your networking hardware supports it, do check for similar error
counts on switches/hubs/routers around your network. While you're
looking at all those machines, don't forget to check through the
equivalents of /var/log/messages and /var/log/all.log (well worth
editing /etc/syslog.conf to enable that) to see if there are any
suspicious log messages
If there's any entries under the Ierrs or Oerrs columns you probably
have a problem. Usually, those columns should read zero all the way,
but certain setups may cause a small number of errors to be registered
at boot time. If the error counts increase over time something is
definitely kaput. It can be as trivial as a badly plugged in network
cable.
ii) Enemy action. Well, not necessarily deliberate attempts to upset
your machine, but some other box that can knock over portmap by some
unfriendly stream of packets. Use ipfw(8) or ipf(8) to filter traffic
to ports 2049 and 111 so that only those machines that are directly
involved in normal NFS activity can access those services. Edit
/etc/hosts.allow to enable the hosts_options(5) controls --- remember
that RPC services and portmap have to be filtered by IP number rather
than hostname.
If the machine in question is exposed to the internet, you should have
all these protections in place already, and you should be logging each
and every unauthorized attempt to connect to those services, reading
those logs regularly and if necessary making representations to the
admins of the boxes used by the malefactors. Personally, I'd avoid
running any sort of RPC based service (and thus running portmap) at
all on a machine in such a position. RPC and portmap is too much of a
gift to anyone seeking to do nefarious things to your systems.
If all you need is occasional access to a file on the NFS server, then
you're almost certainly going to be better off cooking up a script to
do what you want using ssh(1) with passwordless access given using
public key authentication.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks
Savill Way
Marlow
Tel: +44 1628 476614 Bucks., SL7 1TH UK
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-questions" in the body of the message
