If you can't log in -- even on the console -- then rebooting is really
your only option. Ctrl-Alt-Del should bring the system down cleanly if
you haven't disabled that functionality. Otherwise, just toggle the
power.
The symptoms you're seeing could well be due to filesystem problems or
to some filesystem filling up (/tmp is a prime suspect) or due to
running out of memory+swap. Some sort of memory leak sounds pretty
likely actually.
Probably best to bring the system up in single user mode and run fsck on
all the filesystems manually -- that will show if you've got h/w
problems with drives and possibly with disk controllers or cabling too.
Then check for overfull filesystems. You may not find any --
rebooting
can clear a number of conditions where disk space is not released back
to the OS properly after use. You may or may not find any clues as to
what went wrong in the system logs. In the absence of any other clues,
the only option is to monitor the server closely and wait for something
similar to happen again. Hopefully if there is a next time, you'll be
able to catch it and fix the underlying problem before it takes the
machine out a second time.
Yes, I can't log in. I get a login prompt, but no password prompt. I'm
going to try ctrl-alt-del and see what happens.
Crossing fingers...
Sorry I missed that you can't login.
Good luck
So, ctrl-alt-del did the trick. I was able to log in and actually, the
whole box came up and everything seems to be working.
I thought for sure I'd find that my /var directory was full up, but it's
only at 77% (that's the weak spot on this box... I wish I'd made the
/var partition bigger.)
The message log is full of these messages:
38054 Jul 2 08:13:02 qu kernel: swap_pager_getswapspace(16): failed
If I run swapinfo, I get this:
[mas...@qu:log]> swapinfo
Device 1K-blocks Used Avail Capacity
/dev/ar0s1b 2055952 208 2055744 0%
I looked back in the log file to see if there were any clues when the
problem began and found this:
Jul 2 03:19:25 qu kernel: swap_pager: out of swap space
Jul 2 03:19:26 qu kernel: swap_pager_getswapspace(7): failed
Jul 2 03:19:26 qu kernel: pid 93543 (mysqld), uid 88, was killed: out
of swap space
Jul 2 03:19:26 qu kernel: pid 85077 (ruby18), uid 1023, was killed: out
of swap space
Jul 2 03:19:25 qu root: Check for bad ssh behavior
Jul 2 03:20:05 qu root: Check for bad ssh behavior
Jul 2 03:20:49 qu kernel: swap_pager_getswapspace(16): failed
Jul 2 03:20:49 qu kernel: swap_pager_getswapspace(15): failed
Jul 2 03:20:49 qu kernel: swap_pager_getswapspace(14): failed
Jul 2 03:20:49 qu kernel: swap_pager_getswapspace(16): failed
Jul 2 03:20:49 qu last message repeated 2 times
It pretty much goes on forever from there.
Is there any other place I can look for clues as to why I ran out of
swap space? This machine is basically a webserver, running apache/mysql
and ruby on rails. It's been running for over a year with no problems.
No new software introduced on the box, recently.
-- John
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
