On Tue, Oct 07, 2008 at 12:47:49PM -0400, Mark Guzdial wrote:
> We lost one of our Swiki servers last month - it just kept crashing more and
> more often, and we couldn't figure out why. We eventually simply took it
> off-line, and moved our necessary Swikis from that machine to another server.
>
> Now, our main Swiki server, swiki.cc.gatech.edu, is crashing a couple times a
> day. Below are some notes that our tech support folks just sent me.
>
> Has anyone seen anything like this? Any suggestions for debugging or even
> fixing?
>
> Thanks!
> Mark
>
> I had to restart the swiki.cc server software twice
> today. I searched for large XML files and they all seem to be less
> than 10KB.
>
> I did not find a debug.log file...just a LowSpaceDebug.log file.
> Also, one symptom is that when the server is doing fine...the swiki.cc
> squeak process seems to eat about 20% or less of the cpu. But, when
> it gets hosed it takes 100% cpu. I used strace and it's spinning on
> gettimeofday calls...but they are very slow....
>
> ...
> gettimeofday({1223328462, 522789}, NULL) = 0
> gettimeofday({1223328462, 522876}, NULL) = 0
> gettimeofday({1223328462, 522936}, NULL) = 0
> gettimeofday({1223328462, 522991}, NULL) = 0
> ...
>
> When I do the same strace call on coweb.cc, it's doing the same
> gettimeofday calls but much, much more quickly.
Hi Mark,
The gettimeofday() activity is just what a (unix) VM normally does
at the lowest level, keeping its clock up to date and so forth. If it
is doing this "slowly" it just means that the VM (and possibly the
server) is totally bogged down. Given that you are seeing an LowSpaceDebug.log
file, the obvious culprit would be an image growing too large, leading
to swapping and general performance meltdown. Bottom line: Something is
using too much memory, or failing to make objects available for garbage
collection.
It's easy to reproduce this on any Squeak image. Just keep creating
objects until your object memory is bigger than available memory, and
you will get similar symptoms. Results will vary on different operating
systems, but once you are out of physical memory something has got to
give.
Dave
_______________________________________________
Pws mailing list
Pws@cc.gatech.edu
https://mailman.cc.gatech.edu/mailman/listinfo/pws
