Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-21 Thread Martin Langhoff
On Sun, Dec 20, 2009 at 12:57 PM, Martin Langhoff martin.langh...@gmail.com wrote: Yep, I am interested in getting to the bottom of this. I think I have an initial assessment of the situation. Clearly, the mnesia DB got corrupted somehow. Because of that... - the init script did cannot stop

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-21 Thread Martin Langhoff
On Mon, Dec 21, 2009 at 3:14 PM, Martin Langhoff martin.langh...@gmail.com wrote: Now it's up on a pristine state, and I am monitoring it... Ok - the problem seems related to Moodle's control of ejabberd presence service. The sync between Moodle and ejabberd data (in mnesia) was taking too long,

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-21 Thread Devon Connolly
Ok then. Thanks a lot for the assistance. Things seem to be back to normal. I will look closer tomorrow when the kids are here. ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-20 Thread Martin Langhoff
On Sat, Dec 19, 2009 at 7:32 PM, Devon Connolly dev...@gmail.com wrote:  - Is there any disk anomaly? (Reboot forcing a fsck?) Not that I've noticed. Ok, but can you try doing a reboot that forces fsck? As follows: touch /forcefsck reboot or shutdown -Fr now Verify checked out on the

[Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-19 Thread Devon Connolly
Here is another example after it has been running all night. http://pastebin.com/m11537281 As you can see, these runaway beam processes vary greatly in there RAM usage. Also, they are always using 100% of the cpu. I will try to clear the DB now and see what happens. On Fri, Dec 18, 2009 at

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-19 Thread Devon Connolly
Changing the domain, I still get the following error when it tries (and fails to shutdown ejabberd). ___ Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller)

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-19 Thread Martin Langhoff
On Sat, Dec 19, 2009 at 1:31 PM, Devon Connolly dev...@gmail.com wrote: Beam is still consuming 100% of the cpu after a few minutes.  I'm going to leave that script running to see what it does over the next few hours. That's really abnormal. - Is there any disk anomaly? (Reboot forcing a

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-18 Thread Devon Connolly
Don't reinstall. If possible, let's try to debug this. If you're going to give up, just 1 - Backup /var/lib/ejabberd -- just tar it up 2 - Use the 'domain_config' script to change the domain -- this will re-generate the ejabberd mnesia database. What I'd do: change it to 'foo.com' and then

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-18 Thread Martin Langhoff
On Fri, Dec 18, 2009 at 1:37 PM, Devon Connolly dev...@gmail.com wrote: Anyway, back on topic...  Here is that script slightly modified running on a fresh boot.  I'm going to leave this looping and post the file to pastebin.  Here is an initial output after only like 10 minutes.  It will get

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-17 Thread Devon Connolly
XS Version: 0.6 1 GB Physical Ram, 2GB Swap 154 XO's Registered, Any number connected when the problem happens, 0-XX The XS is controlling dhcp but nothing out of the ordinary as far as leases are concerned. No Active Antenna # /home/idmgr/list_registration http://pastebin.com/m762076bb #

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-17 Thread Martin Langhoff
On Thu, Dec 17, 2009 at 11:35 AM, Devon Connolly dev...@gmail.com wrote: XS Version: 0.6 1 GB Physical Ram, 2GB Swap Ok - the RAM is on the low side for an XS but should handle 150 ok. # ejabberdctl connected-users ... I counted 12 lines in the output of connected-users. That should not cause

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-17 Thread Martin Langhoff
On Thu, Dec 17, 2009 at 1:12 PM, Martin Langhoff martin.langh...@gmail.com wrote On Thu, Dec 17, 2009 at 11:35 AM, Devon Connolly dev...@gmail.com wrote: XS Version: 0.6 1 GB Physical Ram, 2GB Swap Ok - the RAM is on the low side for an XS but should handle 150 ok. # ejabberdctl

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-17 Thread Devon Connolly
The server had an uptime of about 50 days before this occurred. There were no problems and nothing has changed in the 2 or so days since this problem began. Like had said previously, it seems to have occurred since reflashing and re-registering a student's XO, but I believe that to be a

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-17 Thread Martin Langhoff
On Thu, Dec 17, 2009 at 9:32 PM, Devon Connolly dev...@gmail.com wrote: The server had an uptime of about 50 days before this occurred.  There were no problems and nothing has changed in the 2 or so days since this problem began.  Like had said previously, it seems to have occurred since

Re: [Server-devel] Ejabberd CPU/RAM Spike - Crashes

2009-12-16 Thread Martin Langhoff
Hi Devon, Sure we can debug this. Lots of questions for you - version of XS? - How much physical RAM? - Number of XOs registered, and in use on the network when the problem happens - Output of the commands suggested in