Re: [freenet-support] Return of the Out of memory, Part LCXIX

2010-04-02 Thread Matthew Toseland
On Tuesday 09 March 2010 20:49:34 Evan Daniel wrote:
 On Tue, Mar 9, 2010 at 3:34 PM, Dennis Nezic denn...@dennisn.dyndns.org 
 wrote:
  As I (and I'm sure others) mentioned before, my node is still going
  down (crashing?) regularly -- roughly weekly. I currently allocate it
  150MB of (precious) ram. Here are my last 2 (of infinity) heap reports
  from the jvm dump:

I have filed a bug for this. Please register on the bug tracker and Monitor 
Issue on this bug:
https://bugs.freenetproject.org/view.php?id=4030

The next build should be slightly improved on this, I have just fixed a bug 
that was using at least 20MB of RAM for no good reason.
 
  Heap
   def new generation   total 46080K, used 41323K
   eden space 40960K, 100% used
   from space 5120K,   7% used
   to   space 5120K,   0% used
   tenured generation   total 102400K, used 99586K
    the space 102400K,  97% used
   compacting perm gen  total 12288K, used 11390K
    the space 12288K,  92% used
     ro space 10240K,  61% used
     rw space 12288K,  60% used
 
  Heap
   def new generation   total 46080K, used 41400K
   eden space 40960K, 100% used
   from space 5120K,   8% used
   to   space 5120K,   0% used
   tenured generation   total 102400K, used 102399K
    the space 102400K,  99% used
   compacting perm gen  total 12288K, used 11214K
    the space 12288K,  91% used
     ro space 10240K,  61% used
     rw space 12288K,  60% used
 
  Is there NO way that freenet can do a better job cleaning up? (I
  believe this happens even if I don't have (much) currently in the queues
  -- Ie. if I did heavy activity days before -- or maybe even on it's own
  without any manually-initiated activity, although I would have to test
  this more :\.)

150M is a bit low if you have stuff queued - especially inserts. Do you still 
get OOMs with *nothing* queued? (I suggest deleting or moving out of the way 
your node.db4o to be sure; if you move it out of the way be careful to move 
persistent-temp-* as well). Exactly what plugins are you running? Currently the 
default is 192M. We're trying to get it to be auto-detected but that is not 
deployed yet.
 
  Maybe we can add more debugging info as to where all the memory is
  allocated? Ie. which structures? (And hopefully decide that we can Let
  Go of some of them :|.)

Here's part of what I do:
wrapper.java.additional.3=-Xloggc:freenet.loggc
(In wrapper.conf)
Then tail -f freenet.loggc. If it is constantly doing Full GC's, it is on the 
edge. At which point, jmap can be very helpful (you might need the JDK):
$ jmap -histo -F pid of freenet
$ jmap -dump:live,format=b,file=dump.filename

The latter is particularly useful. You can then do:
$ jhat dump.filename
This opens a web server on 127.0.0.1:7000 on which you can investigate exactly 
what is using memory. You could alternatively send me the dump, I'm not sure 
whether the tools are backward compatible format-wise.

Another thing that is helpful is a stack dump (on the stats page, or run.sh 
dump, or kill -QUIT pid of freenet, in any case it goes to stdout).
 
 What plugins are you running (complete list)?  Can you provide a copy
 of your full stats page, in advanced mode?

Freetalk+WoT needs quite a bit of memory. XMLSpider needs absurd amounts. All 
of these can be optimised substantially.
 
 Evan Daniel


signature.asc
Description: This is a digitally signed message part.
___
Support mailing list
Support@freenetproject.org
http://news.gmane.org/gmane.network.freenet.support
Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support
Or mailto:support-requ...@freenetproject.org?subject=unsubscribe

Re: [freenet-support] Return of the Out of memory, Part LCXIX

2010-03-09 Thread Evan Daniel
On Tue, Mar 9, 2010 at 3:34 PM, Dennis Nezic denn...@dennisn.dyndns.org wrote:
 As I (and I'm sure others) mentioned before, my node is still going
 down (crashing?) regularly -- roughly weekly. I currently allocate it
 150MB of (precious) ram. Here are my last 2 (of infinity) heap reports
 from the jvm dump:

 Heap
  def new generation   total 46080K, used 41323K
  eden space 40960K, 100% used
  from space 5120K,   7% used
  to   space 5120K,   0% used
  tenured generation   total 102400K, used 99586K
   the space 102400K,  97% used
  compacting perm gen  total 12288K, used 11390K
   the space 12288K,  92% used
    ro space 10240K,  61% used
    rw space 12288K,  60% used

 Heap
  def new generation   total 46080K, used 41400K
  eden space 40960K, 100% used
  from space 5120K,   8% used
  to   space 5120K,   0% used
  tenured generation   total 102400K, used 102399K
   the space 102400K,  99% used
  compacting perm gen  total 12288K, used 11214K
   the space 12288K,  91% used
    ro space 10240K,  61% used
    rw space 12288K,  60% used

 Is there NO way that freenet can do a better job cleaning up? (I
 believe this happens even if I don't have (much) currently in the queues
 -- Ie. if I did heavy activity days before -- or maybe even on it's own
 without any manually-initiated activity, although I would have to test
 this more :\.)

 Maybe we can add more debugging info as to where all the memory is
 allocated? Ie. which structures? (And hopefully decide that we can Let
 Go of some of them :|.)

What plugins are you running (complete list)?  Can you provide a copy
of your full stats page, in advanced mode?

Evan Daniel
___
Support mailing list
Support@freenetproject.org
http://news.gmane.org/gmane.network.freenet.support
Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support
Or mailto:support-requ...@freenetproject.org?subject=unsubscribe