On 3/10/07, Caleb Clausen <[EMAIL PROTECTED]> wrote:
> I have a script (below) which attempts to make an index out of all the
> man pages on my system. It takes a while, mostly because it runs man
> over and over... but anyway, as time goes on the memory usage goes up
> and up and never down. Eventually, it runs out of ram and just starts
> thrashing up the swap space, pretty much grinding to a halt.
Hey Caleb,
Running your test for 15 minutes my memory usage climbed to 30Mb. It
was still slowly climbing which is not a good sign but not enough to
bring my system to a halt. Anyway, I tried using valgrind's memcheck
on it and I couldn't find a leak in the Ferret code. Perhaps it is a
leak in your version of Ruby, although I doubt it. Here is the most
significant output from valgrind with --show-reachable=yes set;
==7636== 110,880 bytes in 6,930 blocks are still reachable in loss
record 15 of 20
==7636== at 0x4020396: malloc (vg_replace_malloc.c:149)
==7636== by 0x40C175F: st_insert (st.c:288)
==7636== by 0x40D1E55: rb_ivar_set (variable.c:1056)
==7636== by 0x40D1FC2: rb_iv_set (variable.c:1959)
==7636== by 0x40D2003: rb_name_class (variable.c:282)
==7636== by 0x408BCBB: boot_defclass (object.c:2462)
==7636== by 0x408D020: Init_Object (object.c:2549)
==7636== by 0x40798A0: rb_call_inits (inits.c:54)
==7636== by 0x4061E5C: ruby_init (eval.c:1382)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
==7636==
==7636==
==7636== 187,248 bytes in 11,703 blocks are still reachable in loss
record 16 of 20
==7636== at 0x4020396: malloc (vg_replace_malloc.c:149)
==7636== by 0x40C184F: st_init_table_with_size (st.c:154)
==7636== by 0x40C18B6: st_init_strtable_with_size (st.c:193)
==7636== by 0x4095FBD: Init_sym (parse.y:5885)
==7636== by 0x4079896: rb_call_inits (inits.c:52)
==7636== by 0x4061E5C: ruby_init (eval.c:1382)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
==7636==
==7636==
==7636== 514,228 bytes in 11,687 blocks are still reachable in loss
record 17 of 20
==7636== at 0x401F6D5: calloc (vg_replace_malloc.c:279)
==7636== by 0x40C1870: st_init_table_with_size (st.c:158)
==7636== by 0x40C1914: st_init_table (st.c:167)
==7636== by 0x40C196F: st_init_numtable (st.c:173)
==7636== by 0x40CFEB6: Init_var_tables (variable.c:28)
==7636== by 0x407989B: rb_call_inits (inits.c:53)
==7636== by 0x4061E5C: ruby_init (eval.c:1382)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
==7636==
==7636==
==7636== 965,584 bytes in 60,349 blocks are still reachable in loss
record 18 of 20
==7636== at 0x4020396: malloc (vg_replace_malloc.c:149)
==7636== by 0x40C1692: st_add_direct (st.c:307)
==7636== by 0x4095D1A: rb_intern (parse.y:6067)
==7636== by 0x40CFED7: Init_var_tables (variable.c:30)
==7636== by 0x407989B: rb_call_inits (inits.c:53)
==7636== by 0x4061E5C: ruby_init (eval.c:1382)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
==7636==
==7636==
==7636== 1,088,800 bytes in 50,609 blocks are still reachable in loss
record 19 of 20
==7636== at 0x4020396: malloc (vg_replace_malloc.c:149)
==7636== by 0x4074E50: ruby_xmalloc (gc.c:121)
==7636== by 0x40CF72F: ruby_strdup (util.c:634)
==7636== by 0x4095CFF: rb_intern (parse.y:6066)
==7636== by 0x40CFED7: Init_var_tables (variable.c:30)
==7636== by 0x407989B: rb_call_inits (inits.c:53)
==7636== by 0x4061E5C: ruby_init (eval.c:1382)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
==7636==
==7636==
==7636== 2,374,520 bytes in 4 blocks are still reachable in loss record 20 of 20
==7636== at 0x4020396: malloc (vg_replace_malloc.c:149)
==7636== by 0x40737F9: add_heap (gc.c:351)
==7636== by 0x4061D74: ruby_init (eval.c:1372)
==7636== by 0x8048600: main (in /usr/bin/ruby1.8)
As you can see, non of this has anything to do with Ferret. If you
haven't used valgrind before and you want to try it there, here is
how;
valgrind --leak-check=yes ruby calebs_test.rb 2> res
You'll probably want to capture the output (like I have here) as it is
*very* long for ruby scripts. Lots of warnings from the ruby
internals. Let me know if you try this and you find anything unusual.
Incidentally, I'm not sure what the other bug you are chasing is but
it may have something to do with the encoding of the man pages. I
don't think they are UTF-8 so if your locale is set to UTF-8 it will
cause some problems in the analysis.
Cheers,
Dave
--
Dave Balmain
http://www.davebalmain.com/
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
