Hi Joel,
You're right that this problem isn't huge if restoring from a checkpoint
since you start from a clean code cache when restoring. The problem I
had was that I never got to the point where I wanted to take the
checkpoint because I was running out of memory.
After applying my fix [1], memory usage stabilized at around the guest
memory size + 1GiB instead of growing to more than 8 GiB (that's when
gem5 got killed).
BTW, you probably want to check out the heap checker in tcmalloc. It's
/way/ faster than Valgrind.
//Andreas
[1]
https://github.com/andysan/gem5/commit/99e25dfe196f2fc5edb250024a8bb23bb5a5fe7d
On 05/15/2013 08:12 PM, Joel Hestness wrote:
Hey guys,
To add a little detail to this, when I cleaned up the memory leaks in the
RubyPort a few weeks ago, I was using Valgrind to find memory errors and
leaks. I saw numerous decode cache entries reported by Valgrind that appear
to never get cleaned up. The Valgrind output for a record is of the
following form:
==18427== 24,016 bytes in 158 blocks are possibly lost in loss record 5,121
of 5,272
==18427== at 0x4C2B1C7: operator new(unsigned long) (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18427== by 0x68BC01:
X86ISAInst::X86Macroop::CALL_NEAR_I::CALL_NEAR_I(X86ISA::ExtMachInst,
X86ISA::EmulEnv) (decoder.cc:33404)
==18427== by 0x551C0C: X86ISA::Decoder::decodeInst(X86ISA::ExtMachInst)
(decoder.cc:108772)
==18427== by 0x40D393: X86ISA::Decoder::decode(X86ISA::ExtMachInst,
unsigned long) (decoder.cc:471)
==18427== by 0x40E0E6: X86ISA::Decoder::decode(X86ISA::PCState&)
(decoder.cc:516)
==18427== by 0x1329AFD: BaseSimpleCPU::preExecute() (base.cc:392)
==18427== by 0x131DC85: TimingSimpleCPU::completeIfetch(Packet*)
(timing.cc:658)
==18427== by 0xCF5DBB: EventQueue::serviceOne() (eventq.cc:207)
==18427== by 0xD43392: simulate(unsigned long) (simulate.cc:72)
==18427== by 0xAEF414: _wrap_simulate (event_wrap.cc:4798)
==18427== by 0x54E73B7: PyEval_EvalFrameEx (in
/usr/lib/libpython2.7.so.1.0)
==18427== by 0x54B2604: PyEval_EvalCodeEx (in
/usr/lib/libpython2.7.so.1.0)
From a cursory skim, it looks like Valgrind classifies all of these as
"possibly lost" records, and these records account for about half of all
the records in a reasonably long simulation. According to the memory size
of each record, these decode cache entries constitute about 10% of all
"possibly lost" memory by capacity (~800kB out of 8.4MB possibly lost
total).
@Andreas: The runs that I'm reporting results from here are when restoring
from a checkpoint, so I don't see any bloat that might be caused by Linux
boot. This suggests the problem may not be terrible when restoring from a
checkpoint.
Joel
On Wed, May 15, 2013 at 8:58 AM, Andreas Sandberg<[email protected]>wrote:
Hi Everyone,
I recently started experimenting with a new x86 user space (based on
Debian Wheezy) and ran into a problem with the decode cache. It seems like
the boot process (probably udev) is starting a lot of small processes,
which causes the decode cache size to explode since it stores every single
static instruction that the decoder encounters. On my system, I ended up
using more than 8 GiB of memory long before the the boot process completed.
My current solution is to set an upper limit on the decode cache size and
when that is reached, I randomly remove half of the entries. In practice,
it might be better to completely flush the cache since that keeps the code
a bit simpler.
Does anyone have any opinions about this? Gabe?
I'll push my proposed fix to my fixes branch [1] on GitHub later today if
anyone wants to have a look.
//Andreas
[1]
https://github.com/andysan/**gem5/tree/fixes<https://github.com/andysan/gem5/tree/fixes>
______________________________**_________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/**listinfo/gem5-dev<http://m5sim.org/mailman/listinfo/gem5-dev>
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
