On Wed, Jan 18, 2012 at 12:30 AM, Rich Drewes <[email protected]> wrote: > On 01/16/2012 09:59 PM, Timo Paulssen wrote: >> >> Hello, >> >> PyPy has a hard limit on its heap size. It can be specified with this >> environment variable: >> >> PYPY_GC_MAX The max heap size. If coming near this limit, it >> will first collect more often, then raise an >> RPython MemoryError, and if that is not enough, >> crash the program with a fatal error. Try values >> like '1.6GB'. > > Thanks for the suggestion. I tried this, and it did not seem to change the > point of failure though the message looks a bit different: > > ---- > drewes@ladastra:/home/drewes$ set | grep PYPY > PYPY_GC_MAX=6GB > drewes@ladastra:/home/drewes$ pypy grouper2.py --pass2 p5parsed-TSU6.txt > > reloading reads . . . > loading reads, on record 1000000 ... > ... loading reads, on record 14000000 > > Traceback (most recent call last): > File "app_main.py", line 51, in run_toplevel > File "grouper2.py", line 982, in <module> > Using too much memory, aborting > Aborted > ---- > > According to 'top', the program is not actually using anywhere near 6GB when > the failure occurs. It is only using about 2GB when it fails.
That means you're running a 32bit program in a 64bit environment >> >> Check out the rest of the evariables for the GC in >> pypy/rpython/memory/gc/minimark.py > > I will do that. > >> >> In general, the pypy GC isn't optimised to know what pages are in memory >> or in swap, so each major collection will cause every single page to be >> touched. If a large part of your pypy process will not fit into the swap, >> you will probably experience very significant slowdowns. > > That is good information, thanks. > > Rich > > _______________________________________________ > pypy-dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/pypy-dev _______________________________________________ pypy-dev mailing list [email protected] http://mail.python.org/mailman/listinfo/pypy-dev
