On 01/16/2012 09:59 PM, Timo Paulssen wrote:
Hello,
PyPy has a hard limit on its heap size. It can be specified with this
environment variable:
PYPY_GC_MAX The max heap size. If coming near this limit, it
will first collect more often, then raise an
RPython MemoryError, and if that is not enough,
crash the program with a fatal error. Try values
like '1.6GB'.
Thanks for the suggestion. I tried this, and it did not seem to change
the point of failure though the message looks a bit different:
----
drewes@ladastra:/home/drewes$ set | grep PYPY
PYPY_GC_MAX=6GB
drewes@ladastra:/home/drewes$ pypy grouper2.py --pass2 p5parsed-TSU6.txt
reloading reads . . .
loading reads, on record 1000000 ...
... loading reads, on record 14000000
Traceback (most recent call last):
File "app_main.py", line 51, in run_toplevel
File "grouper2.py", line 982, in <module>
Using too much memory, aborting
Aborted
----
According to 'top', the program is not actually using anywhere near 6GB
when the failure occurs. It is only using about 2GB when it fails.
Check out the rest of the evariables for the GC in
pypy/rpython/memory/gc/minimark.py
I will do that.
In general, the pypy GC isn't optimised to know what pages are in
memory or in swap, so each major collection will cause every single
page to be touched. If a large part of your pypy process will not fit
into the swap, you will probably experience very significant slowdowns.
That is good information, thanks.
Rich
_______________________________________________
pypy-dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-dev
