Thanks a lot, Armin. I know how os.fork() works and about copy-on-write too, so as soon as I got time, I'll try to fix it. My interest in translating and playing then with pypy was exactly py.execnet =)
Armin Rigo wrote: > Hi, > > On Thu, Aug 28, 2008 at 10:34:51PM +0600, Vetoshkin Nikita wrote: >> Is there a direction, where I could digg to help? Something to begin with. >> I don't have enough experience but I'd like to take part (even very >> small) in this project. > > Sure. Do you understand the issue? It's related to the way a Unix > process starts a new subprocess: os.fork() followed by os.execv() in the > child process. The issue is that os.fork() fails if the parent process' > total memory is more than half of your RAM+swap, because it tries to > create a copy of the process, and the copy would also need more than > half of your RAM+swap. (In reality, os.fork() doesn't copy the memory > at all, but creates "shared pages" of memory so that memory pages are > duplicated only when one of the two processes really modifies it; but > still os.fork() has to mark all the memory as *potentially* used, and > raises MemoryError if there isn't enough.) > > A solution might be to split translate.py in two processes: translate.py > would start a subprocess when it starts, and then do all the work > (consuming a lot of RAM); but when it needs to start new processes, e.g. > call gcc, it would instead ask the subprocess to start the new > processes. More precisely, the code in pypy.translator.tool.cbuild > would stop using distutils directly, and instead ask the subprocess to > use distutils. It looks even easy to do with the help of a > py.execnet.PopenGateway() created when translate.py starts (see the py > lib documentation at http://codespeak.net/py/). > > > A bientot, > > Armin -- --- С уважением, Ветошкин Никита Инженер поддержки систем IP-телефонии Сервисный центр Компания Naumen _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
