On Tue, Feb 2, 2010 at 8:57 PM, Collin Winter <collinwin...@google.com> wrote: >> Wouldn't it be possible to have the compiler approach work >> in three phases in order to reduce the memory footprint and >> startup time hit, ie. >> >> 1. run an instrumented Python interpreter to collect all >> the needed compiler information; write this information into >> a .pys file (Python stats) >> >> 2. create compiled versions of the code for various often >> used code paths and type combinations by reading the >> .pys file and generating an .so file as regular >> Python extension module >> >> 3. run an uninstrumented Python interpreter and let it >> use the .so files instead of the .py ones >> >> In production, you'd then only use step 3 and avoid the >> overhead of steps 1 and 2. > > That is certainly a possibility if we are unable to reduce memory > usage to a satisfactory level. I've added a "Contingency Plans" > section to the PEP, including this option: > http://codereview.appspot.com/186247/diff2/8004:7005/8006.
This would be another good research problem for someone to take and run. The trick is that you would need to add some kind of "linking" step to loading the .so. Right now, we just collect PyObject*'s, and don't care whether they're statically allocated or user-defined objects. If you wanted to pursue offline feedback directed compilation, you would need to write something that basically can map from the pointers in the feedback data to something like a Python dotted name import path, and then when you load the application, look up those names and rewrite the new pointers into the generated machine code. It sounds a lot like writing a dynamic loader. :) It sounds like a huge amount of work, and we haven't approached it. On the other hand, it sounds like it might be rewarding. Reid _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
