Reid Kleckner wrote: > On Tue, Feb 2, 2010 at 8:57 PM, Collin Winter <collinwin...@google.com> wrote: >>> Wouldn't it be possible to have the compiler approach work >>> in three phases in order to reduce the memory footprint and >>> startup time hit, ie. >>> >>> 1. run an instrumented Python interpreter to collect all >>> the needed compiler information; write this information into >>> a .pys file (Python stats) >>> >>> 2. create compiled versions of the code for various often >>> used code paths and type combinations by reading the >>> .pys file and generating an .so file as regular >>> Python extension module >>> >>> 3. run an uninstrumented Python interpreter and let it >>> use the .so files instead of the .py ones >>> >>> In production, you'd then only use step 3 and avoid the >>> overhead of steps 1 and 2. >> >> That is certainly a possibility if we are unable to reduce memory >> usage to a satisfactory level. I've added a "Contingency Plans" >> section to the PEP, including this option: >> http://codereview.appspot.com/186247/diff2/8004:7005/8006. > > This would be another good research problem for someone to take and > run. The trick is that you would need to add some kind of "linking" > step to loading the .so. Right now, we just collect PyObject*'s, and > don't care whether they're statically allocated or user-defined > objects. If you wanted to pursue offline feedback directed > compilation, you would need to write something that basically can map > from the pointers in the feedback data to something like a Python > dotted name import path, and then when you load the application, look > up those names and rewrite the new pointers into the generated machine > code. It sounds a lot like writing a dynamic loader. :)
You lost me there :-) I am not familiar with how U-S actually implements the compilation step and was thinking of it working at the functions/methods level and based on input/output parameter type information. Most Python functions and methods have unique names (when combined with the module and class name), so these could be used for the referencing and feedback writing. The only cases where this doesn't work too well is dynamic programming of the sort done in namedtuples: where you dynamically create a class and then instantiate it. Type information for basic types and their subclasses can be had dynamically (there's also a basic type bitmap for faster lookup) or in a less robust way by name. For the feedback file the names should be a good reference. > It sounds like a huge amount of work, and we haven't approached it. > On the other hand, it sounds like it might be rewarding. Indeed. Perhaps this could be further investigated in a SoC project ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 03 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
