M.-A. Lemburg wrote: > Collin Winter wrote: >> I added startup benchmarks for Mercurial and Bazaar yesterday >> (http://code.google.com/p/unladen-swallow/source/detail?r=1019) so we >> can use them as more macro-ish benchmarks, rather than merely starting >> the CPython binary over and over again. If you have ideas for better >> Mercurial/Bazaar startup scenarios, I'd love to hear them. The new >> hg_startup and bzr_startup benchmarks should give us some more data >> points for measuring improvements in startup time. >> >> One idea we had for improving startup time for apps like Mercurial was >> to allow the creation of hermetic Python "binaries", with all >> necessary modules preloaded. This would be something like Smalltalk >> images. We haven't yet really fleshed out this idea, though. > > In Python you can do the same with the freeze.py utility. See > > http://www.egenix.com/www2002/python/mxCGIPython.html > > for an old project where we basically put the Python > interpreter and stdlib into a single executable. > > We've recently revisited that project and created something > we call "pyrun". It fits Python 2.5 into a single executable > and a set of shared modules (which for various reasons cannot > be linked statically)... 12MB in total. > > If you load lots of modules from the stdlib this does provide > a significant improvement over standard Python. > > Back to the PEP's proposal: > > Looking at the data you currently have, the negative results > currently don't really look good in the light of the small > performance improvements. > > Wouldn't it be possible to have the compiler approach work > in three phases in order to reduce the memory footprint and > startup time hit, ie. > > 1. run an instrumented Python interpreter to collect all > the needed compiler information; write this information into > a .pys file (Python stats) > > 2. create compiled versions of the code for various often > used code paths and type combinations by reading the > .pys file and generating an .so file as regular > Python extension module > > 3. run an uninstrumented Python interpreter and let it > use the .so files instead of the .py ones > > In production, you'd then only use step 3 and avoid the > overhead of steps 1 and 2. > > Moreover, the .so file approach > would only load the code for code paths and type combinations > actually used in a particular run of the Python code into > memory and allow multiple Python processes to share it. > > As side effect, you'd probably also avoid the need to have > C++ code in the production Python runtime - that is unless > LLVM requires some kind of runtime support which is written > in C++.
BTW: Some years ago we discussed the idea of pluggable VMs for Python. Wouldn't U-S be a good motivation to revisit this idea ? We could then have a VM based on byte code using a stack machines, one based on word code using a register machine and perhaps one that uses the Stackless approach. Each VM type could use the PEP 3147 approach to store auxiliary files to store byte code, word code or machine compiled code. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 01 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
