Collin Winter wrote: > I added startup benchmarks for Mercurial and Bazaar yesterday > (http://code.google.com/p/unladen-swallow/source/detail?r=1019) so we > can use them as more macro-ish benchmarks, rather than merely starting > the CPython binary over and over again. If you have ideas for better > Mercurial/Bazaar startup scenarios, I'd love to hear them. The new > hg_startup and bzr_startup benchmarks should give us some more data > points for measuring improvements in startup time. > > One idea we had for improving startup time for apps like Mercurial was > to allow the creation of hermetic Python "binaries", with all > necessary modules preloaded. This would be something like Smalltalk > images. We haven't yet really fleshed out this idea, though.
In Python you can do the same with the freeze.py utility. See http://www.egenix.com/www2002/python/mxCGIPython.html for an old project where we basically put the Python interpreter and stdlib into a single executable. We've recently revisited that project and created something we call "pyrun". It fits Python 2.5 into a single executable and a set of shared modules (which for various reasons cannot be linked statically)... 12MB in total. If you load lots of modules from the stdlib this does provide a significant improvement over standard Python. Back to the PEP's proposal: Looking at the data you currently have, the negative results currently don't really look good in the light of the small performance improvements. Wouldn't it be possible to have the compiler approach work in three phases in order to reduce the memory footprint and startup time hit, ie. 1. run an instrumented Python interpreter to collect all the needed compiler information; write this information into a .pys file (Python stats) 2. create compiled versions of the code for various often used code paths and type combinations by reading the .pys file and generating an .so file as regular Python extension module 3. run an uninstrumented Python interpreter and let it use the .so files instead of the .py ones In production, you'd then only use step 3 and avoid the overhead of steps 1 and 2. Moreover, the .so file approach would only load the code for code paths and type combinations actually used in a particular run of the Python code into memory and allow multiple Python processes to share it. As side effect, you'd probably also avoid the need to have C++ code in the production Python runtime - that is unless LLVM requires some kind of runtime support which is written in C++. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 29 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
