Hey MA, On Fri, Jan 29, 2010 at 11:14 AM, M.-A. Lemburg <m...@egenix.com> wrote: > Collin Winter wrote: >> I added startup benchmarks for Mercurial and Bazaar yesterday >> (http://code.google.com/p/unladen-swallow/source/detail?r=1019) so we >> can use them as more macro-ish benchmarks, rather than merely starting >> the CPython binary over and over again. If you have ideas for better >> Mercurial/Bazaar startup scenarios, I'd love to hear them. The new >> hg_startup and bzr_startup benchmarks should give us some more data >> points for measuring improvements in startup time. >> >> One idea we had for improving startup time for apps like Mercurial was >> to allow the creation of hermetic Python "binaries", with all >> necessary modules preloaded. This would be something like Smalltalk >> images. We haven't yet really fleshed out this idea, though. > > In Python you can do the same with the freeze.py utility. See > > http://www.egenix.com/www2002/python/mxCGIPython.html > > for an old project where we basically put the Python > interpreter and stdlib into a single executable. > > We've recently revisited that project and created something > we call "pyrun". It fits Python 2.5 into a single executable > and a set of shared modules (which for various reasons cannot > be linked statically)... 12MB in total. > > If you load lots of modules from the stdlib this does provide > a significant improvement over standard Python.
Good to know there are options. One feature we had in mind for a system of this sort would be the ability to take advantage of the limited/known set of modules in the image to optimize the application further, similar to link-time optimizations in gcc/LLVM (http://www.airs.com/blog/archives/100). > Back to the PEP's proposal: > > Looking at the data you currently have, the negative results > currently don't really look good in the light of the small > performance improvements. The JIT compiler we are offering is more than just its current performance benefit. An interpreter loop will simply never be as fast as machine code. An interpreter loop, no matter how well-optimized, will hit a performance ceiling and before that ceiling will run into diminishing returns. Machine code is a more versatile optimization target, and as such, allows many optimizations that would be impossible or prohibitively difficult in an interpreter. Unladen Swallow offers a platform to extract increasing performance for years to come. The current generation of modern, JIT-based JavaScript engines are instructive in this regard: V8 (which I'm most familiar with) delivers consistently improving performance release-over-release (see the graphs at the top of http://googleblog.blogspot.com/2009/09/google-chrome-after-year-sporting-new.html). I'd like to see CPython be able to achieve the same thing, like the new implementations of JavaScript and Ruby are able to do. We are aware that Unladen Swallow is not finished; that's why we're not asking to go into py3k directly. Unladen Swallow's memory usage will continue to decrease, and its performance will only go up. The current state is not its permanent state; I'd hate to see the perfect become the enemy of the good. > Wouldn't it be possible to have the compiler approach work > in three phases in order to reduce the memory footprint and > startup time hit, ie. > > 1. run an instrumented Python interpreter to collect all > the needed compiler information; write this information into > a .pys file (Python stats) > > 2. create compiled versions of the code for various often > used code paths and type combinations by reading the > .pys file and generating an .so file as regular > Python extension module > > 3. run an uninstrumented Python interpreter and let it > use the .so files instead of the .py ones > > In production, you'd then only use step 3 and avoid the > overhead of steps 1 and 2. That is certainly a possibility if we are unable to reduce memory usage to a satisfactory level. I've added a "Contingency Plans" section to the PEP, including this option: http://codereview.appspot.com/186247/diff2/8004:7005/8006. Thanks, Collin Winter _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
