Hey Dirkjan, On Thu, Jan 21, 2010 at 11:16 AM, Dirkjan Ochtman <dirk...@ochtman.nl> wrote: > On Thu, Jan 21, 2010 at 18:32, Collin Winter <collinwin...@google.com> wrote: >> I added startup benchmarks for Mercurial and Bazaar yesterday >> (http://code.google.com/p/unladen-swallow/source/detail?r=1019) so we >> can use them as more macro-ish benchmarks, rather than merely starting >> the CPython binary over and over again. If you have ideas for better >> Mercurial/Bazaar startup scenarios, I'd love to hear them. The new >> hg_startup and bzr_startup benchmarks should give us some more data >> points for measuring improvements in startup time. > > Sounds good! I seem to remember from a while ago that you included the > Mercurial test suite in your performance tests, but maybe those were > the correctness tests rather than the performance tests (or maybe I'm > just mistaken). I didn't see any mention of that in the proto-PEP, in > any case.
We used to run the Mercurial correctness tests at every revision, but they were incredibly slow and a bit flaky under CPython 2.6. Bazaar's tests were faster, but were flakier, so we ended up disabling them, too. We only run these tests occasionally. >> One idea we had for improving startup time for apps like Mercurial was >> to allow the creation of hermetic Python "binaries", with all >> necessary modules preloaded. This would be something like Smalltalk >> images. We haven't yet really fleshed out this idea, though. > > Yeah, that might be interesting. I think V8 can do something similar, right? Correct; V8 loads a pre-compiled image of its builtins to reduce startup time. > What I personally would consider interesting for the PEP is a (not too > big) section evaluating where other Python-performance efforts are at. > E.g. does it make sense to propose a u-s merge now when, by the time > 3.3 (or whatever) is released, there'll be a very good PyPy that > sports memory usage competitive for embedded development (already does > right now, I think) and a good tracing JIT? Or when we can compile > Python using Cython, or Shedskin -- probably not as likely; but I > think it might be worth assessing the landscape a bit before this huge > change is implemented. I can definitely work on that. http://codespeak.net:8099/plotsummary.html should give you a quick starting point for PyPy's performance. My reading of those graphs is that it does very well on heavily-numerical workloads, but is much slower than CPython on more diverse workloads. When I initially benchmarked PyPy vs CPython last year, PyPy was 3-5x slower on non-numerical workloads, and 60x slower on one benchmark (./perf.py -b pickle,unpickle, IIRC). My quick take on Cython and Shedskin is that they are useful-but-limited workarounds for CPython's historically-poor performance. Shedskin, for example, does not support the entire Python language or standard library (http://shedskin.googlecode.com/files/shedskin-tutorial-0.3.html). Cython is a super-set of Python, and files annotated for maximum Cython performance are no longer valid Python code, and will not run on any other Python implementation. The advantage of using an integrated JIT compiler is that we can support Python-as-specified, without workarounds or changes in workflow. The compiler can observe which parts of user code are static (or static-ish) and take advantage of that, without the manual annotations needed by Cython. Cython is good for writing extension modules without worrying about the details of reference counting, etc, but I don't see it as an either-or alternative for a JIT compiler. > P.S. Is there any chance of LLVM doing something like tracing JITs? > Those seem somewhat more promising to me (even though I understand > they're quite hard in the face of Python features like stack frames). Yes, you could implement a tracing JIT with LLVM. We chose a function-at-a-time JIT because it would a) be an easy-to-implement baseline to measure future improvement, and b) create much of the infrastructure for a future tracing JIT. Implementing a tracing JIT that crosses the C/Python boundary would be interesting. Collin Winter _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
