Hey Victor Did you look into integrating the pypy benchmarks that we added over the years?
On Thu, Aug 18, 2016 at 9:37 AM, Victor Stinner <victor.stin...@gmail.com> wrote: > Hi, > > After a few months of work, I created a "new" benchmark suite for Python: > > https://github.com/python/benchmarks > > It's based on: > https://hg.python.org/sandbox/benchmarks_perf > which is my fork the CPython benchmark suite: > https://hg.python.org/benchmarks > which is based on Unladen Swallow's benchmark suite (if I understood > correctly). > > Major differences: > > * Use the perf module to run benchmarks in multiple processes and > store results as JSON > * Create virtual environments using requirements.txt to download > dependencies from PyPI (rather than using old copies of libraries) > * Many libraries have been upgraded: see requirements.txt > > The project works on Python 2 and Python 3 (I tested 2.7 and 3.6). > > Known regressions: > > * Memory tracking is broken > * run_compare command is currently broken: use run (store result into > a file) + compare manually > * Some benchmarks have been removed: rietveld, spitfire (not on PyPI), > pystone, gcbench, tuple_gc_hell > * I only tested Linux, I expect issues on Windows. (I didn't try my > perf module on Windows yet.) > > I already allowed all Python core developers to push to the GitHub > project. We can create a new "benchmarks" (or "Performance" maybe?) > team if we want to allow more contributors who are not core > developers. > > PyPy, Pyston, Pyjion, Numba, etc. : Hey! it's now time to start to > take a look at my project and test it ;-) Tell me what is broken, what > is missing, and I will try to help you to move your project to this > new benchmark suite! > > As requested (suggested?) by Brett Canon, the Git repository has no > history, it only contains 1 commit! I'm really sorry of loosing all > the history and all authors, but it allows to start with a much > smaller repository: around 2 MB. The current benchmark repository is > more around 200 MB! > > TODO: > > * continue to upgrade libraries in requirements.txt. I failed to > upgrade Django to 1.10, it complains about a missing template engine > config setting. > * convert more code to the perf module, like "startup" tests > * run benchmarks and analyze results ;-) > * write more documentation explaining how to run reliable benchmarks > * ... > > Victor > _______________________________________________ > Speed mailing list > Speed@python.org > https://mail.python.org/mailman/listinfo/speed _______________________________________________ Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed
