On Wed, 13 Apr 2016 at 13:33 Zachary Ware <zachary.ware+py...@gmail.com> wrote:
> On Wed, Apr 13, 2016 at 1:57 PM, Maciej Fijalkowski <fij...@gmail.com> > wrote: > > Hi > > > > I have a radical idea: to take a pypy benchmark suite, update the > > libraries to newer ones and replace python benchmarks with that. The > > main reason being that pypy has a much better coverage of things that > > are not microbenchmarks, the list (in json): > > > > http://paste.pound-python.org/show/4YVq0fv6pv8rVOSmCTag/ > > > > Which is much more extensive than this: > > > > https://hg.python.org/benchmarks/file/tip/performance > > > > I'm willing to put *some* effort, what do people think? > > I'm in favor. My support has two conditions, though: > > 1) at least a majority of the benchmarks must be Python3 compatible. > Preferably 2/3 compatible, but I assume all of the PyPy benchmarks are > 2 compatible anyway. > Agreed (although I don't care about the 2/3 compatibility, just the 3 compat ;) . > > 2) and there should be an easy way to run the benchmarks against > exactly 1 interpreter (for use with speed.python.org). I initially > tried to set up speed.python.org using the PyPy benchmarks, but > quickly ran into issues with trying to use 'nullpython.py' as the > baseline Python. When I switched to using h.p.o/benchmarks, I added > the '--raw' flag to perf.py which allows the benchmarks to be run on > one interpreter instead of two. It was just a quick hack, though; I > have no problems with that feature completely changing (even invoking > it a different way is ok), so long as it exists. > > This project could probably start its life as > github.com/python/benchmarks and save us from having to migrate > h.p.o/benchmarks to GitHub. > Yep, I'm willing to postpone moving the benchmarks repo from hg.python.org if works starts on this idea and then not move the old repo at all if this succeeds. We could then make the people who care about the benchmarks the maintainers of the new repository and contribute to it directly (which means interested people from CPython, PyPy, Pyston, IronPython, and Jython). That way we all get exposure to everyone's benchmarks and there's no more benchmark fragmentation because some people disagree with another's approach (i.e. no more benchmark silos among the Python implementations). And just because we're talking a new repo, would it be worth considering relying on pip to grab libraries instead of embedding them in the repository, hence shrinking the overall size of the repo?
_______________________________________________ Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed
