On Tue, Jul 24, 2012 at 1:14 PM, Alex Gaynor <alex.gay...@gmail.com> wrote:
> > > On Tue, Jul 24, 2012 at 10:10 AM, Brett Cannon <br...@python.org> wrote: > >> >> >> On Mon, Jul 23, 2012 at 7:34 PM, Maciej Fijalkowski <fij...@gmail.com>wrote: >> >>> On Mon, Jul 23, 2012 at 11:46 PM, Brett Cannon <br...@python.org> wrote: >>> > >>> > >>> > On Mon, Jul 23, 2012 at 4:39 PM, Armin Rigo <ar...@tunes.org> wrote: >>> >> >>> >> Hi Brett, >>> >> >>> >> On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <br...@python.org> >>> wrote: >>> >> > That's what I'm trying to establish; how much have they diverged >>> and if >>> >> > I'm >>> >> > looking in the proper place. >>> >> >>> >> bm_mako.py is not from Unladen Swallow; that's why it is in >>> >> pypy/benchmarks/own/. In case of doubts, check it in the history of >>> >> Hg. The PyPy version was added from virhilo, which seems to be the >>> >> name of his author, on 2010-12-21, and was not changed at all since >>> >> then. >>> > >>> > >>> > OK. Maciej has always told me that a problem with the Unladen >>> benchmarks was >>> > that some of them had artificial loop unrolling, etc., so I had >>> assumed you >>> > had simply fixed those instances instead of creating entirely new >>> > benchmarks. >>> >>> No we did not use those benchmarks. Those were mostly completely >>> artificial microbenchmarks (call, call_method etc.). We decided we're >>> not really that interested in microbenchmarks. >>> >>> > >>> >> >>> >> >>> >> Hg tells me that there was no change at all in the 'unladen_swallow' >>> >> subdirectory, apart from 'unladen_swallow/perf.py' and adding some >>> >> __init__.py somewhere. So at least these benchmarks did not receive >>> >> any pypy-specific adapatations. If there are divergences, they come >>> >> from changes done to the unladen-swallow benchmark suite after PyPy >>> >> copied it on 2010-01-15. >>> > >>> > >>> > I know that directory wasn't changed, but I also noticed that some >>> > benchmarks had the same name, which is why I thought they were forked >>> > versions of the same-named Unladen benchmarks. >>> >>> Not if they're in own/ directory. >>> >> >> OK, good to know. I realized I can't copy code wholesale from PyPy's >> benchmark suite as I don't know the code's history and thus if the >> contributor signed Python's contributor agreement. Can the people who are >> familiar with the code help move benchmarks over where the copyright isn't >> in question? >> >> I can at least try to improve the Python 3 situation by doing things like >> pulling in Vinay's py3k port of Django, etc. to fill in gaps. I will also >> try to get the benchmarks to work with a Python 2.7 control and a Python 3 >> "experimental" target for comparing performance since that's what I need >> (or at least be able to run the benchmarks on their own and writing out the >> results for later comparison). >> >> Anything else that should be worked on? >> >> _______________________________________________ >> Speed mailing list >> Speed@python.org >> http://mail.python.org/mailman/listinfo/speed >> >> > The important thing is that once a benchmark is in the repo it can *never* > change including all the versions of dependencies, only Python can vary, > otherwise you kill the ability to actually do science with the numbers. > > So, e.g., I wouldn't pull in Vijnay's fork, since that's going to be > utterly obsolete in a few weeks probably, I'd wait to have django on py3k > for that work to all be merged into django itself. > If that's happening in a few weeks then I can wait. But remember my desire is to get benchmark numbers between Python 2.7 and Python 3.3 for my November keynotes so I can't punt indefinitely. -Brett > > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > >
_______________________________________________ Speed mailing list Speed@python.org http://mail.python.org/mailman/listinfo/speed
