Zooko, "Zooko O'Whielacronx" <zo...@zooko.com> writes: > I still think we can set up buildbots to do this. Here is a patch to > the GNUmakefile to make a target named "benchmark" which does I think > everything we need. > > Now we can configure the buildmaster to run "make benchmark" once per > day, or once per week, or only after no new patches have arrived in > the last 8 hours, or other policies of your choice. Buildbot will > automatically collect all the benchmark results and make them > browseable on the web. Let's start with that! the catch with this is, that 5 of the buildslaves run on a single physical piece of hardware which even though is quite beefy (4 CPUs and 16G RAM) can only sustain so much load -- especially IO. Running 5 benchmarks in parallel is not going to be much use anyway. On the other hand, if you look at the buildbot.darcs.net waterfall, you will see that the "6.12.1 Debian" slave has been running the benchmarks after each push -- for a while now (many months, maybe a year now?).
It is only running for HEAD (understandably) -- what is missing from the setup is adding comparison in time, both from the buildbot itself and comparison against released versions on the (preferably on the same machine). This is not completely trivial, though. Oh, and a glittery rendering. Yours, Petr. PS: You should only feed that "copy & paste below" section into reST tools. The rest is for your amusement if you are running the benchmark interactively. :) Maybe we need a --quiet option. PPS: Ok, having looked at your output, it is indeed messed up. I think someone will need to look into that. You could install 0.1.7 in the meantime, which I think worked. (cabal install "darcs-benchmark == 0.1.7") _______________________________________________ darcs-users mailing list darcs-users@darcs.net http://lists.osuosl.org/mailman/listinfo/darcs-users
