On Sun, Mar 13, 2011 at 11:30 PM, Stefan Marr <ph...@stefan-marr.de> wrote:
> Hi: > > For the others, I am currently adapting the benchmarking infrastructure > used for Pinocchio to be a bit more general, and enable me to integrate our > RoarVM benchmarking tools. > The goal is to have a framework that allows all kind of benchmarking, > written like unit tests. > One other idea Henrik was interested in is to be able to easily compare the > benchmark results of different versions of method-implementations, to see > whether optimizations were successful. > > Hi Stef. I would love this. Not only to see how successful an optimization was, but also, how much overhead your solution has ;) > > Camillo, even so I don't have a working version yet, I was trying to commit > my refactoring (perhaps for review). > However, the PinocchioVM project seems to be 'global readonly'. > > Would it be better to make it a stand-alone project? > Yes, please. Did you consider the repository PharoBenchmarks (even if the package can be called SBench)? because it would be good to collect them somewhere, and that seems a good enough place. BTW, how do you compare the benchmarks from PBenchmark to the ones that are in PharoBenchmarks ? Finally, I would also love to have such benchmarks run in each hudson build to see wether each commit to pharo make it slower/faster. I know Cyrille was working on that. I cc'ed him, since it would be nice to join forces. When we go with that step, as already mentioned, I would like to rename it. > > PBenchmark is a name that might prevent adoption. Not that I would like to > go into politics here, but perhaps we can consider a new name. > > Since the general idea is to write benchmarks like unit tests, how about > 'SMark'/'SBench' instead of 'SUnit'? > > SBench sounds good for me. > Best regards > Stefan > > > PS: > > One thing, we might want to use as a source of inspiration in the future > is: > http://code.google.com/p/caliper/ > > Thats a microbenchmark framework for Java, also using the unit-testing > metaphor. > There @Param is also neat ( > http://code.google.com/p/caliper/source/browse/trunk/examples/src/main/java/examples/ArraySortBenchmark.java) > and allows to encode input sizes for which the benchmarks should be > executed. > That is something I am not to interested in, but in case the framework > finds adoption that might be something to keep in mind. > > -- > Stefan Marr > Software Languages Lab > Vrije Universiteit Brussel > Pleinlaan 2 / B-1050 Brussels / Belgium > http://soft.vub.ac.be/~smarr > Phone: +32 2 629 2974 > Fax: +32 2 629 3525 > > >
