On 05 Apr 2011, at 15:11, Igor Stasenko wrote: > I don't care :) *sigh* then just flip a coin and you have the same amount of insight with less effort.
It is always astonishing how few people know about the basics when it comes to empirical experiments. I disagree with Camillo's demand for 100x repetition, but it should be definitely run at least a few times. And then you have to look at the results and see how they vary. Otherwise you don't know anything and are just wasting your own time. I will try at the next sprint to make some progress on the framework front. Perhaps, someone could trow together something like the unit-test runner to make at least the basics fool-proof. And at the same time the message to measure the execution time of a block should become deprecated or raise a warning that you are about to fool yourself... It does not have to be statistically rigorous for small experiments, but at the very least you have to convince yourself that your measurements are actually of any value. The computer in front of you is just to complex to make an educated guess. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
