On Thu, Jun 4, 2009 at 11:16 AM, Daniel Carrera <daniel.carr...@theingots.org> wrote: > > The point of the benchmark is not "oh look, it's slower than Perl 5". The > benchmarks are good for testing a specific aspect of the language, so it is > easier to isolate *where* the problem is. This is harder on a real > application.
I think you are confusing profiling with benchmarking. Profiling helps you identify where a problem is. Benchmarking helps you compare two different versions of the same routine. I intend to do use said benchmark as a basic profiling tool as well. If this program performs far outside the expected norm then somewhere inside this program is something that needs to be optimized. Of course, doing that requires additional work by people far smarter than myself but at least I will be able to point them to something to look at. > > Benchmarks help you see where the biggest gains are to be found. No, that is still profiling. > Additionally, they help measure improvement. When Patrick fixed the bug with Measuring (improvement or otherwise) is their primary function. > And your expectation would be wrong. In the real world some things are > faster than others. Remember the 80/20 rule. Benchmarks help you locate the > 20% of the code that needs the most attention. No, that's still profiling. On average the benchmark should even out because we are not looking for bottlenecks but the overall performance of the program. If it doesn't perform on average then there is a problem. Only true profiling will find it but if I can show 30 programs where Rakudo consistently performs 3 to 5 times slower than perl 5 and one program that is 30 times slower - which one do you think @Patrick will be interested in? > > As of this moment I have four benchmarks written, and each benchmark gives > wildly different factors. These are very specialized benchmarks designed to highlight specific language features. The approach I am looking at looks at a wider cross section of the language where you would expect things to even out. > > If you write such a tool, I'd like to try it. Perhaps. Time and ambition will tell. > > Daniel. > Cheers, Joshua Gatcomb a.k.a. Limbic~Region
