On Thu, Jun 4, 2009 at 11:16 AM, Daniel Carrera
<daniel.carr...@theingots.org> wrote:
> The point of the benchmark is not "oh look, it's slower than Perl 5". The
> benchmarks are good for testing a specific aspect of the language, so it is
> easier to isolate *where* the problem is. This is harder on a real
> application.

I think you are confusing profiling with benchmarking.  Profiling
helps you identify where a problem is.  Benchmarking helps you compare
two different versions of the same routine.  I intend to do use said
benchmark as a basic profiling tool as well.  If this program performs
far outside the expected norm then somewhere inside this program is
something that needs to be optimized.  Of course, doing that requires
additional work by people far smarter than myself but at least I will
be able to point them to something to look at.

> Benchmarks help you see where the biggest gains are to be found.

No, that is still profiling.

> Additionally, they help measure improvement. When Patrick fixed the bug with

Measuring (improvement or otherwise) is their primary function.

> And your expectation would be wrong. In the real world some things are
> faster than others. Remember the 80/20 rule. Benchmarks help you locate the
> 20% of the code that needs the most attention.

No, that's still profiling.  On average the benchmark should even out
because we are not looking for bottlenecks but the overall performance
of the program.  If it doesn't perform on average then there is a
problem.  Only true profiling will find it but if I can show 30
programs where Rakudo consistently performs 3 to 5 times slower than
perl 5 and one program that is 30 times slower - which one do you
think @Patrick will be interested in?

> As of this moment I have four benchmarks written, and each benchmark gives
> wildly different factors.

These are very specialized benchmarks designed to highlight specific
language features.  The approach I am looking at looks at a wider
cross section of the language where you would expect things to even

> If you write such a tool, I'd like to try it.

Perhaps.  Time and ambition will tell.

> Daniel.

Joshua Gatcomb
a.k.a. Limbic~Region

Reply via email to