> Yes, at the time I ran three different benchmarks over the non-opencl version
> because I was trying to compare three different versions of the ANSI C code.
> This was, the ANSI C code currently in the trunk, the code in the trunk with
> the patch #473 applied (patch from Vasco that removed the gotos from the
> bool_eval(), and the ANSI C code in the opencl branch, which uses a boolean
> tree in postfix notation. But perhaps this is not the best approach to
> compare the performance. Well, at least I couldn’t really understand which
> number in the benchmark log I should look at to compare. Is is the “Total
> testing time elapsed” or the “VGR” metric? The “VGR” number didn’t make much
> sense, because the version that got higher VGR is the code in which the ‘rt’
> command takes more time to run.
Ah, that explains things. All three results were within volatility tolerance,
effectively identical timings (within 1% deviation). More importantly, they
are “RIGHT” aka valid results, which speaks well to the refactoring cleanup.
Performance may be in the noise, but correctness means it could be committed.
The benchmark is actually just right for understanding the real impact of these
types of changes. It’s a validation and timing framework. As for the timings,
just pay attention to the summary VGR metric (e.g., 32189). All the number
tell different things, but that’s a linear one relevant to tracking performance.
If you made a change and VGR goes from 32k to 35k, that means you sped things
up about 10%. It’s a simple linear metric.
The “total testing time” is irrelevant for what you’re doing. That is
measuring convergence rate, which is intrinsically volatile. All that is
telling is how long it took to see stable performance numbers to within a +-1%
confidence interval (DEVIATION=1), which is going to take a little while given
the TIMEFRAME=60 is also telling it to make sure there’s at least 1 minute of
ray tracing work per frame. In general, it’s going to need to run 2-5 frames
before the performance is stable.
Related to these specific changes, these numbers pass a sanity check. If you
ran a profile (e.g., perf), you’d see that only a fraction of time is spent in
boolean code (10-30% of time, depending on the model). The other 2/3rds are
roughly in traversal and intersection code. It would be interesting to have
isolated timings of just the boolean evaluation, but we’d still want to run the
benchmark to see what the big picture impact is (and verify we’re still getting
valid results).
Cheers!
Sean
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
BRL-CAD Developer mailing list
brlcad-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/brlcad-devel
