> Yes, at the time I ran three different benchmarks over the non-opencl version 
> because I was trying to compare three different versions of the ANSI C code. 
> This was, the ANSI C code currently in the trunk, the code in the trunk with 
> the patch #473 applied (patch from Vasco that removed the gotos from the 
> bool_eval(), and the ANSI C code in the opencl branch, which uses a boolean 
> tree in postfix notation. But perhaps this is not the best approach to 
> compare the performance. Well, at least I couldn’t really understand which 
> number in the benchmark log I should look at to compare. Is is the “Total 
> testing time elapsed” or the “VGR” metric? The “VGR” number didn’t make much 
> sense, because the version that got higher VGR is the code in which the ‘rt’ 
> command takes more time to run.

Ah, that explains things.  All three results were within volatility tolerance, 
effectively identical timings (within 1% deviation).  More importantly, they 
are “RIGHT” aka valid results, which speaks well to the refactoring cleanup.  
Performance may be in the noise, but correctness means it could be committed.

The benchmark is actually just right for understanding the real impact of these 
types of changes.  It’s a validation and timing framework.  As for the timings, 
just pay attention to the summary VGR metric (e.g., 32189).  All the number 
tell different things, but that’s a linear one relevant to tracking performance.

If you made a change and VGR goes from 32k to 35k, that means you sped things 
up about 10%.  It’s a simple linear metric.

The “total testing time” is irrelevant for what you’re doing.  That is 
measuring convergence rate, which is intrinsically volatile.  All that is 
telling is how long it took to see stable performance numbers to within a +-1% 
confidence interval (DEVIATION=1), which is going to take a little while given 
the TIMEFRAME=60 is also telling it to make sure there’s at least 1 minute of 
ray tracing work per frame.  In general, it’s going to need to run 2-5 frames 
before the performance is stable. 

Related to these specific changes, these numbers pass a sanity check.  If you 
ran a profile (e.g., perf), you’d see that only a fraction of time is spent in 
boolean code (10-30% of time, depending on the model).  The other 2/3rds are 
roughly in traversal and intersection code.  It would be interesting to have 
isolated timings of just the boolean evaluation, but we’d still want to run the 
benchmark to see what the big picture impact is (and verify we’re still getting 
valid results).


Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
BRL-CAD Developer mailing list

Reply via email to