On 03/13/2018 07:22 AM, Linas Vepstas wrote:
>> So, the first thing you should do is build a good benchmark tool, then we, >> you and the rest of opencog community, can supply it with a collection of
 >> critical tests.
 > How do you see such tool?

It could perhaps use cxxtest. On the other hand there probably no need to touch C++ at all. Maybe a combination of Scheme and bash scripts, or which ever way you're comfortable with. I suppose it could automatically append the results into a diary, that could perhaps be turned into a plot (which poses the problem of making sure the system is the same).

Maybe it should have some functionality to repeat the benchmark a number of times and gather some stats (mean and std dev), as there tends to be a lot of variance.

Relatedly I wrote this simplistic benchmark tool that merely reruns a number of times the unit tests and gather stats


the problem is that unit tests are not meant to be representative benchmarks. Also benchmarks can be afforded to take more time to run as they don't need to run as often as unit tests. Personally I see no problem having a benchmark suite taking hours. They might also require larger data sets. For these reasons it makes sense to have the benchmark tool live in its own repository. I suppose this repository could be generally dedicated to OpenCog's benchmarks, or it could be for the pattern matcher alone. I don't know.


You received this message because you are subscribed to the Google Groups 
"opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to opencog+unsubscr...@googlegroups.com.
To post to this group, send email to opencog@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit 
For more options, visit https://groups.google.com/d/optout.

Reply via email to