On 03/13/2018 07:22 AM, Linas Vepstas wrote:
>> So, the first thing you should do is build a good benchmark tool,
>> you and the rest of opencog community, can supply it with a
>> critical tests.
> How do you see such tool?
It could perhaps use cxxtest. On the other hand there probably no need
to touch C++ at all. Maybe a combination of Scheme and bash scripts, or
which ever way you're comfortable with. I suppose it could automatically
append the results into a diary, that could perhaps be turned into a
plot (which poses the problem of making sure the system is the same).
Maybe it should have some functionality to repeat the benchmark a number
of times and gather some stats (mean and std dev), as there tends to be
a lot of variance.
Relatedly I wrote this simplistic benchmark tool that merely reruns a
number of times the unit tests and gather stats
the problem is that unit tests are not meant to be representative
benchmarks. Also benchmarks can be afforded to take more time to run as
they don't need to run as often as unit tests. Personally I see no
problem having a benchmark suite taking hours. They might also require
larger data sets. For these reasons it makes sense to have the benchmark
tool live in its own repository. I suppose this repository could be
generally dedicated to OpenCog's benchmarks, or it could be for the
pattern matcher alone. I don't know.
You received this message because you are subscribed to the Google Groups
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send email to firstname.lastname@example.org.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit
For more options, visit https://groups.google.com/d/optout.