On 03/13/2018 07:22 AM, Linas Vepstas wrote:
>> So, the first thing you should do is build a good benchmark tool,
then we,
>> you and the rest of opencog community, can supply it with a
collection of
>> critical tests.
>
> How do you see such tool?
It could perhaps use cxxtest. On the other hand there probably no need
to touch C++ at all. Maybe a combination of Scheme and bash scripts, or
which ever way you're comfortable with. I suppose it could automatically
append the results into a diary, that could perhaps be turned into a
plot (which poses the problem of making sure the system is the same).
Maybe it should have some functionality to repeat the benchmark a number
of times and gather some stats (mean and std dev), as there tends to be
a lot of variance.
Relatedly I wrote this simplistic benchmark tool that merely reruns a
number of times the unit tests and gather stats
https://github.com/opencog/atomspace/blob/master/scripts/benchmark_utests.sh
the problem is that unit tests are not meant to be representative
benchmarks. Also benchmarks can be afforded to take more time to run as
they don't need to run as often as unit tests. Personally I see no
problem having a benchmark suite taking hours. They might also require
larger data sets. For these reasons it makes sense to have the benchmark
tool live in its own repository. I suppose this repository could be
generally dedicated to OpenCog's benchmarks, or it could be for the
pattern matcher alone. I don't know.
Nil
--
You received this message because you are subscribed to the Google Groups
"opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to opencog+unsubscr...@googlegroups.com.
To post to this group, send email to opencog@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit
https://groups.google.com/d/msgid/opencog/e9e08e4a-8f23-0af5-ad58-8c3f28ea7454%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
