Thanks! I made some style changes in c739839 and ec65745 and changed the
random number generators to use the mlpack random number generator object in
1ef2a02 (some of the tests I changed there were not yours, but still this is an
improvement because you can now set the random number seed with
`mlpack::math::RandomSeed()`).
Then, for each test, I added a line at the top:
`math::RandomSeed(std::time(NULL))` and recompiled. I ran a bash script like
this:
```
while(true); do bin/mlpack_test -t GammaDistributionTrainWithProbabilitiesTest;
sleep 1; done
```
This will give output like this:
```
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
Running 1 test case...
/home/ryan/src/mlpack/src/mlpack/tests/distribution_test.cpp(483): fatal error
in "GammaDistributionTrainWithProbabilitiesTest": difference{2.03727%} between
betaReal{6.7000000000000002} and gDist.Beta(0){6.5662282671947327} exceeds 2%
*** 1 failure detected in test suite "mlpackTest"
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
Running 1 test case...
*** No errors detected
```
Using this strategy I found that some of the tolerances needed to be modified
to prevent failures, in the test `GammaDistributionTrainWithProbabilitiesTest`:
the comparison between `gDist` and `gDist2` parameters occasionally failed, so
I loosened the tolerance to 1.5% from 1%. The comparisons between `alphaReal`
and `betaReal` and the trained distributions often failed, so I loosened the
tolerance from 1% to 2.5%. I think this is an ok change to make and I think
these are reasonable bounds for the expected noise.
This strategy is important to test with, because each compilation will
(effectively) have a different random seed associated with it, so to ensure
that tests pass on users' systems, we have to check with many different random
seeds.
Anyway, about the Perceptron code, definitely, if you would like to add tests,
it would be much appreciated! You can create an issue if you like or just open
a PR, whatever you prefer is fine.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/pull/834#issuecomment-269350578_______________________________________________
mlpack mailing list
[email protected]
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
