Yes, that makes sense, but just to be clear, using the same seed does *not*
imply that the algorithm should produce “equivalent” results by some definition
of equivalent if you change the input data. For example, in SGD, the random
seed might be used to select the next minibatch of examples, but
Behaviors at this level of detail, across different ML implementations, are
highly unlikely to ever align exactly. Statistically small changes in
logic, such as "<" versus "<=", or differences in random number generators,
etc, (to say nothing of different implementation languages) will accumulate
o
Dear Matei,
thanks for the feedback!
I used the setSeed option for all randomized classifiers and always used
the same seeds for training with the hope that this deals with the
non-determinism. I did not run any significance tests, because I was
considering this from a functional perspective,
Hi Steffen,
Thanks for sharing your results about MLlib — this sounds like a useful tool.
However, I wanted to point out that some of the results may be expected for
certain machine learning algorithms, so it might be good to design those tests
with that in mind. For example:
> - The classific
Certainly if your tests have found a problem, open a JIRA and/or pull
request with the fix and relevant tests.
More tests generally can't hurt, though I guess we should maybe have a look
at them first. If they're a lot of boilerplate and covering basic functions
already covered by other tests, the
