23 юни 2012, събота, 19:32:36 UTC+3, Joachim Durchholz написа: > > Am 23.06.2012 17:38, schrieb Aleksandar Makelov: > > We want to make sure that the right thing is done with the output from > the > > RNGs, so we manually supply as an additional argument to a given > function > > some particular choice for all the variables inside the function that > come > > from RNGs. > > Ah, I see. > I'm not convinced that it's the best way to design such a thing. Adding > parameters to a function that are purely there for testing purposes is > going to confuse people who aren't into testing. It's also in > contradiction to the "keep interfaces as narrow as possible" principle - > a narrow interface means less things that need to be remembered by > programmers, less things that need to be set up by the caller, less > things that might be misinterpreted. > Yeah, I realized that earlier (it's in one of my posts from above): adding code to the body of the function that is purely intended for testing is ugly from my perspective, too. But on the other hand we need some way of testing these algorithms, don't we?
> Also, it'd adding code to the functions. Which means adding bugs - which > may affect the function if it's running in production. Which kind of > defeats the purpose of testing in the first place. > > The addition of code intended for the user is quite minimal. If you stick with not setting the value of the test parameter, the only difference the flow of execution sees is an additional if statement that checks whether the parameter is there, and then goes back to the logic of the function. > > The reason that we use certain precomputed values is that doing > > the test with some randomly generated set of values as an additional > > argument is essentially going to have to repeat the calculations in the > > function itself (which we want to test) - whereas for concrete values we > > know the answer right away. Does that make sense? > > Not very much, I fear. > As Stefan said, repeating a calculation in test code isn't a useful unit > test, even if you place the unit test into another module. Or if you're > doing the calculation by hand - unless those calculations have been done > by experts in the field and verified by other experts in the field, of > course. > Yes, I think my situation is closer to "doing it by hand" - and even if I'm nowhere near an expert in the field, the calculations haven't required much ingenuity so far :) > > Expanding on Stefan's example. > Assuming you're testing an array-inversion routine. > > We agree on the worst approach to test it: repeat the array inversion > algorithm in the test and see whether it gives the same result as the > code in SymPy. > Actually this kind of test isn't entirely pointless - if the test code > remains stable but the SymPy code evolves into optimizations, this could > serve a useful purpose. On the other hand, you still don't write this > kind of test code until you actually do the optimization. > > The other approach would be to add an "expected result" parameter, and > fail if the result isn't the expected one. > Yeah, that makes sense, but the algorithms I had to test don't return the same answer every time - for example, consider an algorithm that returns a random group element - there is no way to test the result apart from asserting that it belongs to the group, and this test can be cheated rather easily (for example, always return the identity). > This has two problems: > a) It adds an unwanted dependency to the testing modules. At least if > you want to give better diagnostics than just throwing an exception (for > example, you may want to test internal workings that throw exceptions > which get caught). > b) You're supplying precomputed results. You'd still need to explain why > the results are correct. Somebody has to verify that they are, indeed, > correct. > I agree on that, but don't you do that all the time with tests for deterministic algorithms as well? The majority of tests in sympy rely partly on some outside - and not defining - knowledge (for example, the symmetric group on 5 elements having order 120), or on an example calculated by hand; is the validity of these claims explained somewhere (maybe it should be included as comments in the test files)? > > My approach for that would be to test the defining property of the > function: > (matrix_inv(A) * A).is_unit_matrix() > (sorry for ad-hoc invention of matrix functions) > > I.e. you're testing the purpose of the function, not its inner workings. > > Yes, this is obviously a good way to test any algorithm, however in many cases the answer returned by a randomized algorithm (and, not too rarely, by a deterministic algorithm) has no clear defining property (see above, when I talk about random group elements). Oh, and this kind of testing can uncover more bugs. > For example, the above reasoning overlooks that not all matrices can be > inverted. If I'm testing the algorithm, I'll simply overlook the case of > a singular matrix because I'm all thinking inside the algorithm. > If I write my test code with the purpose in mind, I have a better chance > to stumble over the singular case - either because I'm thinking about > matrix theory instead of my algorithm, or because some tests > mysteriously fail, namely, when the RNG happens to generate a singular > matrix. > > I hope that's all understandable. > And I hope I'm not missing the point entirely :-) > -- You received this message because you are subscribed to the Google Groups "sympy" group. To view this discussion on the web visit https://groups.google.com/d/msg/sympy/-/DA6cILR9bvYJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
