On Wednesday, September 3, 2014 2:07:45 PM UTC-5, Sergey Kirpichev wrote:
>
> On Wednesday, September 3, 2014 6:44:06 AM UTC+4, Aaron Meurer wrote:
>>
>> I think such situations are OK, as long as you understand why the test
>> failed before and verified that the new test really tests the same
>> thing.
>>
>
> 1. We can make mistakes here.
>
I think that might apply to replacing things in current master, but if you
go back to the point in history where the fix was not made and have another
example that fails in the same way, there is no reason to include both
failing tests just as one would not try to find all possible failing cases
and add them.
> 2. Probably, we want to be sure, that we never break again the
> fixed example (incl. by different reason).
>
This is what I would refer to as serendipitous coverage. If, today, we
rewrote sympy from scratch, the existing tests would identify errors but
would not necessarily be important to keep. As I see it, there are
different types of tests that:
- safeguard against conceptual errors (e.g. var('x', complex=True,
real=False).is_imaginary is None)
- provide simple coverage, verifying that code is free of conceptual errors
- test functionality that should be present (like keyword handling)
The hard part about writing a good test suite is thinking of all the
significantly different ways that a code path can be traversed. e.g. if a
number is being handled, what aspects of "number" are important to test,
e.g. positive, negative, imaginary, etc... As I have recently found,
sometimes a complex number itself is not enough, you have to think about
the arg of the number.
--
You received this message because you are subscribed to the Google Groups
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sympy.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sympy/c4f98b19-5472-45a3-9b3d-4e3ac77bb7d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
