Hi Terri, I think lots of people in the scientific Python community write their own algorithms and test them. But it's hard to give generic advice here, I think, because it's so dependent on your algorithm.
Here's my try / our approach that has worked well for us over the last decade or so. * first, write automated "smoke" tests that check to see if your code is basically running/working. They should be as dumb and robust as possible. (e.g. the equivalent of "check that 2+2 = 4"). These are by far the most important in my experience, in that they deliver the most value for the least effort. * set up CI on those tests. * check code coverage of your code base, and try to get it to 30-40% by testing the basic code paths. * write a series of basic tests for edge cases (divide by zero, boundary conditions, that kind of thing), trying to cover another 10-20%. * as your code base matures and complexifies, write tests for new functionality and try to cover old functionality as well. Here code coverage is your friend in terms of targeting effort. * whenever you discover a bug, write a test against that bug before fixing it. That way your most error prone bits will get more coverage adaptively. I call this "stupidity driven testing." Lather, rinse, repeat. tl; dr? smoke tests, code coverage analysis, test against buggy code. best, --titus On Mon, Jul 17, 2017 at 11:50:59AM -0400, Terri Yu wrote: > Thanks everyone, those are interesting resources for testing in general. > > I'm using Python's unittest framework and everything is already set up. > The specific problem I need help with is what tests to write, in order to > test numerical floating point output from algorithms. Given the responses > I've gotten, it seems like not many people write their own algorithms > and/or test them. > > Terri > > On Sun, Jul 16, 2017 at 5:50 PM, Jeremy Gray <[email protected]> wrote: > > > Hi Terri, > > > > > > It might also be worth checking out the workshop from this years pycon > > from Eria ma: > > Best Testing Practices for Data Science, on yotube here - > > https://www.youtube.com/watch?v=yACtdj1_IxE > > > > The github repo is here: > > https://github.com/ericmjl/data-testing-tutorial > > > > Cheers, > > Jeremy > > > > On Fri, Jul 14, 2017 at 5:21 PM, Olav Vahtras <[email protected]> > > wrote: > > > >> Dear Terri > >> > >> In addition I can recommend the following resource: > >> > >> pythontesting.net has a podcast series on testing and more, check out > >> the new book on pytest by the site maintainer Brian Okken > >> > >> Regards > >> Olav > >> > >> > >> > >> Olav > >> > 14 juli 2017 kl. 21:36 skrev Ashwin Srinath <[email protected]>: > >> > > >> > If you're using Python, numpy.testing has the tools you'll need: > >> > > >> > https://docs.scipy.org/doc/numpy/reference/routines.testing.html > >> > > >> > There's also pandas.testing for testing code that uses Pandas. > >> > > >> > Thanks, > >> > Ashwin > >> > > >> >> On Fri, Jul 14, 2017 at 3:27 PM, Terri Yu <[email protected]> wrote: > >> >> Hi everyone, > >> >> > >> >> Are there any resources that explain how to write unit tests for > >> scientific > >> >> software? I'm writing some software that processes audio signals and > >> there > >> >> are many parameters. I'm wondering what's the best way to test > >> floating > >> >> point numeric results. > >> >> > >> >> Do I need to test every single parameter? How can I verify accuracy of > >> >> numeric results... use a different language / library? I would like > >> to do a > >> >> good job of testing, but I also don't want to write a bunch of > >> semi-useless > >> >> tests that take a long time to run. > >> >> > >> >> I would appreciate any thoughts you have. > >> >> > >> >> Thank you, > >> >> > >> >> Terri > >> >> > >> >> _______________________________________________ > >> >> Discuss mailing list > >> >> [email protected] > >> >> http://lists.software-carpentry.org/listinfo/discuss > >> > _______________________________________________ > >> > Discuss mailing list > >> > [email protected] > >> > http://lists.software-carpentry.org/listinfo/discuss > >> _______________________________________________ > >> Discuss mailing list > >> [email protected] > >> http://lists.software-carpentry.org/listinfo/discuss > >> > > > > > _______________________________________________ > Discuss mailing list > [email protected] > http://lists.software-carpentry.org/listinfo/discuss -- C. Titus Brown, [email protected] _______________________________________________ Discuss mailing list [email protected] http://lists.software-carpentry.org/listinfo/discuss
