[Antoinw Pitrou] > I think utopia is the word here. Fuzz testing can be useful, but it's > not a replacement for manual testing of carefully selected values.
First, they aren't mutually exclusive. It's trivial to add manually selected cases to a hypothesis test. Second, from my experience; people rarely choose between carefully selected optimal values and Fuzz testing, they usually choose between manually selected trivial values or no test at all. Thirdly, Computers are very good at exhaustively searching multidimensional spaces. If your tool sucks so bad at that that a human can do it better, then your tool needs work. Improving the tool saves way more time than reverting to manual testing. There was a post long ago (I think I read it on Digg.com for some indication) about how to run a cloud-based system correctly. One of the controversial practice the article advocated was disabling ssh on the machine instances. The rational is that you never want to waste your time fiddling with an instance that's not behaving properly. In cloud-systems, instances should not be special. If they fail, blow them away and bring up another. If the failure persists, it's a problem with the *system* not the instance. If you care about individual instances YOU'RE DOING IT WRONG. You need to re-design the system. On Wed, Nov 28, 2018 at 8:19 AM Antoine Pitrou <solip...@pitrou.net> wrote: > On Tue, 27 Nov 2018 22:47:06 -0600 > Abe Dillon <abedil...@gmail.com> wrote: > > > > If we could figure out a cleaner syntax for defining invariants, > > preconditions, and postconditions we'd be half-way to automated testing > > UTOPIA! (ok, maybe I'm being a little over-zealous) > > I think utopia is the word here. Fuzz testing can be useful, but it's > not a replacement for manual testing of carefully selected values. > > Also, the idea that fuzz testing will automatically find edge cases in > your code is idealistic. It depends on the algorithm you've > implemented and the distribution of values chosen by the tester. > Showcasing trivially wrong examples (such as an addition function that > always returns 0, or a tail function that doesn't return the tail) > isn't very helpful for a real-world analysis, IMHO. > > In the end, you have to be rigorous when writing tests, and for most > non-trivial functions it requires that you devise the distribution of > input values depending on the implemented algorithm, not leave that > distribution to a third-party library that knows nothing about your > program. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
