Re: Testing in the D Standard Library
On Monday, 23 January 2017 at 01:52:29 UTC, Chris Wright wrote: On Sun, 22 Jan 2017 20:18:11 +, Mark wrote: Have you considered adding randomized tests to Phobos? Randomized testing is an interesting strategy to use alongside deterministic testing. It produces more coverage over time. However, any given test run only has a fraction of the coverage that you see over a large number of runs. In other words, if the randomized tests catch something, you don't know who dun it. This is generally considered a bad thing. Phobos does have some tests that use a PRNG. They all use a fixed seed. This is a shortcut to coming up with arbitrary test data; it's not a way to increase overall coverage. I think the right way to do it is to have a nightly randomized test build, but since I'm not willing to do the work, I don't have much say. This. So much this. Unit tests that loop over many randomly generated input test vectors are just a waste of everybody's CPU time. Don't get me wrong: fuzzing is also necessary. But it relies on an arbitrary time limit, which is hardly compatible with keeping a test suite fast. Which means it should be done in a another validation process than unit tests.
Re: Testing in the D Standard Library
On 2017-01-21 03:59, Jack Stouffer wrote: But 2.073 isn't released yet. It's now ;) -- /Jacob Carlborg
Re: Testing in the D Standard Library
On Sun, 22 Jan 2017 20:18:11 +, Mark wrote: > Have you considered adding randomized tests to Phobos? Randomized testing is an interesting strategy to use alongside deterministic testing. It produces more coverage over time. However, any given test run only has a fraction of the coverage that you see over a large number of runs. In other words, if the randomized tests catch something, you don't know who dun it. This is generally considered a bad thing. Phobos does have some tests that use a PRNG. They all use a fixed seed. This is a shortcut to coming up with arbitrary test data; it's not a way to increase overall coverage. I think the right way to do it is to have a nightly randomized test build, but since I'm not willing to do the work, I don't have much say.
Re: Testing in the D Standard Library
On Friday, 20 January 2017 at 13:35:40 UTC, Mike Parker wrote: Jack Stouffer details how unit testing, code review, and code coverage are handled in the development and maintenance of Phobos. Thanks, Jack! Blog: https://dlang.org/blog/2017/01/20/testing-in-the-d-standard-library/ Reddit: https://www.reddit.com/r/programming/comments/5p3vlq/testing_in_the_d_standard_library/ Very informative! Have you considered adding randomized tests to Phobos? For instance, for the sum() example, you could generate a random array x, say, 100 times and assert: sum(x) == x[0] + sum(x[1..$]); which is pretty much the defining property of the summation function (along with sum(a)==a[0] for an array of length 1).
Re: Testing in the D Standard Library
On 2017-01-21 03:59, Jack Stouffer wrote: But 2.073 isn't released yet. It's in release candidate. The current output is pretty embarrassing. -- /Jacob Carlborg
Re: Testing in the D Standard Library
On Friday, 20 January 2017 at 16:50:22 UTC, Jacob Carlborg wrote: Could you please create a new image out of the DMD Ddoc output using 2.073.0 instead. It has a completely new default Ddoc theme. But 2.073 isn't released yet.
Re: Testing in the D Standard Library
On 2017-01-20 14:35, Mike Parker wrote: Jack Stouffer details how unit testing, code review, and code coverage are handled in the development and maintenance of Phobos. Thanks, Jack! Blog: https://dlang.org/blog/2017/01/20/testing-in-the-d-standard-library/ Could you please create a new image out of the DMD Ddoc output using 2.073.0 instead. It has a completely new default Ddoc theme. -- /Jacob Carlborg
Re: Testing in the D Standard Library
On Friday, 20 January 2017 at 13:35:40 UTC, Mike Parker wrote: Jack Stouffer details how unit testing, code review, and code coverage are handled in the development and maintenance of Phobos. Thanks, Jack! Blog: https://dlang.org/blog/2017/01/20/testing-in-the-d-standard-library/ Reddit: https://www.reddit.com/r/programming/comments/5p3vlq/testing_in_the_d_standard_library/ No comments on Reddit? I guess everybody is busy with the Trump inauguration.
Re: Testing in the D Standard Library
On Friday, 20 January 2017 at 13:39:10 UTC, Jack Stouffer wrote: Thanks for posting this! Also, there is a typo in the list after the first paragraph. There should be an item which says "A style checker". Sorry about that. A victim of my formatting. Fixed.
Re: Testing in the D Standard Library
On Friday, 20 January 2017 at 13:35:40 UTC, Mike Parker wrote: Jack Stouffer details how unit testing, code review, and code coverage are handled in the development and maintenance of Phobos. Thanks, Jack! Blog: https://dlang.org/blog/2017/01/20/testing-in-the-d-standard-library/ Reddit: https://www.reddit.com/r/programming/comments/5p3vlq/testing_in_the_d_standard_library/ Thanks for posting this! Also, there is a typo in the list after the first paragraph. There should be an item which says "A style checker".
Testing in the D Standard Library
Jack Stouffer details how unit testing, code review, and code coverage are handled in the development and maintenance of Phobos. Thanks, Jack! Blog: https://dlang.org/blog/2017/01/20/testing-in-the-d-standard-library/ Reddit: https://www.reddit.com/r/programming/comments/5p3vlq/testing_in_the_d_standard_library/
