Hi Buddy, * Buddy Burden <[EMAIL PROTECTED]> [2008-03-31 00:05]: > Well, do you agree with this assessment: > > Having a plan stated as an exact number of tests to be run is a > solution to 2 problems. The first is that a test harness must > be able to tell when not enough tests have been run. The second > is that a test harness must be able to tell when too many tests > have been run. Using a success flag at the end of a test script > as opposed to a numeric plan solves, equally well, one of those > two problems. Since it does not solve the trickier of the two, > it is not suitable as a general-purpose solution for all > testing. However, since it does solve the more common of the > two (the far more common, one might argue), its advantages make > it useful, especially for simple test scripts.
I was not commenting on the utility of an “all done” flag – merely on your astonishment that people would consider it best practice to do something that required making lock-step changes in multiple places of the code. Having an “all done” flag is definitely useful and I’m not arguing against it. Note that it doesn’t quite protect you from running too few tests either. You may botch some conditional in your test program and end up skipping tests silently, in which case you will still reach the `all_done()` line, and it’ll look as if all was fine. What it protects you from is dying half-way through the tests without the harness noticing. Of course, that’s by far the most common failure mode. > For my money, the loss of sense of security you get from the > possibility that you might run across one of those uncommon > cases and not notice it is about balanced out by the greater > likelihood of creating more tests and convincing more > programmers to use tests because tests are easier to create and > maintain. In fact, I'm starting to sort of view it as a > graduated thing: deferred plan all the time, 'cause it's easy, > and when you become more seasoned, you naturally gravitate > towards numeric plans for their increased sense of security. > And/or, it may make sense to use deferred plans during > development, but switch to numeric plans before releasing > anything to CPAN. Yes in all points. > For myself, I think chromatic's warning is strongly making me > lean towards the idea that numeric plans are far more useful > than I initially gave them credit for, and I personally will > probably switch over to numeric plans--at least before I > release anything to CPAN, even if I keep using deferred plans > for initial development. But as my primary goal right now is to > get a team of 3-7 programmers who've been writing tests only > haphazardly at best for the past 5-10 years to start writing > tests like crazy, I think having deferred plans to KISS is > really the way to go. Absolutely – getting them to write tests without plans is much better than writing no tests at all. I’m not dogmatic; even a little improvement is still improvement. Once they write a test and it catches ahead of time a serious bug that would have been a pain to find otherwise, they’ll come around all by themselves, anyway. :-) (Cf. <http://c2.com/cgi/wiki?TestInfected>). >> Overall, I'm reminded of something MJD said in his Program >> Repair Shop and Red Flags talk at YAPC::Asia 2007: >> >> Also I'm trying to get away from right and wrong. This is >> not about morals; it's engineering. And so, well, things >> can be suboptimal – maybe they aren't designed as well as >> they could be –, but they're not wrong, probably. > > True, I probably overemphasized my frustration on that point. > Obviously there _can_ be good reasons to violate the principle, > and I figured there probably were in this case. I just wasn't > seeing them, then. But now I do. :-) Heh, no problem. Note that, as I said, I mean this WRT testing and plans as well: just because you have no plan doesn’t mean your testing is “wrong.” You just have room to improve. :-) Regards, -- Aristotle Pagaltzis // <http://plasmasturm.org/>
