This this whole discussion has unhinged a bit from reality, maybe you can give
some concrete examples of the problems you're talking about? You obviously
have some specific breakdowns in mind.
Fergal Daly wrote:
>> Modules do not have a binary state of working or not working. They're
>> composed of piles of (often too many) features. Code can be shippable
>> without
>> every single thing working.
>
> You're right, I was being binary, but you were being unary. There are 3 cases,
>
> 1 the breakage was not so important, so you don't bail no matter what
> version you find.
> 2 it's fuzzy, maybe it's OK to use Foo version X but once Foo version
> X+1 has been released you want to force people to use it
> 3 the breakage is serious, you always want to bail if you find Foo
> version X (and so you definitely don't switch the tests to TODO).
>
> You claimed 2 is always the case. I claimed that 1 and 3 occur.
If I did, that wasn't my intent. I only talked about #2 because it's the only
one that results in the user seeing passing TODO tests, which is what we were
talking about.
> I'm
> happy to say admit that 2 can also occur. The point remains, you would
> not necessarily change your modules requirements as a reaction to X+1
> being released. You might, or you might change it beforehand if it
> really matters or you might not change it at all.
And I might dip my head in whipped cream and go give a random stranger a foot
bath. You seem to have covered all possibilities, good and bad. I'm not sure
to what end.
The final choice, incrementing the dependency version to one that does not yet
exist, boils down to "it won't work". It's also ill advised to anticipate
that version X+1 will fix a given bug as on more than one occasion an
anticipated bug has not been fixed in the next version.
Anyhow, to get back to the point, it boils down to an author's decision how to
deal with a known bug. TODO tests are orthogonal.
>> Maybe we're arguing two different situations. Yours seems to be when there
>> is
>> a broken version of a dependency, but a known working version exists. In
>> this
>> case, you're right, it's better resolved with a rich dependency system.
>
> I think maybe we are.
>
> You're talking about where someone writes a TODO for a feature that
> has never worked. That's legit, although I still think there's
> something odd about it as you personally have nothing "to do". I agree
> it's not dangerous.
Sure you do, you have to watch for when the dependency fixes its bug. But
that's boring and rote, what computers are for! So you write a TODO test to
automate the process. [1]
In a large project, sometimes things get implemented when you implement other
things. This is generally more applicable to bugs, but sometimes to minor
features.
Then there are folks who embrace the whole test first thing and write out lots
and lots of tests beforehand. Maybe you decide not to implement them all
before shipping. Rather than delete or comment out those tests, just wrap
them in TODO blocks. Then you don't have to do any fiddling with the tests
before and after release, something which leads to an annoying shear between
the code the author uses and the code users use.
There is also the "I don't think feature X works in Y environment" problem.
For example, say you have something that depends on symlinks. You could hard
code in your test to skip if on Windows or some such, but that's often too
broad. Maybe they'll add them in a later version, or with a different
filesystem (it's happened on VMS) or with some fancy 3rd party hack. It's
nice to get that information back.
> I'm talking about people converting tests that were working just fine
> to be TODO tests because the latest version of Foo (an external
> module) has a new bug. While Foo is broken, they don't want lots of
> bug reports from CPAN testers that they can't do anything about.
>
> This use of TODO allows you to silence the alarm and also gives you a
> way to spot when the alarm condition has passed. It's convenient for
> developers but it's 2 fingers to users who can now get false passes
> from the test suites,
It still boils down to what known bugs the author is willing to release with.
Once the author has decided they don't want to hear about a broken
dependency, and that the breakage isn't important, the damage is done. The
TODO test is orthogonal.
Again, consider the alternative which is to comment the test out. Then you
have NO information.
So I think the problem you're concerned with is poor release decisions. TODO
tests are just a tool being employed therein.
[1] Don't get to hung up on names, things only get one even though they can do
lots of things. I'm sure you've written lots of perl programs that didn't do
much extracting or reporting.
--
Reality is that which, when you stop believing in it, doesn't go away.
-- Phillip K. Dick
