Fergal Daly wrote:
>> As long as you're releasing a new version, why would you not upgrade your
>> module's dependency to use the version that works?
>
> Your module either is or isn't usable with version X of Foo.
>
> If it is usable then you would not change your dependency before or
> after the bug in version X is fixed (maybe I have a good reason not to
> upgrade Foo and you wouldn't want your module to refuse to install if
> it is actually usable).
>
> If it isn't usable then marking your tests as TODO was the wrong thing
> to do in the first place, you should have bailed out due to
> incompatibility with version X and not bothered to run any tests at
> all. I think Extutils::MM does not have any way to specify complex
> version dependencies but with Module::Build you could say
ETOOBINARY
Modules do not have a binary state of working or not working. They're
composed of piles of (often too many) features. Code can be shippable without
every single thing working.
The TODO test is useful when the working version *does not yet* exist. If
it's a minor feature or bug then rather than hold up the whole release waiting
for someone else to fix their shit, you can mark it TODO and release. This is
the author's decision to go ahead and release with a known bug. We do it all
the time, just not necessarily with a formal TODO test.
> I am basically against the practice of using TODO to cope with
> external breakage. Not taking unexpected passes seriously encourages
> this practice. Apart from there being other ways to handle external
> breakage that seem easier, using TODO is actually dangerous as it can
> cause false passes in 2 ways. Says version X of Foo has a non-serious
> bug so you release version Y of Bar with some tests marked TODO. The
> we risk
Maybe we're arguing two different situations. Yours seems to be when there is
a broken version of a dependency, but a known working version exists. In this
case, you're right, it's better resolved with a rich dependency system.
My case is when a working version of the dependency does not exist, or the
last working version is so old it's more trouble than it's worth. In this
case the author decides the bug is not critical, can't be worked around and
doesn't want to wait for fix in the dependency. The decision is whether or
not to release with a known bug. After that, wrapping it in a TODO test is
just an alternative to commenting it out.
Compare with the more common alternative for shipping with a known bug which
is to simply not have a test at all.
> 1 Version X+1 of Foo is even worse and will cause Bar to eat your dog.
> Sadly for your dog, the test that might have warned him has been
> marked TODO.
If they release Bar with a known bug against Foo X where your dog's fur is
merely a bit ruffled, then that's ok. If version X+1 of Foo causes Bar to eat
your dog then why didn't their tests catch that? Was there not a "dog not
eaten" test? If not then that's just an incomplete test, the TODO test has
nothing to do with that.
The "dog not eaten" test wouldn't have been part of the TODO test, that part
worked fine when the author released and they'd have gotten the "todo passed"
message and known to move it out of the TODO block.
Or maybe they're just a cat person.
Point is, there's multiple points where good testing practice has to break
down for this situation to occur. The use of TODO test is orthogonal.
> 2 You're using version X-1 of Foo, everything is sweet, your dog can
> relax. You upgrade to version Y+1 of Bar which has a newly introduced
> dog-eating bug. This bug goes undetected because the tests are marked
> TODO. So long Fido.
That's the author's (poor) decision to release with a known critical dog
eating bug. The fact that it's in a TODO test is incidental.
> I still have not seen an example of using TODO in this manner that
> isn't better handled in a different way.
>
> As before, I am not advocating changing the current Test::* behaviour
> to fail on unexpected passes as that would just be a mess. It's just
> that whenever this is discussed it ends up with people advocating what
> I consider wrong and dangerous uses of TODO and so I am pointing this
> out again,
Most of the cases above boil down to "author decided to release with a known
critical bug" or "tests didn't check for a possible critical bug".
You're right in that marking something TODO is not an excuse to release with a
known critical bug, but I don't think anyone's arguing that.
--
I do have a cause though. It's obscenity. I'm for it.
- Tom Lehrer
