> The problem is that parent_todo is not considered when setting the OK's in the details in subtests. That means if a subtest ok fails, and the parent is in todo, the 'ok' and 'actual_ok' are both set to false.
Why is this a bug? After all, it's only the top level of TAP that ever matters when it comes to the overall test pass/failure, and a subtest's pass or fail status will be adjusted accordingly for the parent TODO at the *parent's* level. TODOs don't need to propagate downward to all contained subtests. Further, if we required that TODO state must propagate down to all contained subtests, that means that subtests must always be aware of that upper state -- which rules out any use of separate processes to to create the subtest results, which is conceptually possible (something else, thinking that it is a top-level test, can provide us the TAP, and then we nest it as a subtest into another test -- and this should be possible both with TAP strings and with an event streamer). On Mon, Dec 21, 2015 at 4:04 PM, Chad Granum <exodi...@gmail.com> wrote: > Test::Builder appends all Ok's to a structure it can return via > $tb->details. These Ok's have the fields 'ok' and 'actual_ok'. 'actual_ok' > is the true/false value passed to $tb->ok, 'ok' is adjusted to be true if > the test was run under TODO. > > > https://metacpan.org/source/EXODIST/Test-Simple-1.001014/lib/Test/Builder.pm#L845 > > The bug is in subtest handling. When you start a subtest you essentially > create a new Test::Builder singleton, and the TODO state is clear int he > new one, but it has a field 'parent_TODO' that gets set. This is used to > ensure that failures in a subtest when the parent has TODO set do not count > as actual failures. > > The problem is that parent_todo is not considered when setting the OK's in > the details in subtests. That means if a subtest ok fails, and the parent > is in todo, the 'ok' and 'actual_ok' are both set to false. > > The fix is easy, just change line 845 to look at parent_TODO. However this > breaks Test::ClassMoose because it uses details to verify its results. > https://metacpan.org/source/DROLSKY/Test-Class-Moose-0.67/t/basic.t#L55 > > I spoke with autarch about this test since he is the current maintainer. > He agrees with me that 'ok' should be true in that test, and that this is > probably a bug. > > I have not yet figured out what else this may break on cpan, I will try to > do that tomorrow. > > *What I am looking for from cpan-workers:* > > 1) Do you think this is actually a bug, or does it work as you think it > should? > 2) Even if we all agree it is a bug, can we fix it, or do we need to > preserve it to avoid breaking things? > > -Chad > > > > >
