On Sat, Oct 25, 2008 at 4:52 PM, Andreas J. Koenig <[EMAIL PROTECTED]> wrote: > > doesn't return a non-zero exit code for failing tests. (I.e. "make > > test" has exit code 0 even with failing tests.) > > Do we consider this a bug? I'd even volunteer to report it, but > actually I'm not sure.
I think so. Any automation that should report a failure or abort or whatever on "make test" will have a hard time doing so. I already sent it in. If they choose to mark it "WONTFIX", that's up to them. > Exactly. My suggestion for an approach would start from the other end: > if the output resembles the new Test::Harness output, then try to rely > on it. Otherwise let the existing code do its heuristics. Well, it's more subtle than that, really. Since *anything* can be printed to STDOUT by a test, I prefer to do as little text parsing as possible. A recursive Makefile.PL requires grepping the entire output for things resembling Test::Harness results. That's *probably* safe with the new format, but I still tend to want to avoid searching for magic text as much as possible. When there isn't recursive Makefile.PL, we're pretty much guaranteed to have a valid Test::Harness string found first when scanning lines in reverse. > > On the other hand, I don't really think we should be testing parrot > > with CPAN Testers anyway, > > Why? For the same reasons we don't test Perl. > > so this isn't a high priority for me to fix. I'm willing to do the > > first option I gave, above, because it's simple and should break > > anything else. (Unless someone convinces me it's a bad idea). > > Probably too risky? I would not bet on it. Sorry. I meant to say that a more limited check for recursive Makefile.PL should *not* break anything else. It is the simplest change to CPAN::Reporter that would fix this issue with parrot, so I'd prefer to do the simplest possible thing instead of something more involved with larger unknown consequences. -- David
