>>>>> On Sat, 25 Oct 2008 13:25:04 -0400, "David Golden" <[EMAIL PROTECTED]> >>>>> said:
> There are a number of issues in the parrot distribution that deviate > from assumptions about a how a "normal" CPAN distribution behaves and > those cause CPAN::Reporter's heuristics to mis-grade it. > It gets a PASS because it looks like the distribution has a recursive > Makefile.PL (ext/Parrot-Embed/Makefile.PL), so we can't easily > disambiguate the test harness output and instead rely on the exit code > only -- and parrot uses a custom harness program (t/harness) that > doesn't return a non-zero exit code for failing tests. (I.e. "make > test" has exit code 0 even with failing tests.) Do we consider this a bug? I'd even volunteer to report it, but actually I'm not sure. > So -- things that CPAN::Reporter could do: > * tighten the check for recursive Makefile.PL to only include the > first level of sub-directories below the main one > * in the case of a recursive Makefile.PL, parse the output anyway and > do something smart. > There was a reason that I avoided the second approach when dealing > with recursive Makefile.PL's, but I don't recall exactly what it is, > now. CPAN::Reporter parses the output in reverse order of lines > looking for a Test::Harness result line and maybe I didn't want to > risk scanning all of the test output in case some distribution's test > output happened to resemble Test::Harness output. That's a little > less risky with the newer Test::Harness that has a single, well > defined format for output. Exactly. My suggestion for an approach would start from the other end: if the output resembles the new Test::Harness output, then try to rely on it. Otherwise let the existing code do its heuristics. > On the other hand, I don't really think we should be testing parrot > with CPAN Testers anyway, Why? > so this isn't a high priority for me to fix. I'm willing to do the > first option I gave, above, because it's simple and should break > anything else. (Unless someone convinces me it's a bad idea). Probably too risky? I would not bet on it. -- andreas
