This is probably something we should bring up at the toolchain summit. Ether
pointed at a proposal from a while back about a new test status to be used when
the module allows itself to be built, but the installing of dependencies fails:
https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/berlin-consensus.md#cpan-testers-grading
<https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/berlin-consensus.md#cpan-testers-grading>
I didn't know about this, but Haarg and I were talking about this at the
MetaCPAN Hackathon: The decades of tooling built around CPAN Testers expects
PASS, FAIL, NA, UNKNOWN and only those. Of those, NA is most appropriate for
"it's not this distribution's fault that it cannot be installed on this
machine." Unfortunately, this is a rather large bucket: N/A includes "OS
unsupported", "Don't have a native compile toolchain", "Couldn't find a native
prereq", and now may also add "Dependencies not satisfied".
We could further differentiate these statuses with a new field containing a
status code. I really like how SMTP's error codes work, and we could adopt
something similar as: "X.Y.Z".
* "X" is the main type
* PASS = 1
* FAIL = 2
* NA = 3
* UNKNOWN = 4
* "Y" is the step of the process we made it to
* 0 is "complete/unknown"
* 1 is "extract"
* 2 is "build"
* 3 is "test"
* 4 is "install"
* "Z" is a step-specific status code
* "0" is always "generic/unknown"
* But for the "build" step
* 1 can mean "OS unsupported"
* 2 can mean "Compiler needed"
* 3 can mean "Dependencies failed"
This might be over-engineering, and I welcome any suggestions which would be
simpler. We'd also need to think about all the possible things that could go
wrong and make sure we can express them in useful categories (without being
pedantic). This may also not be the best way of expressing them: The main type
is implied by the step/status fields (making it redundant but useful
at-a-glance), and this could be clumsy to query on using SQL.
> On Mar 20, 2017, at 10:52 AM, Kent Fredric <kentfred...@gmail.com> wrote:
>
> I've seen a lot of test failures from DDICK this week that all end up
> being explainable
> as "Didn't satisfy dependencies of my dependencies"
>
> This is annoying as its a transient breakage, so the places the fail
> is being reported
> is not a place I can reasonably fix the issue.
>
> For instance:
>
> http://www.cpantesters.org/cpan/report/8ec51672-03b4-11e7-b9d0-1c36a38678f7
>
> This indicates Getopt::Long::Descriptive's dependency Params::Validate
> is not installed, but its a clear dependency of the version being used
> in the test ( 0.100 )
>
> https://metacpan.org/source/RJBS/Getopt-Long-Descriptive-0.100/Makefile.PL#L23
>
> But Params-Validate is nowhere in the constructed @INC tree.
>
> I'd only assume this is a client misconfiguration somehow, but if
> there's some bug
> in tooling / dependency chains that is causing this I'd rather we got
> to the root of it.
>
> It would also be awfully nice if there was a grade/way of indicating
> from CPAN Testers side that "X dist failed due to transiently missing
> dependencies", which IME is more N/A than FAIL
>
> ( Not to say there isn't any use in testing for dists with randomly
> missing dependencies to see if it still passes, etc, just reporting an
> expected-to-fail condition as a failure is less than useful )
>
> Otherwise, thanks for the tests.
>
>
>
> --
> Kent
>
> KENTNL - https://metacpan.org/author/KENTNL
