Hi Robert,
Hi imacat,
I can only apologize again for the tone in my first email.
On 2005-06-15, at 20:59:14 +0100, Robert Rothenberg wrote:
> > ... And I still think that your test reports are "wrong" in the way that
>
> Test reports are not wrong if they accurately reflect what happens what
> happens on the system.
Sure, and I really, really appreciate the CPAN testers facility.
It helped me more than once to spot real problems with my modules.
> If there's no C compiler, and the tests fail, then the report should show
> that.
^^^^^^^^^^^^^^^^^^
Yes, but there isn't a single test executed.
> A "problem" is that CPANPLUS is not yet sophisticated to recognize this
> and do what it should, which is to not send a report.
Agreed.
> Or maybe CPANPLUS, MakeMaker, etc. should recognize that the module requires
> a C compiler and refuse to try and build it in the first place.
For registered modules, CPAN(PLUS) could use the DLSIP code.
Or there could be some information in the META.yml.
> Or maybe the CPAN::WWW::Testers software can be updated to differentiate
> failure reports due to missing compilers from other kinds of failures.
>
> > a) the source of the failure it isn't obvious by looking at the report,
> > unless you're fluent in Chinese, and
>
> I can't read Chinese and could figure it out:
To me, it looked like it was running 'cl' and failing.
But I couldn't figure out the reason why it was failing.
I was -- wrongly -- assuming the presence of a compiler.
> > > Microsoft (R) Program Maintenance Utility Version 1.50
> > > Copyright (c) Microsoft Corp 1988-94. All rights reserved.
> > >
> > > 'cl' ���O�����Υ~���R�O�B
> > > �i���檺�{���Χ妸�ɡC
> >
> > ^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> > > NMAKE : fatal error U1077: 'C:\WINDOWS\system32\cmd.exe' : return
> code '0x1'
> > > Stop.
>
> and so what if it's in Chinese?
>
> Maybe in the future the testing system will be sophisticated enough to
> read meta-data and know to send test reports in multiple languages (the
> author's and tester's).
Maybe. But the Chinese error above seems to be generated by
the OS rather than by the testing system.
> > b) rarely anyone ever looks at the detailed reports; they browse CPAN
> > search, they see failure on Windows, they assume the modules doesn't
> > work on Windows.
>
> Anyone who uses test reports to determine whether a module works or not
> on their platform would look at the failure reports.
That's how it should be. But I've got different experience.
> CPAN "visitors" should have a minimum technical ability to decipher
> failure reports. If they are evaluating a module that may suit their
> needs, presumably they would not superficially ignore it just because it
> failed in some test reports.
>
> There are many types of failure reports, and often for reasons beyond
> the author's control.
>
> Accept that failures (and even strange unknown and na reports) happen.
I have absolutely no problem with "strange" failures.
But -- after checking a couple of different test reports from that
system -- this looked like a systematic, not a strange, problem to
me, and so I wrote a reply.
I run a smoke box (for the Perl core) myself that produces nothing
but failure reports, but that's because it tests configurations not
covered in other smokes.
> > 3) You install a compiler on your machine, or you stop that specific
> > smoke. There are already a couple of people who have set up
> > Windows smoke boxes. And your other smoke setups are fine, so
> > this suggestion really isn't meant offensive.
>
> So "visitors" would see that your module works on some Windows
> platforms, and not be too worried about failures.
>
> We need platforms without compilers to run tests, so that code which
> recognizes that there is no compiler and acts accordingly can be tested!
Agreed. But then we need the test tools to handle these
platforms correctly.
> I agree 100% with imacat here:
>
> > I'm providing my CPU, my harddisk, my bandwidth, my
> > time to help testing tens of new packages everyday.
>
> This is a volunteer effort. People are trying to contribute to the
> community, and they get little back for it. If they get criticised for
> trying to help out, then they'll stop helping out.
I know, and I really appreciate that. As I said above, CPAN testers
is great, and it would not exist without people like imacat.
OTOH, CPAN would not exist without people contributing to it.
I'm not unfamiliar with the concept of volunteer effort. I've
spent a reasonable amount of my free time over the last couple
of years writing CPAN modules, contributing to other modules and
to the Perl core.
> If you don't like what's happening, do something contructive like submit
> patches to ExtUtils::MakeMaker and Module::Build, CPANPLUS,
> Test::Reporter, CPAN::YACSmoke or CPAN::WWW::Testers to minimize failure
> reports like this one.
I'd love to, but currently I've reached the limit of what I can
do in my free time.
Best regards,
Marcus
