On 2011.7.14 6:00 PM, Eden Cardim wrote:
> It appears there are several distributions on CPAN whose tests don't
> pass when using several cores. This is an annoyance because some of the
> larger deplists nearly always have a module with broken tests in them,
> so testing can't just be a fire-and-forget operation, it also confuses
> some of the less experient developers that have HARNESS_OPTIONS set in
> .bashrc.
Just to make sure, when you say "using several cores" you mean testing in
parallel? Like setting the jobs flag (or -j with prove)?
> I'm wondering if there has been any discussion directed to
> addressing this issue, I've done some research and couldn't find
> anything in that regard. What I'm thinking is that maybe the cpan tester
> service can do an extra multi-process run on each distribution, and
> store the test status along with the other metadata and somehow hand
> that information to Test::Harness so it can automatically run in
> single-process mode when it finds a test suite that doesn't pass in
> multi-process mode. Anyway, I'd just like to get the discussion started,
> and I'd be willing to put in some tuits towards implementing a solution
> if necessary.
I don't like work arounds for test failures, and what you're describing is an
enormous work around database.
I think the first step is to report multi-process test failures as normal
bugs. Especially for the most important modules. Then they can start to fix
them.
Second step is to set up a CPAN smoker doing multi-process testing. Then
we'll start getting ubiquitous testing data and annoying distribution authors
with failures. It will be interesting to see what percentage pass. I suspect
it is higher than we think.
Moving out of the realm of what you can do today, it may be worth adding a key
to the meta file which indicates the tests are not safe for parallel testing.
I prefer this over the inverse (a flag stating that the tests ARE safe for
parallel testing) because moving forward it would be most efficient for large
scale testing if we encourage authors to make their tests safe for parallel
processing.
--
I do have a cause though. It's obscenity. I'm for it.
- Tom Lehrer
