On 1/21/07, Ovid <[EMAIL PROTECTED]> wrote:
--- Steve Peters <[EMAIL PROTECTED]> wrote:
> The primary feature that we've seen missing in the Perl core is the
> ability
> to run tests in parallel. This would greatly reduce our timelines in
> the
> fix-make-make test cycle that we go through in making changes in the
> core.
Yves was also pointing out stuff about the smoke tests. This is
another area I've no experience in. Is this a matter of looking at
Test::Smoke and figuring out how to integrate our work with it and
seeing if it doesn't fall down? After that, I'm not sure what would be
involved in running tests in parallel with this.
I think you should look at the core test suite and Test::Smoke as
being your primary target. If you can make Smoke tests easier to do
and more robust then you have a winner on your hands.
My thinking is like this: core tests whacks of stuff, loads of modules
plus the internals, plus core tests are routinely run on many
platforms and architectures both in smoke form and not. Blead
routinely has to deal with threading errors that hang, segfaults and
the like, so using it there will expose every wart you could imagine.
Thus if you can prove that your parser is suitable for the core then I
think you will have made a pretty big step towards proving that it is
suitable for release in the wild.
> Overall, I believe that we need to be a bit cautious in taking a new
> rewrite of Test::Harness into the core.
No question about it. This is one of the most critical pieces of
software out there. If it fails, everything else falls down, too.
Absolutely, and proving it in core will be a massive confidence boost
for the rest of perldom.
> Before commiting it to the core, I'd
> like to use it in the smoke tests first to make sure that any
> performance issues are discovered in advance,
TAPx::Parser collects a lot more information thann Test::Harness and
the cost is that it currently runs a bit slower, despite my working
very hard to profile and optimize it. I still have more tricks up my
sleeve for that, but I suspect that running tests in parallel would be
the key for this.
And theres the point, smoke tests need access to a lot of that data.
If your harness is designed right it should make Test::Smoke much
easier to implement, and potentially faster even if it is slower than
harness itself.
You really should look into the smoke reports and speak with
Abe-Timmerman about what the harness should be doing to make smoking
easier and more reliable yadayada. He should be able to give you lots
of useful comments on the perl test process and harness features that
would be advantageous to have.
> and that we are able to tap into the
> wide variety of architectures and operating systems that the core
> smoke tests allow.
Though the latest test results and comments for TAPx::Parser look
promising, we don't have access to that 'wide variety of architecture'.
How can we test something so key without throwing the switch? This
concerns me, but I don't know enough about this area to really comment.
As i said earlier, if we are using your parser for testing blead then
given time it will be quite thoroughly tested on a range of
architectures
Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"
