Here's an experimental script that uses Test::Harness::Straps and
XML::Generator to run perl test files and get the sort of XML created
by the ant task - I want to have CruiseControl run my Perl
tests and see the test results:
http://twoalpha.blogspot.com/2007/01/junit-style-xml-from-per
On 21 Jan 2007, at 21:15, Eric Wilhelm wrote:
"just that" being just the dir structure, or also the env var
execute-if-not-dir thing?
Just the dir thing. If people are keen on the env var being able to
point to an executable I can certainly do that too.
--
Andy Armstrong, hexten.net
Eric Wilhelm wrote:
# from Andy Armstrong
# on Sunday 21 January 2007 11:37 am:
On 21 Jan 2007, at 19:16, Eric Wilhelm wrote:
PERL_TEST_HARNESS_DUMP_TAP="$(test_dir_for_this_dist)"
If that isn't enough (which may be the case in Adam's cpan
injection), I
suppose you could do "if the env var
--- Andy Armstrong <[EMAIL PROTECTED]> wrote:
> YAML output Curtis? Shall I add it to the list?
I think this is the way to go. I'm using it for execrc and I'll be using it
for .runtestsrc, so I suppose that keeping The One True Format would be a
good thing (also, YAML is likely to be used in TAP
# from Andy Armstrong
# on Sunday 21 January 2007 11:37 am:
>On 21 Jan 2007, at 19:16, Eric Wilhelm wrote:
>> PERL_TEST_HARNESS_DUMP_TAP="$(test_dir_for_this_dist)"
>>
>> If that isn't enough (which may be the case in Adam's cpan
>> injection), I
>> suppose you could do "if the env var is an exe
On 21 Jan 2007, at 19:16, Eric Wilhelm wrote:
Yes. It should be expected to provide a single directory and not
apply
any policy to that beyond mirroring the t/ directory structure. If
you
need to run it fairly manually, but still stay in your dump framework,
you could do:
PERL_TEST_HARNE
* Andy Armstrong <[EMAIL PROTECTED]> [2007-01-21 16:20]:
> I wasn't really proposing having a lower case 'n' in the
> middle of TEST_HARnESS_DUMP_YAML either :)
AwwW. MiXeD CaSE iS sO MuCH cOOLeR thAn stRAiGhT cAPs.
--
*AUTOLOAD=*_;sub _{s/(.*)::(.*)/print$2,(",$\/"," ")[defined wantarray]/e;$1}
# from Andy Armstrong
# on Sunday 21 January 2007 09:21 am:
>> OR possibly even...
>>
>> /home/me/directory/ADAMK/Dist-Name-0.01.tar.gz/01_compile.t
>> /home/me/directory/ADAMK/Dist-Name-0.01.tar.gz/02_main.t
>
>That might be harder. I guess whatever harness you use to set the
>environment varia
In article <[EMAIL PROTECTED]>, Adam
Kennedy <[EMAIL PROTECTED]> wrote:
> Adam Kennedy wrote:
> > So, seeing as you are going to take over Test::Harness as well, is NOW
> > where I ask you for the PITA-related capturing of a copy of the TAP
> > streams? :)
>
> Let me just clarify what I'm wanti
On 21 Jan 2007, at 18:08, chromatic wrote:
Using the existing Straps, replacing it with another implementation
or both?
Using the existing Straps.
Cool. I'm hopeful we'll be able to emulate that pretty well.
--
Andy Armstrong, hexten.net
On Sunday 21 January 2007 10:03, Andy Armstrong wrote:
> On 21 Jan 2007, at 17:59, chromatic wrote:
> > On Sunday 21 January 2007 04:50, Randal L. Schwartz wrote:
> >> Not that it's part of CPAN, but I also have written a magazine
> >> article
> >> or two using Test::Harness::Straps, so it's in
On 21 Jan 2007, at 17:59, chromatic wrote:
On Sunday 21 January 2007 04:50, Randal L. Schwartz wrote:
Not that it's part of CPAN, but I also have written a magazine
article
or two using Test::Harness::Straps, so it's in the public corpus of
such usage.
I've spoken and published on it too.
On Sunday 21 January 2007 04:50, Randal L. Schwartz wrote:
> Not that it's part of CPAN, but I also have written a magazine article
> or two using Test::Harness::Straps, so it's in the public corpus of
> such usage.
I've spoken and published on it too.
-- c
On 21 Jan 2007, at 16:57, Adam Kennedy wrote:
I had envisioned something like just
PERL_TEST_HARNESS_DUMP_TAP=/home/me/directory
And so you would get something like either...
/home/me/directory/01_compile.t (names matching test name)
/home/me/directory/02_main.t
That's definitely possible. I
Can you imagine the logfile of 177305 tests?
Assuming 50 chars across, that's 5-10meg.
I'm completely fine with 5-10 meg of TAP output.
Please note I'm already dealing with clustered and distributed virtual
machines and giant XML stores. I'm slinging 1 gig system images across
the internet..
Adam Kennedy wrote:
So, seeing as you are going to take over Test::Harness as well, is NOW
where I ask you for the PITA-related capturing of a copy of the TAP
streams? :)
Let me just clarify what I'm wanting here, for those talking about
alternative parsing.
I don't want ANY difference in t
Andy Armstrong wrote:
On 21 Jan 2007, at 14:28, Andy Armstrong wrote:
Yup - point taken. YAML it is then. Everyone happy with that?
By which I mean YAML and/or TAP. There's no real reason not to be able
to output TAP if someone might find that useful.
export TEST_HARNESS_DUMP_TAP=/home/me/t
I dont get this logic.
Why cant something that wants to monitor the test process do something
other than make test?
They can do a make, and or make test-prep or whatever, and then call
into an alternative test harness framework to monitor the tests.
Can you explain why this is a no-go in more d
On 21 Jan 2007, at 15:13, David Golden wrote:
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
export TEST_HARNESS_DUMP_TAP=/home/me/test.tap
export TEST_HARnESS_DUMP_YAML=/home/me/test.yaml
make test
nit: I know it breaks with tradition, but
PERL_TEST_HARNESS_DUMP_TAP, etc.?
When I
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
export TEST_HARNESS_DUMP_TAP=/home/me/test.tap
export TEST_HARnESS_DUMP_YAML=/home/me/test.yaml
make test
nit: I know it breaks with tradition, but PERL_TEST_HARNESS_DUMP_TAP, etc.?
When I asked about environment variables to capture for CP
On 21 Jan 2007, at 14:28, Andy Armstrong wrote:
Yup - point taken. YAML it is then. Everyone happy with that?
By which I mean YAML and/or TAP. There's no real reason not to be
able to output TAP if someone might find that useful.
export TEST_HARNESS_DUMP_TAP=/home/me/test.tap
export TEST_HA
On 21 Jan 2007, at 14:21, demerphq wrote:
Also how will this stream represent things like "test file
segfaulted", "test file hung and was killed", timing out the test
process, etc. All legitimate cases when dealing with core testing.
Yup - point taken. YAML it is then. Everyone happy with that?
On 21 Jan 2007, at 14:00, David Golden wrote:
I only said YAML to get it out there before anyone said XML. ;-)
Phew :)
Adam Kennedy has a real small implementation in YAML::Tiny that could
be used, or you could probably do it by hand, since you only need to
report a few numbers and dump a bi
On 1/21/07, David Golden <[EMAIL PROTECTED]> wrote:
On 1/21/07, demerphq <[EMAIL PROTECTED]> wrote:
> Why cant something that wants to monitor the test process do something
> other than make test?
>
> They can do a make, and or make test-prep or whatever, and then call
> into an alternative test
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
On 21 Jan 2007, at 13:36, demerphq wrote:
> I dont get this logic.
>
> Why cant something that wants to monitor the test process do something
> other than make test?
>
> They can do a make, and or make test-prep or whatever, and then call
> in
--- David Golden <[EMAIL PROTECTED]> wrote:
> Adam Kennedy has a real small implementation in YAML::Tiny that could
> be used, or you could probably do it by hand, since you only need to
> report a few numbers and dump a big string (and YAML has a sort of
> "HERE" document format).
As of a couple
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
On 21 Jan 2007, at 13:39, David Golden wrote:
> Not just TAP. "make test" runs Test::Harness, which runs tests and
> parses the TAP with TAPx::Parser to produce a result. And then
> CPAN.pm/CPAN::Reporter would need to run TAPx::Parser *agai
In trying to get runtests to run against the core Perl test suite on a
freshly built download, I'm having a few difficulties. 'make test' says
this:
u=5.02 s=4.72 cu=297.54 cs=98.73 scripts=934 tests=117325
This implies to me that we have 934 .t files in t/, lib/ and ext/.
'runtests' say
On 1/21/07, demerphq <[EMAIL PROTECTED]> wrote:
Why cant something that wants to monitor the test process do something
other than make test?
They can do a make, and or make test-prep or whatever, and then call
into an alternative test harness framework to monitor the tests.
Can you explain why
On 21 Jan 2007, at 13:39, David Golden wrote:
Not just TAP. "make test" runs Test::Harness, which runs tests and
parses the TAP with TAPx::Parser to produce a result. And then
CPAN.pm/CPAN::Reporter would need to run TAPx::Parser *again* to
figure out what happened. That's not a step forward.
On 21 Jan 2007, at 13:36, demerphq wrote:
I dont get this logic.
Why cant something that wants to monitor the test process do something
other than make test?
They can do a make, and or make test-prep or whatever, and then call
into an alternative test harness framework to monitor the tests.
Ca
On 21 Jan 2007, at 13:28, Adam Kennedy wrote:
That said, potentially if all the TAP is merged into a single YAML
document, I can probably parse it and split it up myself.
Given that TAPx::Parser is a TAP parser I believe that dumping raw
TAP is workable.
We certainly /could/ generate YAML
On 21 Jan 2007, at 13:28, Abe Timmerman wrote:
I see now that on OpenVMS you also use IPC::Open3, that in turn
uses fork(). fork() is not implemented on OpenVMS, so this will not
work.
Although I'm not a VMS expert, I do have a testdrive account, and
can test some stuff if that helps.
Do
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
On 21 Jan 2007, at 13:15, David Golden wrote:
>> Can you sketch out the interface you'd like?
>
> I vote for saving structured text files with a well-defined format in
> a well-defined location. (YAML?)
Or TAP? :)
Not just TAP. "make test
On 1/21/07, Adam Kennedy <[EMAIL PROTECTED]> wrote:
The main reason I want a raw TAP stream is so that later I can rerun the
TAP parser/harness later with improved parsing code.
That said, potentially if all the TAP is merged into a single YAML
document, I can probably parse it and split it up m
On 1/21/07, David Golden <[EMAIL PROTECTED]> wrote:
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
> > Of course, it also depends upon what
> > output information is being scraped. If it's only something simple
> > like 'All
> > tests successful', then this is an easier task.
>
> I pres
On 21 Jan 2007, at 13:23, David Golden wrote:
This is a false assumption. You've got to consider that the
"canonical" way to run tests for a distribution is still "make test"
and thus Test::Harness is getting called from a command-line embedded
in a Makefile. The program calling "make test" (e.
Ovid wrote:
--- demerphq <[EMAIL PROTECTED]> wrote:
[SNIP]
Likewise you should review the work of the smoke suite for handling
oddball cases like VMS, you certainly shouldnt think that windows
will
be the oddest platform your code will run on if it is core
integrated.
Well, so far, I have o
On 21 Jan 2007, at 13:15, David Golden wrote:
Can you sketch out the interface you'd like?
I vote for saving structured text files with a well-defined format in
a well-defined location. (YAML?)
Or TAP? :)
--
Andy Armstrong, hexten.net
On 21 Jan 2007, at 13:02, Ovid wrote:
--- Andy Armstrong <[EMAIL PROTECTED]> wrote:
I presume anyone who wanted to know that currently would just be
calling
T::H::execute_tests which returns three hashes that summarise the
test
results.
I'd prefer an object get returned instead. It's muc
Ovid wrote:
--- demerphq <[EMAIL PROTECTED]> wrote:
[SNIP]
Likewise you should review the work of the smoke suite for handling
oddball cases like VMS, you certainly shouldnt think that windows
will
be the oddest platform your code will run on if it is core
integrated.
Well, so far, I have o
On 21 Jan 2007, at 12:56, Andy Armstrong wrote:
If there's any code out there that gets intimate with
Test::Harness::Straps et al we're not easily going to be able to
support
it with TAPx::Harness::Compatible.
I need to clarify this based on Randal's post. I'm pretty sure we can
support /u
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
> Of course, it also depends upon what
> output information is being scraped. If it's only something simple
> like 'All
> tests successful', then this is an easier task.
I presume anyone who wanted to know that currently would just be call
On 21 Jan 2007, at 12:50, Randal L. Schwartz wrote:
Andy> I have a CPAN mirror that I'm planning to grep in it's
entirety looking
Andy> for evidence of dependencies on Test::Harness::Straps|Point|
whatever.
Not that it's part of CPAN, but I also have written a magazine article
or two using T
On 1/21/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
On 21 Jan 2007, at 11:31, Adam Kennedy wrote:
> So, seeing as you are going to take over Test::Harness as well, is
> NOW where I ask you for the PITA-related capturing of a copy of the
> TAP streams? :)
Almost certainly :)
Can you sketch out
--- Andy Armstrong <[EMAIL PROTECTED]> wrote:
> I presume anyone who wanted to know that currently would just be calling
> T::H::execute_tests which returns three hashes that summarise the test
> results.
I'd prefer an object get returned instead. It's much safe to call
$harness->all_passed than
On 21 Jan 2007, at 12:23, Ovid wrote:
I have a CPAN mirror that I'm planning to grep in it's entirety
looking for evidence of dependencies on Test::Harness::Straps|Point|
whatever.
That's a great start. I assume you have a copy of Schwern's grep_cpan
script?
Didn't. Do now :)
http://www.sch
> "Andy" == Andy Armstrong <[EMAIL PROTECTED]> writes:
Andy> I have a CPAN mirror that I'm planning to grep in it's entirety looking
Andy> for evidence of dependencies on Test::Harness::Straps|Point| whatever.
Not that it's part of CPAN, but I also have written a magazine article
or two using
--- Andy Armstrong <[EMAIL PROTECTED]> wrote:
> I have a CPAN mirror that I'm planning to grep in it's entirety
> looking for evidence of dependencies on Test::Harness::Straps|Point|
> whatever.
That's a great start. I assume you have a copy of Schwern's grep_cpan
script?
> My current plan
On 21 Jan 2007, at 11:31, Adam Kennedy wrote:
So, seeing as you are going to take over Test::Harness as well, is
NOW where I ask you for the PITA-related capturing of a copy of the
TAP streams? :)
Almost certainly :)
Can you sketch out the interface you'd like?
--
Andy Armstrong, hexten.ne
On 21 Jan 2007, at 09:07, Ovid wrote:
I suspect what Andy was meaning was that it's OK to break the
underbelly of Straps, Point, etc., and not the actual Test::Harness
API, since it's the former I was referring to.
TAPx::Harness::Compatible is still intended to pass all of the
Test::Harness tes
So, seeing as you are going to take over Test::Harness as well, is NOW
where I ask you for the PITA-related capturing of a copy of the TAP
streams? :)
Adam K
Ovid wrote:
Hi all,
Andy Lester sent me an email (which I'm relaying with permission)
saying that he feels the TAPx::Parser distributi
Andy Lester schrieb:
I suspect that some of the stuff in Test::Harness, such as Straps.pm
and Point.pm, would go away. That's a bit of concern for me as I've no
way of knowing how ubiquitous their use is. I do know that a few
people are overriding straps -- 'prove' even has a switch to let you
On 1/21/07, Ovid <[EMAIL PROTECTED]> wrote:
--- Steve Peters <[EMAIL PROTECTED]> wrote:
> The primary feature that we've seen missing in the Perl core is the
> ability
> to run tests in parallel. This would greatly reduce our timelines in
> the
> fix-make-make test cycle that we go through in m
--- Steffen Mueller <[EMAIL PROTECTED]> wrote:
> Andy Lester schrieb:
> > That's why this is gonna be 3.0. Break the API all you want.
>
> Woah there! This is still going to be on CPAN and pushed to all
> clients
> semi-automatically. A year ago or so, a bug in a single Test:: module
> cascade
55 matches
Mail list logo
