I haven't used Test::NoWarnings. However, for me personally, I don't
like to suppress output unless I am catching it in an explicit test. The
code I am working on is designed to throw warnings or errors under
certain conditions. Especially in code that is designed to run under a
web server, it is very, very bad to have output spewing to STDOUT that
was not intended.
I have been using Capture::Stdout/Capture::Stderr for this purpose. It's
designed to run in the context of the Test::XXX packages.
You can capture warnings (FORCE_CAPTURE_WARN => 1) but you have to be
pretty careful. If you are merely swallowing up your output, you could
end up missing warnings thrown by the interpreter ("use of uninitialized
value in concat").
If you're going to capture output, I think you have to be methodical
about it and add a test for intended and unintended output. If it's
output that needs to be suppressed in a later bugfix, then wrap its
capture in a TODO block but do capture the output and handle it. Later
you can change the test to a "negative" test ie;
(is($stderr->read(), undef, '... execution of this code snippet should
not produce warnings';)
or
(is(!stderr->read(), ' ... there should be no more messages on STDERR');
One caveat I have found when explicitly capturing your
STDERR/STDOUT/WARN output in tests is that once you commit to it in your
test suite, you can get yourself stuck. I rarely execute code under
development outside a test anymore. So, if I happen to add a debug
statement in the code and I have a test like the one above which tests
for the existence of messages, it will fail, AND I lose the debug output
unless I am very specific about which handle I am testing. It's not a
fatal flaw but it has to be dealt with.
This can be worked around a number of ways. One is to modify your
test(s) while you're debugging to suppress capture (a klunky workaround
prone to errors), throw to a handle you're not capturing, or copy a
modified version of your test that doesn't capture any output.
Spurious output can actually skew your test results, especially if you
are running Test::Harness with the merge (STDOUT/STDERR) option. You can
run 'prove -p --merge' against your test to individually debug your
test's output and make sure it will play nicely with Test::Harness.
I have also found that in certain versions of perl (5.6.1), even with
the {FORCE_CAPTURE_WARN} flag, warnings appear not to be thrown even
though the output exists on the TTY. Your mileage my vary.
So I would agree that eliminating or handling Warning output is a good
idea. Doing this at a global harness level rather than in the individual
tests though is where I might disagree on the approach.
On Tue, 2008-06-10 at 07:49 -0400, David Golden wrote:
> On Tue, Jun 10, 2008 at 12:28 AM, Gabor Szabo <[EMAIL PROTECTED]> wrote:
> > The issue I am trying to solve is how to catch and report
> > when a test suit gives any warnings?
>
> Are there situations where a test suite should give warnings? I.e.
> stuff that the user should see that shouldn't get swallowed by the
> harness running in a quiet (not verbose) mode?
>
> For example, I have some tests in CPAN::Reporter that test timing out
> a command. Since that could look like a test has hung (to an
> impatient tester) I make a point to use warn to flag to the user that
> the test is sleeping for a timeout test.
>
> Looks like this:
>
> $ Build test --test_files=t/13_record_command.t
> t/13_record_command......18/37 # sleeping for timeout test
> t/13_record_command......22/37 # sleeping for timeout test
> t/13_record_command......26/37 # sleeping for timeout test
> t/13_record_command......ok
> All tests successful.
> Files=1, Tests=37, 19 wallclock secs ( 0.01 usr 0.01 sys + 6.51 cusr
> 2.06 csys = 8.59 CPU)
>
> So is there a better way to do this than warn?
>
> That said, if you try this at home (with Proc::ProcessTable), you'll
> also get a lovely warning from Proc::ProcessTable having a
> non-portable v-string. That is a warning that should perhaps be
> fixed, though it turns out to be upstream. Should I clutter my code
> with stuff to suppress it? Maybe.
>
> But I don't see how I can have the one without the other.
>
> --David
