(As posted to Perl Monks, but probably a good question for this list as well)
As I've been developing CPAN::Reporter, I've been thinking about what
information I wished I had when examining failure reports on CPAN
Testers. Test::Reporter already includes the results of "perl -V". As
part of CPAN::Reporter, I include a list of prerequisites specified
and the module versions loaded (and I believe that CPANPLUS does this
as well).
Now I'm considering what else might be useful. From an initial small
list I had, I've brainstormed a longer list of additional things that
might be useful to include. For example:
Toolchain module versions
It seemed like it might be useful to know what versions of the module
toolchain are in use. These aren't listed in prerequisites and could
be a potential source of test problems (particularly false negatives).
* CPAN
* CPANPLUS -- though not really relevant for CPAN::Reporter
* ExtUtils::MakeMaker
* Module::Build, along with ExtUtils::CBuilder and ExtUtils::ParseXS
Environment variables
Many test programs change behavior based on environment variables. In
addition, it seemed to me that some environment variables might answer
other questions about the operating environment at the time of the
failure.
* qr/PERL/ -- anything specific to Perl and kind enough to flag it
in the name of the variable
* PATH
* SHELL
* TERM
* AUTOMATED_TESTING -- used by many authors to toggle tests for
non-interactive situations
* AUTHOR_TESTING -- I use this for pod/coverage tests, though I
probably should have called it PERL_AUTHOR_TESTING
* INCLUDE -- win32 gcc path to header files
* LIB -- win32 gcc path to library files
* LD_LIBRARY_PATH -- GNU/linux path to library files
Other stuff
This is more of a catch-all, but it seemed like many of these might be
useful to know when investigating a test failure report.
* perl executable name -- $^X -- helpful for detecting
spaces/non-standard locations
* UID/EUID and GID/EGID -- $<, $>, $(, $) -- running as root or as
a normal user?
* umask -- affects files created during tests
* Win32::GetOSVersion() -- only for win32, of course, but would
give more detail than just "MSWin32" from $^O (e.g. Win98? Vista?)
Program versions
This is a much more speculative idea, as there is no standard way to
ask for a version for an external program. It might or might not
really matter except in very unusual XS cases.
* compiler
* $Config{make}
Feedback needed
I'm interested in what people think of these lists. Which of these
seem useful? Which are overkill? Are there things missing that you'd
really like to have in a test report. Which would you prioritize?
Regards,
David Golden
