Hi,
attached are two patches for ctest and cdash from Volkan (on CC).
They are not ready for inclusion, but before we continue to work in this, we'd
like to get some comments whether this is a good idea, the right approach,
etc.
So, here we go.
We are using cdash a lot, and for many of our tests the log is quite verbose
and somewhat hard to read. When looking for the reason why a test failed, it
can be somewhat hard to find the line which contains ERROR.
Also, a lot of stuff is repeated every line, which also makes the log harder
to read, e.g. the directory part of the file where the log came from:
timestamp /some/where/deep/in/a/subdir/test.py: Something...
timestamp /some/where/deep/in/a/subdir/test.py: happend...
timestamp /some/where/deep/in/a/subdir/test.py: here...
timestamp /some/where/deep/in/a/subdir/test.py: ...
The directory part is actually so wide that we have to scroll to see actual
text.
So, to help with this, we came up with the idea to use HTML to format the log
so that it becomes easier to read.
The attached patch for ctest adds a new test property GENERATE_HTML, and if
this is enabled, ctest generates HTML. It searches the lines which made the
test fail and colors them red, and the lines which make a test pass are
colored green.
This makes it easier to find the fail/succeed lines.
(In the patch the fail strings are still hardcoded, this would have to use the
respective test property.)
On the cdash side, if /span is found in the log, it is considered HTML and
not escaped. I guess this should be done in a different way, which does not
involve parsing the log text.
Comments so far ?
For making our log lines shorter, we are not sure yet how to do that.
We could add support for squish XML to ctest, so that ctest reads the squish
XML and converts it to HTML which is in some way shorter, maybe a HTML table,
or something.
Or we could add a test property with a regular expression, if that regexp
matches, ctest replaces it with a shorter version or something.
Or we could modify our test scripts so that they already generate the pretty
HTML, which ctest then simply sends to cdash.
That's the ideas we have so far, but it doesn't feel quite right yet.
So, what do you think ?
Alex
From 446c27d8ff38704ffd6f8c828d49d4e7110fb1ac Mon Sep 17 00:00:00 2001
From: Kitware Robot kwro...@kitware.com
Date: Thu, 4 Apr 2013 00:01:08 -0400
Subject: added html generation capability to colorize plain output
---
Source/CTest/cmCTestRunTest.cxx | 47 +-
Source/CTest/cmCTestRunTest.h|7 -
Source/CTest/cmCTestTestHandler.cxx |5
Source/CTest/cmCTestTestHandler.h|1 +
Source/cmSetTestsPropertiesCommand.h |4 ++-
Source/cmTest.cxx|4 +++
7 files changed, 66 insertions(+), 4 deletions(-)
diff --git a/Source/CTest/cmCTestRunTest.cxx b/Source/CTest/cmCTestRunTest.cxx
index 5eabf3f..b7cb482 100644
--- a/Source/CTest/cmCTestRunTest.cxx
+++ b/Source/CTest/cmCTestRunTest.cxx
@@ -60,7 +60,17 @@ bool cmCTestRunTest::CheckOutput()
// Store this line of output.
cmCTestLog(this-CTest, HANDLER_VERBOSE_OUTPUT,
this-GetIndex() : line std::endl);
- this-ProcessOutput += line;
+
+ if( this-TestProperties-GenerateHtml == true )
+{
+// If user set GENERATE_HTML flag
+this-ProcessOutput += this-StrictCheckColorize(line);
+}
+ else
+{
+this-ProcessOutput += line;
+}
+
this-ProcessOutput += \n;
}
else // if(p == cmsysProcess_Pipe_Timeout)
@@ -71,6 +81,41 @@ bool cmCTestRunTest::CheckOutput()
return true;
}
+// text: String to be checked and colorized for PASS and FAIL conditions
+std::string cmCTestRunTest::StrictCheckColorize(const std::string text)
+{
+ // Check for the following FAIL and PASS patterns and return
+ // the line itself after colorization
+ // If no expressions are found, do nothing
+
+ // NOTE: Braced initializers are not available in c++11
+
+ const std::string FAILS[] = {FAIL, Error, ERROR, Fail, fail, Fatal, FATAL};
+ const std::string PASSES[] = {Pass, PASS, pass};
+ const std::vectorstd::string fails(FAILS, FAILS + sizeof(FAILS)/sizeof(*FAILS));
+ const std::vectorstd::string passes(PASSES, PASSES + sizeof(PASSES)/sizeof(*PASSES));
+ double cannotFound = std::string::npos;
+ double foundPos = cannotFound;
+
+ for ( int i = 0; i fails.size(); i++ )
+{
+foundPos = text.find(fails[i]);
+if ( foundPos != cannotFound )
+ {
+ return std::string(span style='color:#AA') + text + /span;
+ }
+}
+ for ( int i = 0; i passes.size(); i++ )
+{
+foundPos = text.find(passes[i]);
+if ( foundPos != cannotFound )
+ {
+ return std::string(span style='color:#00AA00') + text + /span;
+ }
+}
+ return text;
+}
+
//-
// Streamed
