Hi Rohit,
I guess we've each observed output that the other has not :-)
Per your (1): I've seen a few cases where lines between test_end and
test_start do belong to the preceding one, though they are indeed the
exception, not the rule. Looking at the name to assign to previous seems to
catch the proper ones in all the cases I've seen (default test lists don't
repeat the same test twice in a row).
Per your (2): Haven't seen that, interesting. That's a bit harder to sort
out, but the script could at least catch the lines that report PASS/FAIL/BROK
etc. And perhaps lines that don't look like a collection of var=value.
Hmmm.... maybe all it needs is another regexp test, see patch below (against
the genhtml.pl file I posted, which btw was against its latest version, 1.3).
I haven't taken time to look at why things get out of order in the first place,
which seems like the correct thing to fix (well, I looked enough to notice that
fflush(stdout) was really there :-). Though making the script handle it
doesn't hurt, and I've simply been trying to get an accurate look at test
results, this was the quickest way to do it.
-Marc
--- genhtml-v4.pl 2009-05-21 01:08:14.747493000 -0700
+++ genhtml.pl 2009-05-22 02:07:47.756226000 -0700
@@ -116,18 +116,18 @@
}
# Read test result parameters and test output
- while ($line !~ /$end_tag/) {
+ while (1) {
+ get_line(\$line) or last TEST;
+ last if $line =~ /$end_tag/;
($read_output = 1, next) if $line =~ /$output_tag/;
($read_output = 0, next) if $line =~ /$execution_tag/;
- if ($read_output) {
+ if ($read_output or $line !~
/^(\s*\w+=(".*"|\S*))+\s*$/) {
push @output_lines, $line;
} else {
while ($line =~ s/^\s*(\w+)=(".*"|\S*)//) {
$values{$1} = $2;
}
}
- } continue {
- get_line(\$line) or last TEST;
}
# Treat lines that follow <<<end_test>>> as output lines
________________________________
From: rohit verma [mailto:[email protected]]
Sent: Friday, May 22, 2009 00:13
To: [email protected]
Cc: Marc Gauthier; LTP List
Subject: Re: [LTP] [PATCH] Detect test results more accurately when generating
HTML
Hi,
The suggestion to - "Process test output lines that appear between
<<<test_end>>>
and the following <<<test_start>>>. Assign to the preceding test where the
name matches, else to the following test." does not solve the problem
completely.
My observation is:
1. Lines between <<<test_end>>> and <<<test_start>>> belongs to the following
test case and not to the preceding one.
2. Also, at times I have observed the test output is spread in such a way that
few lines of the test output fall between <<<test_end>>> and <<<test_start>>>
and rest of lines appear between the <<<test_start>>> and <<<test_output>>> of
the corresponding testcase.
Regards,
rohit
On Fri, May 22, 2009 at 12:22 PM, Subrata Modak
<[email protected]<mailto:[email protected]>> wrote:
On Thu, 2009-05-21 at 01:13 -0700, Marc Gauthier wrote:
> Hi, my first patch to this list. Actually not a patch,
> but the files themselves, to avoid line-wrapping issues
> and because the diff is twice the size of the new file.
>
> -Marc
Thanks Marc.
Rohit,
Can you test this and see if these solves the problem that you were
discussing for long ?
Regards--
Subrata
>
>
> -----------
> Detect test results more accurately when generating HTML
>
> Process test output lines that appear between <<<test_end>>>
> and the following <<<test_start>>>. Assign to the preceding
> test where the name matches, else to the following test.
>
> If a single test has multiple types of results (e.g. both
> FAIL and WARN), report only the most significant one, to
> avoid mis-computing the total number of PASS tests or
> total counts that don't add up to the number of tests.
>
> If a test's output has no explicit result (PASS, FAIL, etc),
> look at the exit value to determine whether it passed.
>
> Setting the SHOW_UNRESOLVED environment variable to 1
> classifies as UNRESOLVED any test with no explicit result
> and a zero exit code.
>
> Setting the SUMMARY_OUTPUT environment variable to 1
> causes only one line of output per test to be shown, for a
> tighter page that allows quickly scanning the results.
>
> Show percentage of each result type in summary section.
>
> Simplify parsing a bit.
>
> Signed-off-by: Marc Gauthier <[email protected]<mailto:[email protected]>>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals. Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________ Ltp-list mailing list
> [email protected]<mailto:[email protected]>
> https://lists.sourceforge.net/lists/listinfo/ltp-list
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
