Re: svn commit: r1872919 - /subversion/trunk/subversion/tests/libsvn_subr/mergeinfo-test.c

2020-02-04 Thread Julian Foad 

Daniel Shahaf wrote:

julianf...@apache.org wrote on Fri, Jan 17, 2020 at 14:54:46 -:

@@ -2089,18 +2130,22 @@ test_rangelist_merge_random_canonical_in
+if (failure_mode)
{
-printf("testcase FAIL: %s / %s\n",
+printf("first example of a failure mode: %s / %s\n"

[...]

Test programs aren't supposed to print arbitrary stuff to stdout.

Specifically, the interface between run_tests.py and the test programs
specifies¹ that test programs shall only print "PASS:" and "FAIL:" lines by
default.  ¹ Citation: subversion/tests/README:34


Thanks for querying this, and for pinging me as I'd forgotten to get 
back to it.


That README does not say "only" or "by default"; it says "Upon finishing 
a test, the program reports the results in [this] format [...]".


Many tests print non-prefixed informative lines before the 
PASS/FAIL/XPASS/XFAIL/SKIP line when they XFAIL, and a few tests do when 
they pass.


This test is in line with existing tests: it prints these diagnostics 
only when it XFAILs.



We could of course extend that to support printing of new info, for
example, as "INFO:" lines.  So would it make sense to prepend "INFO:" to that
line?


This sounds like a reasonable wish for general improvement of the test 
suite.


- Julian

Re: svn commit: r1872919 - /subversion/trunk/subversion/tests/libsvn_subr/mergeinfo-test.c

2020-01-17 Thread Daniel Shahaf 
julianf...@apache.org wrote on Fri, Jan 17, 2020 at 14:54:46 -:
> @@ -2089,18 +2130,22 @@ test_rangelist_merge_random_canonical_in
> +if (failure_mode)
>{
> -printf("testcase FAIL: %s / %s\n",
> +printf("first example of a failure mode: %s / %s\n"
> +   "  %s\n",
> rangelist_to_string(rlx, iterpool),
> +   rangelist_to_string(rly, iterpool),
> +   failure_mode);

Test programs aren't supposed to print arbitrary stuff to stdout.

Specifically, the interface between run_tests.py and the test programs
specifies¹ that test programs shall only print "PASS:" and "FAIL:" lines by
default.  We could of course extend that to support printing of new info, for
example, as "INFO:" lines.  So would it make sense to prepend "INFO:" to that
line?

Cheers,

Daniel

¹ Citation: subversion/tests/README:34