With the recent transition to JUnit4 runner and changes to
LayoutEngineTestCase, it is now someone difficult to determine (from output
data alone) which layoutengine test file failed when a regression occurs.
In particular, the exception trace generated from EvalCheck, TrueCheck,
etc. does not include the test case input file name. So one ends up with
something like the following in
the TEST-org.apache.fop.layoutengine.LayoutEngineTestCase.txt file in
builds/test-reports:

Testcase: runTest[342] took 0.017 sec
        Caused an ERROR
Expected XPath expression to evaluate to '2', but got '' (XPath:
//pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/text)
java.lang.RuntimeException: Expected XPath expression to evaluate to '2',
but got '' (XPath:
//pageViewport[@nr=1]/page/regionViewport/regionBody/mainReference/span/flow/block/lineArea/inlineparent/tex\
t)
  at org.apache.fop.layoutengine.EvalCheck.doCheck(EvalCheck.java:86)
at org.apache.fop.layoutengine.EvalCheck.check(EvalCheck.java:60)
 at
org.apache.fop.layoutengine.LayoutEngineTestCase.doATChecks(LayoutEngineTestCase.java:258)
at
org.apache.fop.layoutengine.LayoutEngineTestCase.checkAll(LayoutEngineTestCase.java:191)
 at
org.apache.fop.layoutengine.LayoutEngineTestCase.runTest(LayoutEngineTestCase.java:172)

Testcase: runTest[343] took 0.012 sec

Unfortunately, there is no way to correlate runTest[342] with a specific
test case input file. It would be very useful (for AT and IF tests) to also
include the test case input file name/path in this output. Otherwise, one
is forced to run junit in a debugger with a breakpoint on EvalCheck,
TrueCheck, etc.

Reply via email to