J. Pietschmann wrote:
> 
> John Austin wrote:
> > RedHat 9.0 (my system anyhow) includes a command 'pdftopbm' 
> that will
> > convert a PDF to multiple PBM (protable Bit Map) files that might be
> > comparable.
> ...
>  >It would certainly help detect pixel-sized changes.
> > That might help regression testing.
> 
> We need regression tests badly. Some problems to ponder:
> a) Tests need to be automated for actually being useful.
>   JUnit seems the way to go. Unfortuanately, it's still
>   underutiliyed in FOP.
> b) We don't have much *unit* tests. There's only the
>   UtilityCodeTestSuite.java. We need much more tests for
>   basic functionality. The problem seems to be however
>   that an elaborated test harness needs to be written in
>   order to do unt tests for, e.g. layout managers.
> c) In order to test the whole engine at once, from FO input
>   to generating PDF/whatever, well, a binary compare with
>   a pregenerated PDF would be as sufficient as comparing
>   bitmap images. Problems here:
>   + The files to compare against are binary, and consume
>    a lot of space. Well, take a look at GenericFOPTestCase.java
>    which uses MD5 sums, one for the FO in order to detect
>    accidental changes to the source, and one for the result.
>   + Even small changes have potential to break the whole test
>    suite, even if nothing important changed, let's say the
>    order of entries in a PDF dictionary. Rendering bitmaps
>    from PDF eliminates this, but then you wont find regressions
>    in non-visible stuff.
> All in all, if there are 143 template PDFs and a change causes
> mismatches for all, what will you do? Examine everything,
> comparing pixels, check whether there are visible differences
> at all, and then judge whether the original or the newly
> generated PDF is at fault? I don't think this will be done
> often.
> 
> Ideas welcome!
> 
> J.Pietschmann
> 

As an alternative approach for c) one could create tests along 
the following lines: Suppose you want to test left margin 
properties of a block. For this a simple fo file is rendered as 
a bitmap. The bitmap will not be compared to a reference bitmap
but some elementary assertions are calculated. For instance one
such assertion could be: "The rectangle of width 1 inch of the
left edge is blank." I don't know of a tool that can do this
but it should be pretty straight forward to implement. 

So, in the test suit one has a piece of fo containing a test 
document and some assertions in java or coded in xml that should
be fulfilled by the rendered image of the fo. 

Assertions could contain some of the following pieces:
- a specified rectangle is blank (or of some specific color)
- a specified rectangle has only colors x and y (to test background
and foreground colors of a block).
- a given line contains the color pattern white, yellow, 
(white, black)*, green, white. IE. a regular expression on the colors
along a line. This could be used to test border colors along 
a horizontal or vertical line through a bordered block.
- along a given line the size of the first and last black region
is at least xxx inches (to test border thickness)

The advantage of this approach seems to me that it is relatively
easy to maintain. The test suite is small (no binaries). It can
easily be automated in junit. 

On the other hand, the approach is limited to relatively simple
test cases. For instance it will not be possible to test font 
height, font style and text adjustments easily.

Peter


Reply via email to