On Fri, 2003-12-26 at 05:29, Peter Kullmann wrote:
> J. Pietschmann wrote:
> >
> > John Austin wrote:
> > > RedHat 9.0 (my system anyhow) includes a command 'pdftopbm'
> > that will
> > > convert a PDF to multiple PBM (protable Bit Map) files that might be
> > > comparable.
> > ...
> > >It would certainly help detect pixel-sized changes.
> > > That might help regression testing.
I wasn't thinking of using graphics as the primary means of
comparing output. It was just a thought that one could use
visualization in some circumstances:
+ pixels that were white in both images would be
rendered as white
+ pixels that were black in both images would be
rendered as black
+ black pixels in the first image that were white
in the second could be rendered as red
+ white pixels in the first image that were black
in the second could be rendered as blue
I thought of the idea of overlaying images for comparison when
I was scrolling through the side-by-side renderings of PDF's
that Finn posted yesterday (what does 'yesterday' mean in a
discussion that crosses the International Date Line ?)
Of course, this color-based scheme breaks down for test cases
that use color.
> >
> > We need regression tests badly. Some problems to ponder:
> > a) Tests need to be automated for actually being useful.
> > JUnit seems the way to go. Unfortuanately, it's still
> > underutiliyed in FOP.
> > b) We don't have much *unit* tests. There's only the
> > UtilityCodeTestSuite.java. We need much more tests for
> > basic functionality. The problem seems to be however
> > that an elaborated test harness needs to be written in
> > order to do unt tests for, e.g. layout managers.
> > c) In order to test the whole engine at once, from FO input
> > to generating PDF/whatever, well, a binary compare with
> > a pregenerated PDF would be as sufficient as comparing
> > bitmap images. Problems here:
> > + The files to compare against are binary, and consume
> > a lot of space. Well, take a look at GenericFOPTestCase.java
> > which uses MD5 sums, one for the FO in order to detect
> > accidental changes to the source, and one for the result.
> > + Even small changes have potential to break the whole test
> > suite, even if nothing important changed, let's say the
> > order of entries in a PDF dictionary. Rendering bitmaps
> > from PDF eliminates this, but then you wont find regressions
> > in non-visible stuff.
> > All in all, if there are 143 template PDFs and a change causes
> > mismatches for all, what will you do? Examine everything,
> > comparing pixels, check whether there are visible differences
> > at all, and then judge whether the original or the newly
> > generated PDF is at fault? I don't think this will be done
> > often.
Use tests for binary equality to detect differences. Visualization
might be one tool, useful in following up on detected differences.
I might want to use the technique to compare the effects of changes
to a document. For example:
What happens on page 7 when I change space-before="10pt" to
space-before="15pt" ?
A colorized visualization would give me a better idea than separate
files. Remember that our brains are all quite different. Your
rote visual memory ability is probably much better than mine. You
might learn more from a side-by-side comparison than I would.
Crap. Now I have to give an example. Perhaps it won't take that
long.
> >
> > Ideas welcome!
> >
> > J.Pietschmann
> >
>
> As an alternative approach for c) one could create tests along
> the following lines: Suppose you want to test left margin
> properties of a block. For this a simple fo file is rendered as
> a bitmap. The bitmap will not be compared to a reference bitmap
> but some elementary assertions are calculated. For instance one
> such assertion could be: "The rectangle of width 1 inch of the
> left edge is blank." I don't know of a tool that can do this
> but it should be pretty straight forward to implement.
Probably not that hard to do once you get inside an image file
in a program. Especially if you know the colors will be black
(0,0,0) and white (255,255,255) or a small number of selected
colors.
> So, in the test suit one has a piece of fo containing a test
> document and some assertions in java or coded in xml that should
> be fulfilled by the rendered image of the fo.
>
> Assertions could contain some of the following pieces:
> - a specified rectangle is blank (or of some specific color)
> - a specified rectangle has only colors x and y (to test background
> and foreground colors of a block).
> - a given line contains the color pattern white, yellow,
> (white, black)*, green, white. IE. a regular expression on the colors
> along a line. This could be used to test border colors along
> a horizontal or vertical line through a bordered block.
> - along a given line the size of the first and last black region
> is at least xxx inches (to test border thickness)
>
> The advantage of this approach seems to me that it is relatively
> easy to maintain. The test suite is small (no binaries). It can
> easily be automated in junit.
>
> On the other hand, the approach is limited to relatively simple
> test cases. For instance it will not be possible to test font
> height, font style and text adjustments easily.
>
> Peter
--
John Austin <[EMAIL PROTECTED]>
