Hi,
here are some thoughts about a static test framework for gegl operations.
* Functional Tests
These can be as simple as a set of files including a composition,
reference output and a test description. A perl/ruby script feeds the
composition through the gegl binary and compares the output with
reference data. Optionally some PNGs + HTML are created for visual inspection.
Test Case:
foo.xml composition
foo.xxx reference data; format to be determinded
foo.specs.yaml test specification
foo.input1.xxx optional input data
Output:
output/foo.xxx compositing result
output/foo.result.yaml pass/fail info + basic statistic measures
Optional report output:
report/foo.output.png
report/foo.reference.png
report/foo.diff.png
report/foo.html
The test specification foo.specs.yaml looks like:
comment: Regression test for Bug #1234
errorbounds:
peak-absolute: 1e-6
The result foo.result.yaml something like:
testname : foo
result : pass
errors :
peak-absolute : 1e-10
mean-square : 1e-15
differing-pixels : 123
[...]
timing :
total : 500 # ms
The data format .xxx should be a float format understood by ImageMagick,
candidates so far are MIFF, PFM, FITS, TIFF.
Reference data is most easily created using Octave. Probably a 'make reference'
target
will generate them from some ground truth m-files.
* Unit Tests
Unit tests for operations are most comfortably expressed as functional tests.
However, there are some tests which are common for all/many operations:
- rendering must be independent of tile size
- operations with spatial parameters should be scale invariance, i.e.
applying a operation on a downscaled composition using parameters which have
been
downscaled by the same factor should give a good approximation of applying the
unscaled operation on the unscaled composition and downscaling afterwards.
- neutral settings
Somewhat unrelated, i'd also like to install a round trip test for GEGL's
XML format, but haven't found a good way to deal with value rounding yet.
Comments, other things to consider, do's and don'ts?
Is somebody already working on this?
peter
_______________________________________________
Gegl-developer mailing list
Gegl-developer@lists.XCF.Berkeley.EDU
https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
