Has anyone had a chance to have a look at this? If this is looks decent I'd go ahead and:
- Try and get a comprehensive set of RAW files from multiple camera models either by asking users for them or someone that already has a large set of samples (dpreview/lensrentals/etc) - Ask you guys for a set of xmp files to use for regression testing different combinations of modules (right now only the basic no changes xmp file is used) - Start adding tests for fixed bugs to make sure they don't regress - Move the actual image files out of git leaving just the metadata there and download them from a HTTP server somewhere instead. This would avoid growing the git repo and would allow integrating this into a subdir of the darktable repo - Figure out how to run this using travis-ci so the testsuite gets run on every commit and every pull request Cheers, Pedro On Fri, Mar 28, 2014 at 2:10 AM, Pedro Côrte-Real <pe...@pedrocr.net> wrote: > Hi everyone, > > As discussed on IRC I setup a simple testing framework for darktable using > Ruby: > > https://github.com/pedrocr/darktable-tests > > Right now it's pretty basic. What it does is automate the process of > comparing the darktable conversion against a reference file. The > comparison so far is pretty simple, it checks that the width and > height is the same and then does the difference between the two files > and checks that no channel (RGB) differs by more than 5 in any pixel. > > As it turns out these tests are enough to get 7 failing tests out of 7 > images, two because of width+height and 5 because of image > differences. The width+height is because I flipped the reference > images for this bug so that we have a failing test that will pass when > this is correct: > > http://www.darktable.org/redmine/issues/9876 > > The image differences is because I generated the reference images with > current git master and then the tests are running with whatever is in > $PATH (1.4.1 in my case). This correctly identifies this bug: > > http://www.darktable.org/redmine/issues/9874 > > And more misteriously also finds differences in some pixels in two > cases not affected by any of these bugs. Maybe a change in highlight > recovery or something like it? > > Comments are very welcome. > > Cheers, > > Pedro ------------------------------------------------------------------------------ Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/NeoTech _______________________________________________ darktable-devel mailing list darktable-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/darktable-devel
