Hello!
I think this is a great idea that we really need.
About how do we do it, I wouldn't choose a hash function, because a very
small detail would make the hash differ and in the end the tests would fail
all the time.
I would include the propper-rendered images together with the test sif file
and then after rendering, use ImageMagick compare program (
http://www.imagemagick.org/script/compare.php) to measure the overall image
distortion between the original image and the new rendered image (it offers
several measurement methods).
When executing make tests we could show the result of the tests and set a
threshold value from which the tests would fail.
We can set this threshold value very low, so the images have to be really
similar. The idea is that if there is an increment of 1 in the value of a
channel in a pixel because of whatever-bug that we have fixed, the test
would still pass.
About putting the test files in a separate repository, for me is ok but I
don't see any reason why not including the test files in our repository in
the future, so we can include the "tests" objetive in the makefile for
easier testing.
Cheers!
Diego.
2012/9/27 Konstantin Dmitriev <[email protected]>
> 2012/9/27 Carlos López González <[email protected]>:
> > Hi!
> > Recently I've subscribed to cairo mailing list and I've realized that
> they
> > do a good practice that consists on add a test sample code for each bug
> they
> > find out (when possible).
> >
> > I think that it would be good to have a kind of testing area in the
> > repository code that would consists in a list of sif files that produced
> a
> > bug in the past with its corresponding bug ID and the commit ID that
> fixed
> > it.
> >
> > There should be a way to test the buggy sif files to automatically know
> if
> > they produce a good result when rendered with a new version of synfig.
> I'm
> > thinking on a md5sum operation over the rendered file and compare it to
> the
> > good md5sum one provided on the test area. But I'm worried that between
> > architectures and platforms there could be some non significant
> differences
> > that would make the md5sum different.
> >
> > The idea is that someone does 'make tests' it would render all the buggy
> > files and then would do the test comparison and give a result of all OK
> or
> > what did fail.
> >
> > Do you like the idea? Does anyone know another way to preform a test for
> the
> > synfig render operation?
> >
> > Apart of the buggy files there could be some basics samples (for each
> > primitive and filter for example) that could be tested to continue
> > confirming that the render is looking fine.
> >
> > For testing the 32/64 bits and different platforms involved I think that
> we
> > can do some tests by sending here or in the forum, sample render image
> > results from a basic shape and do the md5sum for all of them to see what
> > happen.
> >
> > I hope you make some comments on this. I think it would help us to make a
> > better and consistent coding along the time.
>
> Hi!
> I think this is a very good idea. It's kind of unit testing. ^__^
> Two notes:
> * Images compared by md5sum should be uncompressed - that way we can
> avoid influence by changes in compression algorythms or
> architecture-specific optimizations.
> * My suggestion is to put sample sif files in separate repository at
> github.
> Cheers!
> K.
> --
> http://morevnaproject.org/
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://ad.doubleclick.net/clk;258768047;13503038;j?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> Synfig-devl mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/synfig-devl
>
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://ad.doubleclick.net/clk;258768047;13503038;j?
http://info.appdynamics.com/FreeJavaPerformanceDownload.html
_______________________________________________
Synfig-devl mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/synfig-devl
