On 01/05/15 08:39, Anton Khirnov wrote: > FATE tests so many things that it's not really a good benchmark of any > specific codec.
Actually it can be useful to see the global impact of a patch assuming somebody has the time and the will to benchmark. I'm mostly done with the first POC of the concept: - it takes measurement per-test (30 times to have a meaninful avg and std) - all the data is then stored to be further processed in sqlite since I'm too lazy to feed it to pandas or R. - I check if the difference per-test between different commits is meaningful, expecting to have the test testing unchanged code to show nothing meaningful. The main problem I'm seeing in this approach is that it takes hours to execute on a xeon (since it is a fate -j 1) so it is not really usable per-commit in practice until fate doesn't learn to run only the tests impacted by the code. lu _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
