Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 11/02/2013 01:19 AM, Luca Barbato wrote: lu - still with no time to write documentation to the wiki... I seem to be unable to find the LCOV coverage report of libav anywhere :( I've tried googling, as well as a lucky-guess with coverage.libav.org (similar to ffmpeg's URL [1]), but to no avail. It might be nice to have the links to LCOV maybe in the FATE docs [2] or the GcovCoverageHowTo [3] maybe? Thanks, Pb == References: [1] http://coverage.ffmpeg.org/index.html [2] http://www.libav.org/fate.html [3] https://wiki.libav.org/GcovCoverageHowTo ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 10/27/2013 12:07 PM, Luca Barbato wrote: On 27/10/13 10:14, Peter B. wrote: How do I add the reference files, required for tests to be able to diff? You can upload to our local ftp[1] and drop me a note. [1] http://upload.libav.org I get a 404 when I go to that URL :( I propose it should be ftp://; instead of http://;. I've uploaded my FFV1 testfiles to: ftp://upload.libav.org/incoming/fate-ffv1 Patch for FFV1 FATE tests will follow as soon as I've cleaned out my try-n-error leftovers ;) Regards, Pb ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 03/11/13 12:15, Peter B. wrote: On 11/02/2013 01:19 AM, Luca Barbato wrote: lu - still with no time to write documentation to the wiki... I seem to be unable to find the LCOV coverage report of libav anywhere :( I've tried googling, as well as a lucky-guess with coverage.libav.org (similar to ffmpeg's URL [1]), but to no avail. It depends on your options enabled, so it would be meaningful only if we integrate it in FATE, actually might be a neat idea. It might be nice to have the links to LCOV maybe in the FATE docs [2] or the GcovCoverageHowTo [3] maybe? I'll work on it if nobody does before (again projects taking priority). lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 03/11/13 12:44, Peter B. wrote: On 10/27/2013 12:07 PM, Luca Barbato wrote: On 27/10/13 10:14, Peter B. wrote: How do I add the reference files, required for tests to be able to diff? You can upload to our local ftp[1] and drop me a note. [1] http://upload.libav.org I get a 404 when I go to that URL :( I propose it should be ftp://; instead of http://;. I've uploaded my FFV1 testfiles to: ftp://upload.libav.org/incoming/fate-ffv1 Patch for FFV1 FATE tests will follow as soon as I've cleaned out my try-n-error leftovers ;) great! Thank you =) lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 11/03/2013 12:53 PM, Luca Barbato wrote: On 03/11/13 12:15, Peter B. wrote: I seem to be unable to find the LCOV coverage report of libav anywhere :( I've tried googling, as well as a lucky-guess with coverage.libav.org (similar to ffmpeg's URL [1]), but to no avail. It depends on your options enabled, so it would be meaningful only if we integrate it in FATE, actually might be a neat idea. Ok. So it's not done by default. I just found it nice to be able to compare my FATE results to the official ones, in order to spot LCOV regressions. It might be nice to have the links to LCOV maybe in the FATE docs [2] or the GcovCoverageHowTo [3] maybe? I'll work on it if nobody does before (again projects taking priority). When there's LCOV coverage report generated for libav, you could send me the link and I can try to add it to the docs? Regards, Pb ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 03/11/13 12:57, Peter B. wrote: On 11/03/2013 12:53 PM, Luca Barbato wrote: On 03/11/13 12:15, Peter B. wrote: I seem to be unable to find the LCOV coverage report of libav anywhere :( I've tried googling, as well as a lucky-guess with coverage.libav.org (similar to ffmpeg's URL [1]), but to no avail. It depends on your options enabled, so it would be meaningful only if we integrate it in FATE, actually might be a neat idea. Ok. So it's not done by default. I just found it nice to be able to compare my FATE results to the official ones, in order to spot LCOV regressions. It might be nice to have the links to LCOV maybe in the FATE docs [2] or the GcovCoverageHowTo [3] maybe? I'll work on it if nobody does before (again projects taking priority). When there's LCOV coverage report generated for libav, you could send me the link and I can try to add it to the docs? I'll bake it in FATE probably so as for the warnings it will tell you if your past run changed something and were. (now I have to figure out which is the best tool for this purpose) lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 30/10/13 23:40, Peter B. wrote: On 10/29/2013 08:57 PM, Luca Barbato wrote: On 10/28/2013 03:18 PM, Peter B. wrote: All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? I would still need an answer on this. I'd like to reduce my current fate-suite/ffv1 sample set size of ~9.9 MiB, by generating the required videos by FATE, instead of having to upload them to the fate-suite files. Is that possible? Yes we can have first the encoder test check against a reference and then use the produced files to check the decoder. Sounds a good idea. I'm right now trying to implement this. I've found out that the files generated during the encoding tests are stored in tests/data/fate. Is there any variable, I could use in the .mak files for the FATE tests, that points to that location? $(TARGET_PATH)/tests/data/fate/ lu - still with no time to write documentation to the wiki... ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 10/29/2013 08:57 PM, Luca Barbato wrote: On 10/28/2013 03:18 PM, Peter B. wrote: All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? I would still need an answer on this. I'd like to reduce my current fate-suite/ffv1 sample set size of ~9.9 MiB, by generating the required videos by FATE, instead of having to upload them to the fate-suite files. Is that possible? Yes we can have first the encoder test check against a reference and then use the produced files to check the decoder. Sounds a good idea. I'm right now trying to implement this. I've found out that the files generated during the encoding tests are stored in tests/data/fate. Is there any variable, I could use in the .mak files for the FATE tests, that points to that location? Thanks, Pb ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
Hello again, I've now added encoding tests, too. The current coverage of FFV1 in my setup is now: Filename | Line Coverage | Functions | Branches ffv1.c 99.0 % 101 / 102 100.0 % 7 / 7 80.0 % 48 / 60 ffv1.h 100.0 % 49 / 49 100.0 % 2 / 2 81.8 % 18 / 22 ffv1dec.c 77.9 % 475 / 610 100.0 % 16 / 16 55.7 % 307 / 551 ffv1enc.c 73.6 % 452 / 614 84.6 % 11 / 13 43.5 % 365 / 839 On 10/28/2013 03:18 PM, Peter B. wrote: All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? I would still need an answer on this. I'd like to reduce my current fate-suite/ffv1 sample set size of ~9.9 MiB, by generating the required videos by FATE, instead of having to upload them to the fate-suite files. Is that possible? If so, I'd be happy if someone could provide me hints to where to find an example or documentation for this. Thanks, Pb ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 29/10/13 19:29, Peter B. wrote: Hello again, I've now added encoding tests, too. The current coverage of FFV1 in my setup is now: Filename | Line Coverage | Functions | Branches ffv1.c99.0 % 101 / 102 100.0 % 7 / 7 80.0 % 48 / 60 ffv1.h 100.0 % 49 / 49 100.0 % 2 / 2 81.8 % 18 / 22 ffv1dec.c 77.9 % 475 / 610 100.0 % 16 / 16 55.7 % 307 / 551 ffv1enc.c 73.6 % 452 / 614 84.6 % 11 / 13 43.5 % 365 / 839 Looks quite nice. On 10/28/2013 03:18 PM, Peter B. wrote: All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? I would still need an answer on this. I'd like to reduce my current fate-suite/ffv1 sample set size of ~9.9 MiB, by generating the required videos by FATE, instead of having to upload them to the fate-suite files. Is that possible? Yes we can have first the encoder test check against a reference and then use the produced files to check the decoder. Sounds a good idea. If so, I'd be happy if someone could provide me hints to where to find an example or documentation for this. I pestered somebody else than me to write a Wiki page about it, I'll do soon myself I guess... there are vsynth tests that do what you want IIRC, soon I'll get you more details. Thanks a lot. lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
Quoting Luca Barbato lu_z...@gentoo.org: On 27/10/13 10:14, Peter B. wrote: 1) How do I add the reference files, required for tests to be able to diff? You can upload to our local ftp[1] and drop me a note. I'd like to keep the set of testvideos as small as possible, but I'm trying to improve the LCOV coverage of the FFV1 tests as high as possible. I've seen that the current FFV1 tests were in vcodec.mak, and if I understood it correctly, only encoding of 2 videos was tested. No decoding. Therefore, I've added additional FFV1 tests to lossless-video.mak, and transcoded the files used for Lagarith FATE-tests to a new folder in fate-suite/ffv1. Next to the Lagarith testvideos, I've used 4 frames of a test-signal generated by us, because the Lagarith videos were mainly still images, and I wanted to have at least a few frames with a visual change in them. So far my files used for FFV1 testing are around 8,5 MiB in total. I have several files for different pix_fmts to test decoding as thoroughly as necessary. All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? If there are any objections or suggestions regarding this approach, I'm happy to hear about them, so I can learn. 2) How do I get a LCOV HTML report locally? https://wiki.libav.org/GcovCoverageHowTo Worked (almost) perfectly. Thank you very much! I had to add -b . to the command shown in your Wiki, in order for lcov to find the source. So my call looks as follows: [quote] $ lcov -b . --directory . --capture --output-file coverage.info [/quote] I've already used it with my new FFV1 set, and the line-coverage for ffv1dec.c already increased. I've seen in FFmpeg's docs about lcov [1], that they've included Makefile-rules, such as: - make lcov: for generating HTML output - make lcov-reset: for resetting coverage measurements Do these also work in Libav? (They're not mentioned in your HowTo, so I thought I'd ask) Regards, Pb == References: [1] http://www.ffmpeg.org/developer.html ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 28/10/13 15:18, Peter B. wrote: Quoting Luca Barbato lu_z...@gentoo.org: On 27/10/13 10:14, Peter B. wrote: 1) How do I add the reference files, required for tests to be able to diff? You can upload to our local ftp[1] and drop me a note. I'd like to keep the set of testvideos as small as possible, but I'm trying to improve the LCOV coverage of the FFV1 tests as high as possible. I've seen that the current FFV1 tests were in vcodec.mak, and if I understood it correctly, only encoding of 2 videos was tested. No decoding. Therefore, I've added additional FFV1 tests to lossless-video.mak, and transcoded the files used for Lagarith FATE-tests to a new folder in fate-suite/ffv1. Next to the Lagarith testvideos, I've used 4 frames of a test-signal generated by us, because the Lagarith videos were mainly still images, and I wanted to have at least a few frames with a visual change in them. So far my files used for FFV1 testing are around 8,5 MiB in total. I have several files for different pix_fmts to test decoding as thoroughly as necessary. All the videos used for these tests could actually be generated, but I don't know how to add commands for generating the videos in FATE automatically so they are available for the actual tests. Could anyone point me at examples for doing so? If there are any objections or suggestions regarding this approach, I'm happy to hear about them, so I can learn. 2) How do I get a LCOV HTML report locally? https://wiki.libav.org/GcovCoverageHowTo Worked (almost) perfectly. Thank you very much! I had to add -b . to the command shown in your Wiki, in order for lcov to find the source. So my call looks as follows: Thanks I fixed it. [quote] $ lcov -b . --directory . --capture --output-file coverage.info [/quote] I've already used it with my new FFV1 set, and the line-coverage for ffv1dec.c already increased. I've seen in FFmpeg's docs about lcov [1], that they've included Makefile-rules, such as: - make lcov: for generating HTML output - make lcov-reset: for resetting coverage measurements Do these also work in Libav? Currently I'm still pondering which between lcov and gcovr is the best (gcovr is easier as the text output goes, lcov html features are much better) lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 28/10/13 18:38, Luca Barbato wrote: - make lcov: for generating HTML output - make lcov-reset: for resetting coverage measurements Do these also work in Libav? Forgot to say, yes they work =) lu ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
I was trying to read myself into FATE, in order to be able to add additional tests for FFV1. I've already read the documentation about FATE [1], and I've already managed to add new testing rules for FFV1 (in tests/fate/vcodec.mak). btw: The FATE docs [1] only describe how to run the existing tests, but I didn't find information about how to create new ones. So far so good, but now I have some questions: 1) How do I add the reference files, required for tests to be able to diff? 2) How do I get a LCOV HTML report locally? I'd like to cover more lines of FFV1's code, and therefore would like to be able to view the LCOV - code coverage report [2] from my local tests. I assumed that running the fate.sh script for submitting FATE results would create those HTML pages. I've created a fate_config.sh file, based on doc/fate_config.sh.template and ran the tests. Now I've got the $workdir with the compile.log, configure.log, report, etc. Any suggestions how I could have a local LCOV view? Thanks and regards, Pb == References: [1] http://www.libav.org/fate.html [2] http://coverage.ffmpeg.org/ffmpeg/libavcodec/ffv1enc.c.gcov.html ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On 27/10/13 10:14, Peter B. wrote: I was trying to read myself into FATE, in order to be able to add additional tests for FFV1. I've already read the documentation about FATE [1], and I've already managed to add new testing rules for FFV1 (in tests/fate/vcodec.mak). btw: The FATE docs [1] only describe how to run the existing tests, but I didn't find information about how to create new ones. So far so good, but now I have some questions: 1) How do I add the reference files, required for tests to be able to diff? You can upload to our local ftp[1] and drop me a note. 2) How do I get a LCOV HTML report locally? https://wiki.libav.org/GcovCoverageHowTo I'd like to cover more lines of FFV1's code, and therefore would like to be able to view the LCOV - code coverage report [2] from my local tests. I assumed that running the fate.sh script for submitting FATE results would create those HTML pages. No it would provide a FATE report. I've created a fate_config.sh file, based on doc/fate_config.sh.template and ran the tests. Any suggestions how I could have a local LCOV view? In theory all you need is in the wiki page I linked. I'll write a nicer howto on add FATE tests soon (I plan to refactor fate a little more) [1] http://upload.libav.org ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
[libav-devel] [PATCH] fate: add ffv1.0 test
From: Michael Niedermayer michae...@gmx.at Signed-off-by: Michael Niedermayer michae...@gmx.at --- tests/fate/vcodec.mak |3 ++- tests/ref/vsynth/vsynth1-ffv1.0 |4 tests/ref/vsynth/vsynth2-ffv1.0 |4 3 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 tests/ref/vsynth/vsynth1-ffv1.0 create mode 100644 tests/ref/vsynth/vsynth2-ffv1.0 diff --git a/tests/fate/vcodec.mak b/tests/fate/vcodec.mak index ed9fa2e..6d247a6 100644 --- a/tests/fate/vcodec.mak +++ b/tests/fate/vcodec.mak @@ -52,8 +52,9 @@ fate-vsynth%-dv-50: ENCOPTS = -dct int -s pal -pix_fmt yuv422p \ fate-vsynth%-dv-50: DECOPTS = -sws_flags neighbor fate-vsynth%-dv-50: FMT = dv -FATE_VCODEC-$(call ENCDEC, FFV1, AVI) += ffv1 +FATE_VCODEC-$(call ENCDEC, FFV1, AVI) += ffv1 ffv1.0 fate-vsynth%-ffv1: ENCOPTS = -slices 4 -strict -2 +fate-vsynth%-ffv1.0: CODEC = ffv1 FATE_VCODEC-$(call ENCDEC, FFVHUFF, AVI) += ffvhuff diff --git a/tests/ref/vsynth/vsynth1-ffv1.0 b/tests/ref/vsynth/vsynth1-ffv1.0 new file mode 100644 index 000..2a4f41c --- /dev/null +++ b/tests/ref/vsynth/vsynth1-ffv1.0 @@ -0,0 +1,4 @@ +91c237f18bc19975077c85175daed734 *tests/data/fate/vsynth1-ffv1.0.avi +2655364 tests/data/fate/vsynth1-ffv1.0.avi +c5ccac874dbf808e9088bc3107860042 *tests/data/fate/vsynth1-ffv1.0.out.rawvideo +stddev:0.00 PSNR:999.99 MAXDIFF:0 bytes: 7603200/ 7603200 diff --git a/tests/ref/vsynth/vsynth2-ffv1.0 b/tests/ref/vsynth/vsynth2-ffv1.0 new file mode 100644 index 000..962c17c --- /dev/null +++ b/tests/ref/vsynth/vsynth2-ffv1.0 @@ -0,0 +1,4 @@ +3a757276e299bf88c30e06dfb53f1c99 *tests/data/fate/vsynth2-ffv1.0.avi +3525792 tests/data/fate/vsynth2-ffv1.0.avi +dde5895817ad9d219f79a52d0bdfb001 *tests/data/fate/vsynth2-ffv1.0.out.rawvideo +stddev:0.00 PSNR:999.99 MAXDIFF:0 bytes: 7603200/ 7603200 -- 1.7.9.5 ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
Re: [libav-devel] [PATCH] fate: add ffv1.0 test
On Tue, Oct 1, 2013 at 4:33 PM, Vittorio Giovara vittorio.giov...@gmail.com wrote: From: Michael Niedermayer michae...@gmx.at Signed-off-by: Michael Niedermayer michae...@gmx.at --- tests/fate/vcodec.mak |3 ++- tests/ref/vsynth/vsynth1-ffv1.0 |4 tests/ref/vsynth/vsynth2-ffv1.0 |4 3 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 tests/ref/vsynth/vsynth1-ffv1.0 create mode 100644 tests/ref/vsynth/vsynth2-ffv1.0 diff --git a/tests/fate/vcodec.mak b/tests/fate/vcodec.mak index ed9fa2e..6d247a6 100644 --- a/tests/fate/vcodec.mak +++ b/tests/fate/vcodec.mak @@ -52,8 +52,9 @@ fate-vsynth%-dv-50: ENCOPTS = -dct int -s pal -pix_fmt yuv422p \ fate-vsynth%-dv-50: DECOPTS = -sws_flags neighbor fate-vsynth%-dv-50: FMT = dv -FATE_VCODEC-$(call ENCDEC, FFV1, AVI) += ffv1 +FATE_VCODEC-$(call ENCDEC, FFV1, AVI) += ffv1 ffv1.0 fate-vsynth%-ffv1: ENCOPTS = -slices 4 -strict -2 +fate-vsynth%-ffv1.0: CODEC = ffv1 FATE_VCODEC-$(call ENCDEC, FFVHUFF, AVI) += ffvhuff diff --git a/tests/ref/vsynth/vsynth1-ffv1.0 b/tests/ref/vsynth/vsynth1-ffv1.0 new file mode 100644 index 000..2a4f41c --- /dev/null +++ b/tests/ref/vsynth/vsynth1-ffv1.0 @@ -0,0 +1,4 @@ +91c237f18bc19975077c85175daed734 *tests/data/fate/vsynth1-ffv1.0.avi +2655364 tests/data/fate/vsynth1-ffv1.0.avi +c5ccac874dbf808e9088bc3107860042 *tests/data/fate/vsynth1-ffv1.0.out.rawvideo +stddev:0.00 PSNR:999.99 MAXDIFF:0 bytes: 7603200/ 7603200 diff --git a/tests/ref/vsynth/vsynth2-ffv1.0 b/tests/ref/vsynth/vsynth2-ffv1.0 new file mode 100644 index 000..962c17c --- /dev/null +++ b/tests/ref/vsynth/vsynth2-ffv1.0 @@ -0,0 +1,4 @@ +3a757276e299bf88c30e06dfb53f1c99 *tests/data/fate/vsynth2-ffv1.0.avi +3525792 tests/data/fate/vsynth2-ffv1.0.avi +dde5895817ad9d219f79a52d0bdfb001 *tests/data/fate/vsynth2-ffv1.0.out.rawvideo +stddev:0.00 PSNR:999.99 MAXDIFF:0 bytes: 7603200/ 7603200 Unless anyone objects, I think that this is fine to drop as well, the ffv1 test is enough. Vittorio ___ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
