Re: [FFmpeg-devel] [PATCH v5 2/2] fate: add tests for psnr and ssim filter

[Carl Eugen Hoyos]

2017-10-04 10:54 GMT+02:00 Tobias Rapp :
> On 27.09.2017 17:20, Carl Eugen Hoyos wrote:
>>
>> 2017-09-27 15:30 GMT+02:00 Tobias Rapp :
>>>
>>> Metadata filter output is passed through an Awk script comparing floats
>>
>>
>> Is awk needed on the host or the target system?
>
>
> Awk is started by fate-run.sh directly, so it is needed on the
> host system AFAICS.

> Note that Awk was already used before in fate-run.sh, which was one
> reason why I didn't use the proposed Perl script.

Thank you!

Carl Eugen
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH v5 2/2] fate: add tests for psnr and ssim filter

[Tobias Rapp]

On 27.09.2017 17:20, Carl Eugen Hoyos wrote:

2017-09-27 15:30 GMT+02:00 Tobias Rapp :

Metadata filter output is passed through an Awk script comparing floats


Is awk needed on the host or the target system?


Awk is started by fate-run.sh directly, so it is needed on the host 
system AFAICS. Note that Awk was already used before in fate-run.sh, 
which was one reason why I didn't use the proposed Perl script.


Regards,
Tobias

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH v5 2/2] fate: add tests for psnr and ssim filter

[Carl Eugen Hoyos]

2017-09-27 15:30 GMT+02:00 Tobias Rapp :
> Metadata filter output is passed through an Awk script comparing floats

Is awk needed on the host or the target system?

Carl Eugen
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [PATCH v5 2/2] fate: add tests for psnr and ssim filter

[Tobias Rapp]

Metadata filter output is passed through an Awk script comparing floats
against reference values with specified "fuzz" tolerance to account for
architectural differences (e.g. x86-32 vs. x86-64).

Signed-off-by: Tobias Rapp 
---
tested on Linux x86-32/64 and Mips (Qemu)

v5:
  - fixed trailing whitespace within reference files

v4:
  - fixed iteration order of final print loop
  - update of debug output

v3:
  - avoid precision loss due to rounding of float values

v2:
  - removed CPUFLAGS work-around for ssim filter issue
  - added metadata float value post-processing script

 tests/fate-run.sh |  9 +
 tests/fate/filter-video.mak   | 14 
 tests/ref/fate/filter-refcmp-psnr-rgb | 45 
 tests/ref/fate/filter-refcmp-psnr-yuv | 45 
 tests/ref/fate/filter-refcmp-ssim-rgb | 30 
 tests/ref/fate/filter-refcmp-ssim-yuv | 30 
 tests/refcmp-metadata.awk | 64 +++
 7 files changed, 237 insertions(+)
 create mode 100644 tests/ref/fate/filter-refcmp-psnr-rgb
 create mode 100644 tests/ref/fate/filter-refcmp-psnr-yuv
 create mode 100644 tests/ref/fate/filter-refcmp-ssim-rgb
 create mode 100644 tests/ref/fate/filter-refcmp-ssim-yuv
 create mode 100644 tests/refcmp-metadata.awk

diff --git a/tests/fate-run.sh b/tests/fate-run.sh
index 9aa9a22..c5480c7 100755
--- a/tests/fate-run.sh
+++ b/tests/fate-run.sh
@@ -234,6 +234,15 @@ lavftest(){
 ${base}/lavf-regression.sh $t lavf tests/vsynth1 "$target_exec" 
"$target_path" "$threads" "$thread_type" "$cpuflags" "$target_samples"
 }
 
+refcmp_metadata(){
+refcmp=$1
+pixfmt=$2
+fuzz=${3:-0.001}
+ffmpeg $FLAGS $ENC_OPTS \
+-lavfi 
"testsrc2=size=300x200:rate=1:duration=5,format=${pixfmt},split[ref][tmp];[tmp]avgblur=4[enc];[enc][ref]${refcmp},metadata=print:file=-"
 \
+-f null /dev/null | awk -v ref=${ref} -v fuzz=${fuzz} -f 
${base}/refcmp-metadata.awk -
+}
+
 video_filter(){
 filters=$1
 shift
diff --git a/tests/fate/filter-video.mak b/tests/fate/filter-video.mak
index d1e1341..78cd471 100644
--- a/tests/fate/filter-video.mak
+++ b/tests/fate/filter-video.mak
@@ -747,6 +747,20 @@ FATE_FILTER_SAMPLES-$(call ALLYES, MOV_DEMUXER 
H264_DECODER AAC_FIXED_DECODER PC
 fate-filter-meta-4560-rotate0: tests/data/file4560-override2rotate0.mov
 fate-filter-meta-4560-rotate0: CMD = framecrc -flags +bitexact -c:a aac_fixed 
-i $(TARGET_PATH)/tests/data/file4560-override2rotate0.mov
 
+REFCMP_DEPS = FFMPEG LAVFI_INDEV TESTSRC2_FILTER AVGBLUR_FILTER METADATA_FILTER
+
+FATE_FILTER_SAMPLES-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += 
fate-filter-refcmp-psnr-rgb
+fate-filter-refcmp-psnr-rgb: CMD = refcmp_metadata psnr rgb24
+
+FATE_FILTER_SAMPLES-$(call ALLYES, $(REFCMP_DEPS) PSNR_FILTER) += 
fate-filter-refcmp-psnr-yuv
+fate-filter-refcmp-psnr-yuv: CMD = refcmp_metadata psnr yuv422p
+
+FATE_FILTER_SAMPLES-$(call ALLYES, $(REFCMP_DEPS) SSIM_FILTER) += 
fate-filter-refcmp-ssim-rgb
+fate-filter-refcmp-ssim-rgb: CMD = refcmp_metadata ssim rgb24
+
+FATE_FILTER_SAMPLES-$(call ALLYES, $(REFCMP_DEPS) SSIM_FILTER) += 
fate-filter-refcmp-ssim-yuv
+fate-filter-refcmp-ssim-yuv: CMD = refcmp_metadata ssim yuv422p
+
 FATE_SAMPLES_FFPROBE += $(FATE_METADATA_FILTER-yes)
 FATE_SAMPLES_FFMPEG += $(FATE_FILTER_SAMPLES-yes)
 FATE_FFMPEG += $(FATE_FILTER-yes)
diff --git a/tests/ref/fate/filter-refcmp-psnr-rgb 
b/tests/ref/fate/filter-refcmp-psnr-rgb
new file mode 100644
index 000..f06db57
--- /dev/null
+++ b/tests/ref/fate/filter-refcmp-psnr-rgb
@@ -0,0 +1,45 @@
+frame:0pts:0   pts_time:0
+lavfi.psnr.mse.r=1381.80
+lavfi.psnr.psnr.r=16.73
+lavfi.psnr.mse.g=896.00
+lavfi.psnr.psnr.g=18.61
+lavfi.psnr.mse.b=277.38
+lavfi.psnr.psnr.b=23.70
+lavfi.psnr.mse_avg=851.73
+lavfi.psnr.psnr_avg=18.83
+frame:1pts:1   pts_time:1
+lavfi.psnr.mse.r=1380.37
+lavfi.psnr.psnr.r=16.73
+lavfi.psnr.mse.g=975.91
+lavfi.psnr.psnr.g=18.24
+lavfi.psnr.mse.b=435.72
+lavfi.psnr.psnr.b=21.74
+lavfi.psnr.mse_avg=930.67
+lavfi.psnr.psnr_avg=18.44
+frame:2pts:2   pts_time:2
+lavfi.psnr.mse.r=1403.20
+lavfi.psnr.psnr.r=16.66
+lavfi.psnr.mse.g=954.05
+lavfi.psnr.psnr.g=18.34
+lavfi.psnr.mse.b=494.22
+lavfi.psnr.psnr.b=21.19
+lavfi.psnr.mse_avg=950.49
+lavfi.psnr.psnr_avg=18.35
+frame:3pts:3   pts_time:3
+lavfi.psnr.mse.r=1452.80
+lavfi.psnr.psnr.r=16.51
+lavfi.psnr.mse.g=1001.02
+lavfi.psnr.psnr.g=18.13
+lavfi.psnr.mse.b=557.39
+lavfi.psnr.psnr.b=20.67
+lavfi.psnr.mse_avg=1003.74
+lavfi.psnr.psnr_avg=18.11
+frame:4pts:4   pts_time:4
+lavfi.psnr.mse.r=1401.25
+lavfi.psnr.psnr.r=16.67
+lavfi.psnr.mse.g=1009.80
+lavfi.psnr.psnr.g=18.09
+lavfi.psnr.mse.b=602.42
+lavfi.psnr.psnr.b=20.33
+lavfi.psnr.mse_avg=1004.49
+lavfi.psnr.psnr_avg=18.11
diff --git a/tests/ref/fate/filter-refcmp-psnr-yuv 
b/tests/ref/fate/filter-refcmp-psnr-yuv
new file mode 100644
index 000..0e634ed
---