#10586: Impossible to retrieve libvmaf result from AVFrame at the output of
filter
graph
-------------------------------------+-------------------------------------
Reporter: ichlubna | Owner: (none)
Type: defect | Status: new
Priority: normal | Component: avfilter
Version: git-master | Resolution:
Keywords: libvmaf | Blocked By:
frame filter libav psnr |
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Description changed by ichlubna:
Old description:
> There is no way to retrieve vmaf score from the resulting frame produced
> at the end of the filter graph. The same approach which works for psnr
> and ssim filters is not working.
>
> When I use [https://ffmpeg.org/ffmpeg-filters.html#psnr psnr] or
> [https://ffmpeg.org/ffmpeg-filters.html#psnr ssim] filter I can list the
> result from the [https://ffmpeg.org/doxygen/4.1/structAVFrame.html frame]
> read from the sink buffer of the filter graph which is stored in the
> metadata [https://ffmpeg.org/doxygen/4.1/structAVDictionary.html
> dictionary] as:
> {{{
> char *entries = nullptr;
> av_dict_get_string(resultFrame->metadata, &entries, ',', '|');
> std::cout << entries;
> }}}
>
> I use this filter:
> {{{
> const AVFilter *vmafFilter = avfilter_get_by_name("libvmaf");
> }}}
>
> The same cannot be used for [https://ffmpeg.org/ffmpeg-
> filters.html#libvmaf libvmaf]. The metadata dictionary is empty. I
> searched the whole frame structure and couldn't find the result anywhere.
> Shouldn't this be unified?
>
> I don't see the storing of the result in the frame anywhere in the
> [https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_libvmaf.c
> source code]. Storing to the frame is visible in the
> [https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_psnr.c#L223
> code of psnr] and
> [https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_ssim.c#L388
> code of ssim]. I think that the storing of the vmaf score is just missing
> in the code.
>
> The only workaround is to call the
> [https://github.com/FFmpeg/FFmpeg/blob/9e531370b3da27aa72703b9bb40cdd697e88607e/libavfilter/vf_libvmaf.c#L654
> uninit] function of the vmaf filter which triggers the log message with
> vmaf score (having set the log level to AV_LOG_INFO) but this is not
> usable for per-frame vmaf result which would be similar to psnr and ssim.
> It would require a new construction of the filters in the graph after
> processing each frame.
>
> I think that something like this this is missing in the
> [https://github.com/FFmpeg/FFmpeg/blob/9e531370b3da27aa72703b9bb40cdd697e88607e/libavfilter/vf_libvmaf.c#L142
> do_vmaf] function:
>
> {{{
> double vmaf_score;
> vmaf_score_pooled(s->vmaf, s->model[0], pool_method_map(s->pool),
> &vmaf_score, 0, s->frame_cnt - 1);
> set_meta(&dist->metadata, "lavfi.vmaf", 0, vmaf_score);
> }}}
>
> I have tried doing so but got this error with increasing index for each
> frame:
>
> {{{
> libvmaf ERROR vmaf_predict_score_at_index(): no feature '��%�}U' at index
> 1
> }}}
New description:
There is no way to retrieve vmaf score from the resulting frame produced
at the end of the filter graph. The same approach which works for psnr and
ssim filters is not working.
When I use [https://ffmpeg.org/ffmpeg-filters.html#psnr psnr] or
[https://ffmpeg.org/ffmpeg-filters.html#psnr ssim] filter I can list the
result from the [https://ffmpeg.org/doxygen/4.1/structAVFrame.html frame]
read from the sink buffer of the filter graph which is stored in the
metadata [https://ffmpeg.org/doxygen/4.1/structAVDictionary.html
dictionary] as:
{{{
char *entries = nullptr;
av_dict_get_string(resultFrame->metadata, &entries, ',', '|');
std::cout << entries;
}}}
I use this filter:
{{{
const AVFilter *vmafFilter = avfilter_get_by_name("libvmaf");
}}}
The same cannot be used for [https://ffmpeg.org/ffmpeg-
filters.html#libvmaf libvmaf]. The metadata dictionary is empty. I
searched the whole frame structure and couldn't find the result anywhere.
Shouldn't this be unified?
I don't see the storing of the result in the frame anywhere in the
[https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_libvmaf.c
source code]. Storing to the frame is visible in the
[https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_psnr.c#L223
code of psnr] and
[https://github.com/FFmpeg/FFmpeg/blob/13a3e2a9b4697a1403d3e58eb5e8ef9c72561d54/libavfilter/vf_ssim.c#L388
code of ssim]. I think that the storing of the vmaf score is just missing
in the code.
The only workaround is to call the
[https://github.com/FFmpeg/FFmpeg/blob/9e531370b3da27aa72703b9bb40cdd697e88607e/libavfilter/vf_libvmaf.c#L654
uninit] function of the vmaf filter which triggers the log message with
vmaf score (having set the log level to AV_LOG_INFO) but this is not
usable for per-frame vmaf result which would be similar to psnr and ssim.
It would require a new construction of the filters in the graph after
processing each frame.
I think that something like this this is missing in the
[https://github.com/FFmpeg/FFmpeg/blob/9e531370b3da27aa72703b9bb40cdd697e88607e/libavfilter/vf_libvmaf.c#L142
do_vmaf] function:
{{{
double vmaf_score;
vmaf_score_pooled(s->vmaf, s->model[0], pool_method_map(s->pool),
&vmaf_score, 0, s->frame_cnt - 1);
set_meta(&dist->metadata, "lavfi.vmaf", 0, vmaf_score);
}}}
I have tried doing so but got this error with increasing index for each
frame:
{{{
libvmaf ERROR vmaf_predict_score_at_index(): no feature '��%�}U' at index
1
}}}
But seems like this is working fine and does the expected! Polled would
probably return the computed merged score while this at_index function
returns score for each frame individually. Also need to go one index down
to get a good result as the counter is not incremented yet:
{{{
vmaf_score_at_index(s->vmaf, s->model[0], &vmaf_score, s->frame_cnt - 2);
}}}
--
--
Ticket URL: <https://trac.ffmpeg.org/ticket/10586#comment:6>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker_______________________________________________
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
ffmpeg-trac-requ...@ffmpeg.org with subject "unsubscribe".
