Re: [FFmpeg-devel] [PATCH 1/2] avfilter/avfilter: add avfilter_print_config_formats()
Another Ping
Thanks,
softworkz
> -Original Message-
> From: Paul B Mahol
> Sent: Thursday, November 3, 2022 10:58 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Cc: softworkz
> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avfilter/avfilter: add
> avfilter_print_config_formats()
>
> On 10/11/22, softworkz wrote:
> > From: softworkz
> >
> > Prints the following to AVBPrint:
> >
> > For pass-through filter links:
> >
> > "All (passthrough)"
> >
> > For filters using query_formats:
> >
> > "Dynamic"
> >
> > For filters using query_formats where a call to query_formats
> > succeeds (example):
> >
> > "Dynamic, Defaults: [yuv420p, yuvj420p, yuva420p, nv12, nv21]"
> >
> > For all other filters (example):
> >
> > "[s16p, s32p, fltp, dblp]"
> >
> > Except in case when the number of formats equals the number of
> > available formats:
> >
> > "All"
> >
> > Signed-off-by: softworkz
> > ---
> > doc/APIchanges | 3 ++
> > libavfilter/avfilter.c | 102
> +++-
> > libavfilter/avfilter.h | 12 +
> > libavfilter/avfiltergraph.c | 14 +++--
> > libavfilter/internal.h | 9
> > libavfilter/version.h | 4 +-
> > 6 files changed, 136 insertions(+), 8 deletions(-)
> >
> > diff --git a/doc/APIchanges b/doc/APIchanges
> > index cbb579612e..6e2a528b04 100644
> > --- a/doc/APIchanges
> > +++ b/doc/APIchanges
> > @@ -14,6 +14,9 @@ libavutil: 2021-04-27
> >
> > API changes, most recent first:
> >
> > +2022-10-11 - xx - lavf 59.50.100 - avfilter.h
> > + Add add avfilter_print_config_formats().
> > +
> > 2022-10-05 - 37d5ddc317 - lavu 57.39.100 - cpu.h
> >Add AV_CPU_FLAG_RVB_BASIC.
> >
> > diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
> > index cc5505e65b..8cc665e19c 100644
> > --- a/libavfilter/avfilter.c
> > +++ b/libavfilter/avfilter.c
> > @@ -196,6 +196,104 @@ void avfilter_link_free(AVFilterLink **link)
> > av_freep(link);
> > }
> >
> > +static unsigned get_nb_pix_fmts()
> > +{
> > +unsigned i = 0;
> > +while (av_pix_fmt_desc_get(i++)) {}
> > +return i - 1;
> > +}
> > +
> > +static unsigned get_nb_sample_fmts()
> > +{
> > +unsigned i = 0;
> > +while (av_get_sample_fmt_name(i++)) {}
> > +return i - 1;
> > +}
> > +
> > +int avfilter_print_config_formats(AVBPrint *bp, const struct
> AVFilter
> > *filter, int for_output, unsigned pad_index)
> > +{
> > +AVFilterGraph *graph;
> > +AVFilterContext *filter_context;
> > +AVFilterFormatsConfig *config;
> > +enum AVMediaType media_type;
> > +int ret = 0;
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_PASSTHROUGH) {
> > +av_bprintf(bp, "All (passthrough)");
> > +return 0;
> > +}
> > +
> > +graph = avfilter_graph_alloc();
> > +if (!graph) {
> > +av_log(NULL, AV_LOG_ERROR, "Failed to create
> filtergraph\n");
> > +ret = AVERROR(ENOMEM);
> > +goto cleanup;
> > +}
> > +
> > +filter_context = avfilter_graph_alloc_filter(graph, filter,
> "filter");
> > +if (!filter_context) {
> > +av_log(NULL, AV_LOG_ERROR, "Failed to create filter\n");
> > +ret = AVERROR(ENOMEM);
> > +goto cleanup;
> > +}
> > +
> > +avfilter_init_str(filter_context, NULL);
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC)
> > +av_bprintf(bp, "Dynamic");
> > +
> > +if (!for_output && pad_index >= filter_context->nb_inputs
> > +|| for_output && pad_index >= filter_context->nb_outputs)
> > +goto cleanup;
> > +
> > +avfilter_graph_config(graph, graph);
> > +
> > +for (unsigned i = 0; i < filter_context->nb_inputs; i++)
> > +filter_context->inputs[i] = (AVFilterLink
> > *)av_mallocz(sizeof(AVFilterLink));
> > +
> > +for (unsigned i = 0; i < filter_context->nb_outputs; i++)
> > +filter_context->outputs[i] = (AVFilterLink
> > *)av_mallocz(sizeof(AVFilterLink));
> > +
> > +ff_filter_query_formats(filter_context);
> > +
> > +config = for_output ? _context->outputs[pad_index]-
> >incfg :
> > _context->inputs[pad_index]->outcfg;
> > +
> > +if (!config || !config->formats)
> > +goto cleanup;
> > +
> > +media_type= for_output ? filter->outputs[pad_index].type :
> > filter->inputs[pad_index].type;
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC) {
> > +if (config->formats && config->formats->nb_formats)
> > +av_bprintf(bp, ", Default: ");
> > +}
> > +
> > +if (config->formats == NULL)
> > +av_bprintf(bp, "unknown");
> > +else if (media_type == AVMEDIA_TYPE_VIDEO &&
> > config->formats->nb_formats == get_nb_pix_fmts() ||
> > + media_type == AVMEDIA_TYPE_AUDIO &&
> > config->formats->nb_formats == get_nb_sample_fmts())
> > +av_bprintf(bp, "All");
> > +else {
> > +for (unsigned i = 0; i < config->formats->nb_formats; i++)
Re: [FFmpeg-devel] [PATCH 0/3] Add option to log timing
Ping again Thanks > -Original Message- > From: ffmpegagent > Sent: Wednesday, August 24, 2022 9:38 PM > To: ffmpeg-devel@ffmpeg.org > Cc: softworkz > Subject: [PATCH 0/3] Add option to log timing > > This pathcset adds two logging flags: 'timing' and 'datetiming'. > > Usage: > > ffmpeg -loglevel +timing > > or > > ffmpeg -loglevel +datetiming > > softworkz (3): > avutil/log: support logging of date and timing information > fftools/opt_common: add timing and datetiming log flags > doc/fftools-common-opts: document log timing flags > > doc/APIchanges | 3 +++ > doc/fftools-common-opts.texi | 4 > fftools/opt_common.c | 12 > libavutil/log.c | 32 +--- > libavutil/log.h | 10 ++ > libavutil/version.h | 4 ++-- > 6 files changed, 60 insertions(+), 5 deletions(-) > > > base-commit: 48cb2c7a8a2deca40dd2f143848dd5addc25465c > Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr- > ffstaging-37%2Fsoftworkz%2Fsubmit_logtiming-v1 > Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr- > ffstaging-37/softworkz/submit_logtiming-v1 > Pull-Request: https://github.com/ffstaging/FFmpeg/pull/37 > -- > ffmpeg-codebot ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Ping on help output, logging and VAAPI overlay
Friendly reminder on these two: > Print filter input/output formats in help output > https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7737 > > > Add option to log timing > https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7290 Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation without DCE
> -Original Message- > From: ffmpeg-devel On Behalf Of > Carl Eugen Hoyos > Sent: Sunday, November 27, 2022 6:45 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation > without DCE > > Am So., 27. Nov. 2022 um 18:16 Uhr schrieb Soft Works > : > > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Carl Eugen Hoyos > > > Sent: Sunday, November 27, 2022 5:47 PM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation > > > without DCE > > Please consider fixing your mail client. What did it do wrong? > > > Am So., 27. Nov. 2022 um 17:29 Uhr schrieb Soft Works > > > : > > > > > > > How do you set this up when you are compiling with MSVC? > > > > > > My configure line to compile with MSVC is: > > > $ src/configure --enable-gpl --toolchain=msvc --host-os=win32 > > > > > > But maybe I misunderstand your question? > > > From what I remember, running configure in MSYS literally > took ages, I don't know if this has improved. Nope. It's still as horribly long as it was, even with everything update to latest.. > > What I meant is the environment. Are you running this from > > an MSYS shell? Or WSL? > > WSL Ah alright. Probably a better choice when all you need is the scripting environment. > > And does running configure output VS project files > > I don't think so. Sorry, I must have confused this with some external script or something. I was asking because when it's about msvc, I'm used to picture someone would be using VS as an IDE. So, yes, the plain msvc compiler doesn't have a problem with DCE. The problem is when creating projects to compile and work with the code in the VS IDE. And what VS does is pre-parsing the code in the background while you're working on it, so you don't really need to compile as you see errors and warning while you write them. As there's no actual linking being done, this procedure cannot "simulate" the DCE process and it can only assume that those symbols that DCE would eliminate are missing symbols. The result is that you cannot work in the IDE because it is full of errors due to this. Likely, this is also the use case of the submitter of this patch. I'm using VS as well, not for final compilation but for writing and debugging as this allows me to work in the way I want to work. Currently, I'm using Matt Oliver's VS project generator, which does a really good job in working around these problems by creating empty definitions for all these loose ends that are missing (as long as a final compilation hasn't determined that they aren't actually needed. But this does also have drawbacks: when changes are made in the ffmpeg base, those definitions may not match anymore and the projects need to be re-generated (which is painful). And the same applies when switching branches, so I usually have multiple sets of VS project/solutions, but these still get outdated and I need to re-create again. For these reasons, I would very much welcome when this situation could be improved. And not only for those but I also had a situation at some time where I wasted a lot of time and got quite desperate on something that turned out to be caused by the code relying on DCE - which wasn't clear to me in that situation. When you know about it, then you can live with it, but when not, then it just adds to the experience for those who are new to ffmpeg. Also, I'm not sure whether that is a reasonable use case for DCE, that it allows you to spare writing a few conditionals in the code. Anyway, I would find it nice if the code would not rely on DCE anymore. Best wishes, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation without DCE
> -Original Message- > From: ffmpeg-devel On Behalf Of > Hendrik Leppkes > Sent: Sunday, November 27, 2022 6:51 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation > without DCE > > > > > And does running configure output VS project files, so that > > you can compile inside VS (I think this existed at some > > earlier time at least, but I was under the impression that > > it's broken)..? > > > > This was never a feature, and likely never will be. We have our own > build system and its the only supported one. > The compiler is the same regardless, the msvc compiler. Then I have probably mixed this up with some tool or script who did that. I remember it was supposed to create VS project files in a subfolder named Build (not ffbuild). (and it was many years ago) Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation without DCE
> -Original Message- > From: ffmpeg-devel On Behalf Of > Carl Eugen Hoyos > Sent: Sunday, November 27, 2022 5:47 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation > without DCE > > Am So., 27. Nov. 2022 um 17:29 Uhr schrieb Soft Works > : > > > How do you set this up when you are compiling with MSVC? > > My configure line to compile with MSVC is: > $ src/configure --enable-gpl --toolchain=msvc --host-os=win32 > > But maybe I misunderstand your question? What I meant is the environment. Are you running this from an MSYS shell? Or WSL? And does running configure output VS project files, so that you can compile inside VS (I think this existed at some earlier time at least, but I was under the impression that it's broken)..? Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation without DCE
> -Original Message- > From: ffmpeg-devel On Behalf Of > Carl Eugen Hoyos > Sent: Sunday, November 27, 2022 4:52 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 0/5] Fix FFmpeg compilation > without DCE > > Am Sa., 26. Nov. 2022 um 20:17 Uhr schrieb L. E. Segovia > : > > > > Hi again, > > > > Pinging again for review. I've asked for clarification about how > should > > the EXTERNAL_ checks be dealt with, but I've not received any > > response here. > > As said before: > I regularly compile with MSVC without this patch set and I believe > the patch set makes our code less readable and harder to maintain. > > Carl Eugen > ___ IMO, relying on DCE makes the code less accessible and understandable, than having explicit declarations that are clearly indicating which code is included under which conditions. How do you set this up when you are compiling with MSVC? Best, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Saturday, November 26, 2022 3:36 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 11:31 PM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Saturday, November 26, 2022 2:01 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > support > >> P010 format > >> > >> On 11/25/2022 9:58 PM, Soft Works wrote: > >>> > >>> > >>>> -Original Message- > >>>> From: ffmpeg-devel On Behalf > Of > >>>> James Almer > >>>> Sent: Saturday, November 26, 2022 12:58 AM > >>>> To: ffmpeg-devel@ffmpeg.org > >>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > >> support > >>>> P010 format > >>>> > >>>> On 11/25/2022 8:51 PM, Soft Works wrote: > >>>>> > >>>>> > >>>>>> -Original Message- > >>>>>> From: ffmpeg-devel On Behalf > >> Of > >>>>>> James Almer > >>>>>> Sent: Saturday, November 26, 2022 12:35 AM > >>>>>> To: ffmpeg-devel@ffmpeg.org > >>>>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > >>>> support > >>>>>> P010 format > >>>>>> > >>>>>> On 11/25/2022 8:26 PM, Dong, Ruijing wrote: > >>>>>>> [AMD Official Use Only - General] > >>>>>>> > >>>>>>> Will it make sense to accept P010 input, however encode to > h264 > >>>>>> 8bit? > >>>>>> > >>>>>> If it works (the encoder accepts the 10 bit input even if it > >>>> encodes > >>>>>> it > >>>>>> as 8bit), then i don't see why not. I assume it would also be > >>>> faster > >>>>>> than using swscale to convert said 10bit input to nv12 before > >>>> passing > >>>>>> that to the encoder. > >>>>>> > >>>>>> Removing support for a pixel format as input in an encoder > needs > >> a > >>>>>> reason other than "It's rarely used", more so when it's a > single > >>>>>> line. > >>>>>> It either needs to not work, or somehow get in the way of > >> further > >>>>>> improvements. > >>>>> > >>>>> Oh sorry, I noticed that there was a misunderstanding. > >>>>> > >>>>> When I said "It's rarely used", I didn't mean that as a > >>>> justification > >>>>> for the removal, it was meant as an explanation why none of the > >>>>> hwaccels has implemented it. > >>>>> > >>>>> softworkz > >>>> > >>>> Alright, then i'll repeat my question: Does it work? > >>> > >>> No. > >> > >> What does this encoder currently do when you feed it p010 input? > What > >> does it output? > > > > An error: > > > > > > 1. 10bit from HW context: > > > > > > [graph 0 video input from stream 0:0 @ 01dc301aeec0] w:3840 > h:2160 pixfmt:yuv420p10le tb:1/6 fr:6/1001 sar:1/1 > > [auto_scale_0 @ 01dc2362a700] w:iw h:ih flags:'' interl:0 > > [hwupload@f1 @ 01dc2944ef00] auto-inserting filter > 'auto_scale_0' between the filter 'graph 0 video input from stream > 0:0' and the filter 'hwupload@f1' > > [auto_scale_0 @ 01dc2362a700] w:3840 h:2160 fmt:yuv420p10le > sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0 > > [AVHWDeviceContext @ 01dc444f9a00] D3D11 Init > > [AVHWDeviceContext @ 01dc444fab80] D3D11 Init > > [vpp_qsv@f2 @ 01dc22a3d880] VPP: input is video memory surface > > [vpp_qsv@f2 @ 01dc22a3d880] VPP: output is video memory surface > > [auto_scale_0 @ 01dc2362a700] w:3840 h:2160 fmt:yuv420p10le > sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0 > > Last message repeated 2 times > > [h264_qsv @ 01dc161b6040] Using input frames context (format > qsv) with h264_qsv encoder. > > [h264_qsv @ 01dc161
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Saturday, November 26, 2022 2:01 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 9:58 PM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Saturday, November 26, 2022 12:58 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > support > >> P010 format > >> > >> On 11/25/2022 8:51 PM, Soft Works wrote: > >>> > >>> > >>>> -Original Message- > >>>> From: ffmpeg-devel On Behalf > Of > >>>> James Almer > >>>> Sent: Saturday, November 26, 2022 12:35 AM > >>>> To: ffmpeg-devel@ffmpeg.org > >>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > >> support > >>>> P010 format > >>>> > >>>> On 11/25/2022 8:26 PM, Dong, Ruijing wrote: > >>>>> [AMD Official Use Only - General] > >>>>> > >>>>> Will it make sense to accept P010 input, however encode to h264 > >>>> 8bit? > >>>> > >>>> If it works (the encoder accepts the 10 bit input even if it > >> encodes > >>>> it > >>>> as 8bit), then i don't see why not. I assume it would also be > >> faster > >>>> than using swscale to convert said 10bit input to nv12 before > >> passing > >>>> that to the encoder. > >>>> > >>>> Removing support for a pixel format as input in an encoder needs > a > >>>> reason other than "It's rarely used", more so when it's a single > >>>> line. > >>>> It either needs to not work, or somehow get in the way of > further > >>>> improvements. > >>> > >>> Oh sorry, I noticed that there was a misunderstanding. > >>> > >>> When I said "It's rarely used", I didn't mean that as a > >> justification > >>> for the removal, it was meant as an explanation why none of the > >>> hwaccels has implemented it. > >>> > >>> softworkz > >> > >> Alright, then i'll repeat my question: Does it work? > > > > No. > > What does this encoder currently do when you feed it p010 input? What > does it output? An error: 1. 10bit from HW context: [graph 0 video input from stream 0:0 @ 01dc301aeec0] w:3840 h:2160 pixfmt:yuv420p10le tb:1/6 fr:6/1001 sar:1/1 [auto_scale_0 @ 01dc2362a700] w:iw h:ih flags:'' interl:0 [hwupload@f1 @ 01dc2944ef00] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 video input from stream 0:0' and the filter 'hwupload@f1' [auto_scale_0 @ 01dc2362a700] w:3840 h:2160 fmt:yuv420p10le sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0 [AVHWDeviceContext @ 01dc444f9a00] D3D11 Init [AVHWDeviceContext @ 01dc444fab80] D3D11 Init [vpp_qsv@f2 @ 01dc22a3d880] VPP: input is video memory surface [vpp_qsv@f2 @ 01dc22a3d880] VPP: output is video memory surface [auto_scale_0 @ 01dc2362a700] w:3840 h:2160 fmt:yuv420p10le sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0 Last message repeated 2 times [h264_qsv @ 01dc161b6040] Using input frames context (format qsv) with h264_qsv encoder. [h264_qsv @ 01dc161b6040] Encoder: input is video memory surface [h264_qsv @ 01dc161b6040] Using the average variable bitrate (AVBR) ratecontrol method [h264_qsv @ 01dc161b6040] Current pixel format is unsupported [h264_qsv @ 01dc161b6040] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters. Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height [AVHWDeviceContext @ 01dc444f9a00] D3D11 Uninit [AVIOContext @ 01dc16197c80] Statistics: 0 bytes written, 0 seeks, 0 writeouts [AVHWDeviceContext @ 01dc444fab80] D3D11 Uninit [AVIOContext @ 01dc161839c0] Statistics: 131146 bytes read, 2 seeks Conversion failed! 2. 10bit from SW context: [graph 0 video input from stream 0:0 @ 019e915dee00] w:3840 h:2160 pixfmt:yuv420p10le tb:1/6 fr:6/1001 sar:1/1 [auto_scale_0 @ 019ee99936c0] w:iw h:ih flags:'' interl:0 [format @ 019ee9993240] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format' [auto_scale_0 @ 019ee
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Saturday, November 26, 2022 12:58 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 8:51 PM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Saturday, November 26, 2022 12:35 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > support > >> P010 format > >> > >> On 11/25/2022 8:26 PM, Dong, Ruijing wrote: > >>> [AMD Official Use Only - General] > >>> > >>> Will it make sense to accept P010 input, however encode to h264 > >> 8bit? > >> > >> If it works (the encoder accepts the 10 bit input even if it > encodes > >> it > >> as 8bit), then i don't see why not. I assume it would also be > faster > >> than using swscale to convert said 10bit input to nv12 before > passing > >> that to the encoder. > >> > >> Removing support for a pixel format as input in an encoder needs a > >> reason other than "It's rarely used", more so when it's a single > >> line. > >> It either needs to not work, or somehow get in the way of further > >> improvements. > > > > Oh sorry, I noticed that there was a misunderstanding. > > > > When I said "It's rarely used", I didn't mean that as a > justification > > for the removal, it was meant as an explanation why none of the > > hwaccels has implemented it. > > > > softworkz > > Alright, then i'll repeat my question: Does it work? No. > If it does > not, > then that should have been mentioned in the patch as the reason why > this > is being done. For me it was clear, albeit a bit short. Anyway, I'll let Haihao, respond. Best, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Saturday, November 26, 2022 12:35 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 8:26 PM, Dong, Ruijing wrote: > > [AMD Official Use Only - General] > > > > Will it make sense to accept P010 input, however encode to h264 > 8bit? > > If it works (the encoder accepts the 10 bit input even if it encodes > it > as 8bit), then i don't see why not. I assume it would also be faster > than using swscale to convert said 10bit input to nv12 before passing > that to the encoder. > > Removing support for a pixel format as input in an encoder needs a > reason other than "It's rarely used", more so when it's a single > line. > It either needs to not work, or somehow get in the way of further > improvements. Oh sorry, I noticed that there was a misunderstanding. When I said "It's rarely used", I didn't mean that as a justification for the removal, it was meant as an explanation why none of the hwaccels has implemented it. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Saturday, November 26, 2022 12:25 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 8:20 PM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Friday, November 25, 2022 8:48 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > support > >> P010 format > >> > >> On 11/25/2022 3:03 PM, Soft Works wrote: > >>> > >>> > >>>> -Original Message- > >>>> From: ffmpeg-devel On Behalf > Of > >>>> Anton Khirnov > >>>> Sent: Friday, November 25, 2022 2:46 PM > >>>> To: FFmpeg development discussions and patches >>>> de...@ffmpeg.org> > >>>> Cc: Haihao Xiang > >>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > >> support > >>>> P010 format > >>>> > >>>> Why? > >>> > >>> It's rarely used. There's not even a hwaccel that can decode > this. > >> > >> The dxva2/d3d11 hwaccels seemingly do > > > > No, only for HEVC and VP9. Probably for AV1 as well in the near > future. > > Well, then that is a good reason to not remove support in this > encoder > for a pixel format that not only our software scaler can generate, > but > decoders using hwaccel backends can output too. That would be right when QSV would support it, but it doesn't, neither for decoding nor for encoding and there are no plans to implement it. I know this, because I had checked back with Intel on it, as I had been wondering about the existence of the MFX_PROFILE_AVC_HIGH10 constant, which is the profile for H.264 10bit encoding. It existed because an earlier version of MSDK had supported this, but only with their software implementations of H.264, never with HWA. And unlikely with ffmpeg as that constant has never been used there. Nvidia doesn't support it either. Neither decoding nor encoding. They don't even have a GUID constant mapping for the profile. https://www.ffmpeg.org/doxygen/3.2/nvEncodeAPI_8h_source.html (Line 151-) Best wishes, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Friday, November 25, 2022 8:48 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > On 11/25/2022 3:03 PM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Anton Khirnov > >> Sent: Friday, November 25, 2022 2:46 PM > >> To: FFmpeg development discussions and patches >> de...@ffmpeg.org> > >> Cc: Haihao Xiang > >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't > support > >> P010 format > >> > >> Why? > > > > It's rarely used. There's not even a hwaccel that can decode this. > > The dxva2/d3d11 hwaccels seemingly do No, only for HEVC and VP9. Probably for AV1 as well in the near future. But not for H.264. Not for the 3 decoder GUIDs that ffmpeg supports.. https://www.ffmpeg.org/doxygen/5.1/dxva2_8c_source.html (Line 39-41) ...neither for any of the others which are known. I'm doing some comprehensive detection of HW capabilities, which includes enumerating DXVA2/DX11VA decoders and "render targets" (which map to color formats). I have a large collection of such detection logs and I've never seen any H.264 DXVA2/DX11VA decoder which would support 10bit decoding. Here's an example how it looks like for a system with contemporary Nvidia and Intel GPUs: "Detection Log", "Debug: Create DXGI Factory", "Debug: DXGI Factory created", "Debug: Detected 3 video adapters.", "Info: Adapter #0: 'NVIDIA Quadro RTX 4000' Id:7857 (Driver: , Vendor: 4318)", "Debug: Device successfully created", "Debug: VideoDevice successfully created", "Info: 86695f12-340e-4f04-9fd3-9253dd327460: DXVA2_ModeMPEG2and1_VLD MPEG-2 & MPEG-1 variable-length decoder - mpeg2video", "Info: ee27417f-5e28-4e65-beea-1d26b508adc9: DXVA2_ModeMPEG2_VLD MPEG-2 variable-length decoder - mpeg2video", "Info: 6f3ec719-3735-42cc-8063-65cc3cb36616: DXVA2_ModeMPEG1_VLD MPEG-1 variable-length decoder - mpeg1video", "Info: 1b81bea4-a0c7-11d3-b984-00c04f2e73c5: DXVA2_ModeVC1_D2010 VC-1 variable-length decoder (2010) - vc1", "Info: 1b81bea3-a0c7-11d3-b984-00c04f2e73c5: DXVA2_ModeVC1_D VC-1 variable-length decoder - vc1", "Info: 32fcfe3f-de46-4a49-861b-ac71110649d5: - ", "Info: d79be8da-0cf1-4c81-b82a-69a4e236f43d: DXVA2_ModeH264_VLD_Stereo_Progressive_NoFGT H.264 MVC variable-length decoder, stereo, progressive - h264", "Info: f9aaccbb-c2b6-4cfc-8779-5707b1760552: DXVA2_ModeH264_VLD_Stereo_NoFGT H.264 MVC variable-length decoder, stereo - h264", "Info: 1b81be68-a0c7-11d3-b984-00c04f2e73c5: DXVA2_ModeH264_E H.264 variable-length decoder, no film grain technology - h264", "Info: 5b11d51b-2f4c-4452-bcc3-09f2a1160cc0: DXVA2_ModeHEVC_VLD_Main H.265 variable-length decoder, Main profile - hevc", "Info: 107af0e0-ef1a-4d19-aba8-67a163073d13: DXVA2_ModeHEVC_VLD_Main10 H.265 variable-length decoder, Main 10 profile - hevc", "Info: 20bb8b0a-97aa-4571-8e99-64e60606c1a6: - ", "Info: 15df9b21-06c4-47f1-841e-a67c97d7f312: - ", "Info: efd64d74-c9e8-41d7-a5e9-e9b0e39fa319: DXVA2_ModeMPEG4pt2_VLD_Simple MPEG-4 Part 2 variable-length decoder, Simple profile - mpeg4", "Info: ed418a9f-010d-4eda-9ae3-9a65358d8d2e: DXVA2_ModeMPEG4pt2_VLD_AdvSimple_NoGMC MPEG-4 Part 2 variable-length decoder, Simple & Advanced profile, no GMC - mpeg4", "Info: 9947ec6f-689b-11dc-a320-0019dbbc4184: DXVA2_nVidia_MPEG4_ASP MPEG-4 Part 2 nVidia bitstream decoder - mpeg4", "Info: 33fcfe41-de46-4a49-861b-ac71110649d5: - ", "Info: 463707f8-a1d0-4585-876d-83aa6d60b89e: DXVA2_ModeVP9_VLD_Profile0 VP9 Profile 0 - vp9", "Info: a4c749ef-6ecf-48aa-8448-50a7a1165ff7: DXVA2_ModeVP9_VLD_10bit_Profile2 VP9 10 bit Profile 2 - vp9", "Info: dda19dc7-93b5-49f5-a9b3-2bda28a2ce6e: - ", "Info: 6affd11e-1d96-42b1-a215-93a31f09a53d: - ", "Info: 914c84a3-4078-4fa9-984c-e2f262cb5c9c: - ", "Info: 8a1a1031-29bc-46d0-a007-e9b092ca6767: - ", "Debug:DXVA2_ModeMPEG2and1_VLD - ColorFormat #0: NV12 => nv12", "Debug: Supported Resolutions: 160x120, 176x144, 320x240, 352x288, 720x480, 720x240, 720x576, 1280x720, 1920x1080, 2048x1024, 2048x1080, 3140x2160, 3680x1536, 3840x2160", "Debug: Unsupported Resolutions: 4096x2048, 4096x2160, 4096x2176, 7680x4320, 8192x4320, 8192x4352", "Debug: Max Resolution: 3840x2160", "Debug:DXVA2_ModeMPEG2and1_VLD - ColorFormat #1: Opaque420 => yuv42
Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Friday, November 25, 2022 2:46 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Haihao Xiang > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support > P010 format > > Why? It's rarely used. There's not even a hwaccel that can decode this. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] lavfi/qsvvpp: provide a default framerate if needed
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Xiang, Haihao
> Sent: Thursday, November 24, 2022 10:19 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Haihao Xiang
> Subject: [FFmpeg-devel] [PATCH 3/4] lavfi/qsvvpp: provide a default
> framerate if needed
>
> From: Haihao Xiang
>
> VPP in the SDK requires the frame rate to be set to a valid value,
> otherwise init will fail, so always set a default framerate when the
> input link doesn't have a valid framerate.
>
> Signed-off-by: Haihao Xiang
> ---
> libavfilter/qsvvpp.c | 8
> 1 file changed, 8 insertions(+)
>
> diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c
> index a088f6b61f..a588a37610 100644
> --- a/libavfilter/qsvvpp.c
> +++ b/libavfilter/qsvvpp.c
> @@ -324,6 +324,14 @@ static int fill_frameinfo_by_link(mfxFrameInfo
> *frameinfo, AVFilterLink *link)
> frameinfo->CropH = link->h;
> frameinfo->FrameRateExtN = link->frame_rate.num;
> frameinfo->FrameRateExtD = link->frame_rate.den;
> +
> +/* Apparently VPP in the SDK requires the frame rate to be set
> to some value, otherwise
> + * init will fail */
> +if (frameinfo->FrameRateExtD == 0 || frameinfo->FrameRateExtN ==
> 0) {
> +frameinfo->FrameRateExtN = 25;
> +frameinfo->FrameRateExtD = 1;
> +}
> +
> frameinfo->AspectRatioW = link->sample_aspect_ratio.num ?
> link->sample_aspect_ratio.num : 1;
> frameinfo->AspectRatioH = link->sample_aspect_ratio.den ?
> link->sample_aspect_ratio.den : 1;
>
> --
LGTM. I have this in place for about a year.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/4] lavfi/qsvvpp: change the output frame's width and height
> -Original Message- > From: ffmpeg-devel On Behalf Of > Xiang, Haihao > Sent: Thursday, November 24, 2022 10:19 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Chen,Wenbin > Subject: [FFmpeg-devel] [PATCH 1/4] lavfi/qsvvpp: change the output > frame's width and height > > From: "Chen,Wenbin" > > Make sure the size of the output frame always matches the agreed upon > image size. > > Signed-off-by: Wenbin Chen > --- > libavfilter/qsvvpp.c | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c > index 8428ee89ab..bf719b2a29 100644 > --- a/libavfilter/qsvvpp.c > +++ b/libavfilter/qsvvpp.c > @@ -487,15 +487,14 @@ static QSVFrame *query_frame(QSVVPPContext *s, > AVFilterLink *outlink) > if (!out_frame->frame) > return NULL; > > -out_frame->frame->width = outlink->w; > -out_frame->frame->height = outlink->h; > - > ret = map_frame_to_surface(out_frame->frame, > _frame->surface); > if (ret < 0) > return NULL; > } > > +out_frame->frame->width = outlink->w; > +out_frame->frame->height = outlink->h; > out_frame->surface.Info = s->vpp_param.vpp.Out; > > return out_frame; > -- Which problem case does this address? Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 4/4] lavf/vf_vpp_qsv: scale_mode can be applied to color conversion
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Xiang, Haihao
> Sent: Thursday, November 24, 2022 10:20 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Haihao Xiang
> Subject: [FFmpeg-devel] [PATCH 4/4] lavf/vf_vpp_qsv: scale_mode can
> be applied to color conversion
>
> From: Haihao Xiang
>
> Signed-off-by: Haihao Xiang
> ---
> libavfilter/vf_vpp_qsv.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavfilter/vf_vpp_qsv.c b/libavfilter/vf_vpp_qsv.c
> index 4a053f9145..17f2989245 100644
> --- a/libavfilter/vf_vpp_qsv.c
> +++ b/libavfilter/vf_vpp_qsv.c
> @@ -492,7 +492,7 @@ static int config_output(AVFilterLink *outlink)
> }
> }
>
> -if (inlink->w != outlink->w || inlink->h != outlink->h) {
> +if (inlink->w != outlink->w || inlink->h != outlink->h ||
> in_format != vpp->out_format) {
> if (QSV_RUNTIME_VERSION_ATLEAST(mfx_version, 1, 19)) {
> memset(>scale_conf, 0, sizeof(mfxExtVPPScaling));
> vpp->scale_conf.Header.BufferId=
> MFX_EXTBUFF_VPP_SCALING;
> --
LGTM. But maybe the warning below should be adjusted, as it would
be confusing when it says scaling even though no scaling is
configured.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper type for NALU type
> -Original Message- > From: Andreas Rheinhardt > Sent: Wednesday, November 23, 2022 4:20 AM > To: Soft Works ; FFmpeg development > discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use > proper type for NALU type > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Andreas Rheinhardt > >> Sent: Sunday, July 3, 2022 12:21 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Cc: Andreas Rheinhardt > >> Subject: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper > >> type for NALU type > >> > >> Signed-off-by: Andreas Rheinhardt > >> --- > > > > Nice! That's helpful for the QSV SEI parsing. The one missing bit > > is the HDR data (AVMasteringDisplayMetadata and > AVContentLightMetadata) > > assignment which still seems to remain in hevcdec. Would it make > sense > > for factor this out as well? > > > > While the H.264 and HEVC syntax and semantics for these coincide, the > HEVC implementation of it is not completely spec-compliant with > respect > to its persistency: The persistence ends when the coded video > sequence > ends and if the new coded video sequence does not have these SEIs, > then > we would need to attach an AVMasteringDisplayMetadata and/or > AVContentLightMetadata just to cancel the earlier values. I just > don't > want to enter this and this includes not extending it to H.264. Just to clarify - by "factoring out" AVMasteringDisplayMetadata and AVContentLightMetadata, I didn't mean in a way that it gets unified to handle H.264 and H.265. I just meant to move it into a separate module, so that it can be used by qsvdec.c without needing to link to the whole H.265 implementation. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper type for NALU type
> -Original Message- > From: Andreas Rheinhardt > Sent: Wednesday, November 23, 2022 4:20 AM > To: Soft Works ; FFmpeg development > discussions and patches > Subject: Re: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use > proper type for NALU type > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Andreas Rheinhardt > >> Sent: Sunday, July 3, 2022 12:21 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Cc: Andreas Rheinhardt > >> Subject: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper > >> type for NALU type > >> > >> Signed-off-by: Andreas Rheinhardt > >> --- > > > > Nice! That's helpful for the QSV SEI parsing. The one missing bit > > is the HDR data (AVMasteringDisplayMetadata and > AVContentLightMetadata) > > assignment which still seems to remain in hevcdec. Would it make > sense > > for factor this out as well? > > > > While the H.264 and HEVC syntax and semantics for these coincide, the > HEVC implementation of it is not completely spec-compliant with > respect > to its persistency: The persistence ends when the coded video > sequence > ends and if the new coded video sequence does not have these SEIs, > then > we would need to attach an AVMasteringDisplayMetadata and/or > AVContentLightMetadata just to cancel the earlier values. I just > don't > want to enter this and this includes not extending it to H.264. > Anyway, I have now rebased this patchset on top of master: > https://github.com/mkver/FFmpeg/tree/h2645_sei (with no noteworthy > conflicts; basically the only change is that the second to last > commit > now adds proper newlines to the newly created files). I intend to > apply > it in two days unless there are objections. > > - Andreas Hi, I'm not sure whether you seen the patchset I had submitted meanwhile? It doesn't go as far as yours in a way that I didn't try to merge H264/5 functionality, but now it consequently splits out the bits that are meant for common use. I was able to eliminate all those makefile dependencies which you had criticized (for valid reasons). I hadn't seen any hint from you whether you are intending to proceed with your patchset, that's why I didn't know whether I should have waited. But I don't mind when you apply yours now, I will look into it and adapt. Maybe you find the time to take only a quick look and let me know whether this goes into the right direction? https://github.com/ffstaging/FFmpeg/pull/31/files Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > Timo Rothenpieler > Sent: Tuesday, November 22, 2022 11:59 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > On 22.11.2022 17:02, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Timo Rothenpieler > >> Sent: Tuesday, November 22, 2022 4:16 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: > propagate > >> AVERROR_UNRECOVERABLE when needed > >> > >> > >> > >> On 22/11/2022 14:31, James Almer wrote: > >>> On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > >>>> On 22/11/2022 14:07, James Almer wrote: > >>>>> Based on a patch by Soft Works. > >>>>> > >>>>> Signed-off-by: James Almer > >>>>> --- > >>>>> libavutil/cuda_check.h | 4 > >>>>> 1 file changed, 4 insertions(+) > >>>>> > >>>>> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >>>>> index f5a9234eaf..33aaf9c098 100644 > >>>>> --- a/libavutil/cuda_check.h > >>>>> +++ b/libavutil/cuda_check.h > >>>>> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >>>>> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >>>>> err_string); > >>>>> av_log(avctx, AV_LOG_ERROR, "\n"); > >>>>> + // Not recoverable > >>>>> + if (err == CUDA_ERROR_UNKNOWN) > >>>>> + return AVERROR_UNRECOVERABLE; > >>>> > >>>> Why does specifically CUDA_ERROR_UNKNOWN get mapped to > >> unrecoverable? > >>> > >>> It's the code that Soft Works found out was returned repeatedly > no > >>> matter how many packets you fed to the encoder, which meant it > was > >> stuck > >>> in an unrecoverable state. See > >>> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > >>> > >>> If you know of cases where this error could be returned in valid > >>> recoverable scenarios that are not already handled in some other > >> way, > >>> what do you suggest could be done? > >> > >> It's more that that very much depends on the context the function > is > >> used in. > >> In the context of an active de/encoding loop, every non-success > >> return > >> would be unrecoverable. Or even during init. > >> > >> While in other places a failure just breaks a single frame and if > it > >> somehow recovery, it can continue on. Though that's usually the > >> exception. > > > > Can you give an example for this in ffmpeg where CUDA returns an > > error and ffmpeg continues? > > Almost all the filters will continue when one filtering step fails. When looking at vf_scale_cuda, all calls are checked and return AVERROR_EXTERNAL back in case of error. Where is the "one error is forgiven" handling? > > At the time when I had submitted the patch, I had replied this to > James > > after he had argued that the behavior would be correct as is and > > a user who would want ffmpeg to exit in such case, must specify > > the -xerror parameter. > > > > > > softworkz: > >> When there's an error in a filter => ffmpeg exits > >> When there's an error in an encoder => ffmpeg exits > >> When there's an error initializing a decoder or an encoder > >> => ffmpeg exits > >> > >> So why shouldn't ffmpeg exit when there's an unrecoverable error > >> in a decoder? > > > > > > AFAIR, in case of filtering and encoding, any "normal" CUDA > > error is already sufficient to cause ffmpeg to exit. > > > > > >> In any case, this function does not seem the right place to map an > >> arbitrary return code to such a result. > > > > That depends on whether you consider CUDA_ERROR_UNKNOWN as > > generally unrecoverable. When you do - like I did - then > > it's the right place IMO. > > > > If you doubt it, then yes - it wouldn't be the right place. > > Why is the function not unconditionally returning an unrecoverable > error > then? > Like, 90% of current error returns, independent of CUDA, in > filters/codecs can be considered UNRECOVERABLE then. Why "then"? I'm only considering CUDA_ERROR_UNKNOWN as unrecoverable I have no knowledge about the other errors. That's why I would make the change only affect CUDA_ERROR_UNKNOWN. > Which does make > sense that you want them to about. > But you in turn give up the informational character of error codes. That's why I wouldn't to that. > Makes me think this should be some kind of additional flag instead. And update all the hundreds of usages? I have no bias in this regard (I can easily say that because James has submitted :-) Best wishes, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Tuesday, November 22, 2022 3:41 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > On 11/22/2022 11:33 AM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Tuesday, November 22, 2022 2:31 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: > propagate > >> AVERROR_UNRECOVERABLE when needed > >> > >> On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > >>> On 22/11/2022 14:07, James Almer wrote: > >>>> Based on a patch by Soft Works. > >>>> > >>>> Signed-off-by: James Almer > >>>> --- > >>>> libavutil/cuda_check.h | 4 > >>>> 1 file changed, 4 insertions(+) > >>>> > >>>> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >>>> index f5a9234eaf..33aaf9c098 100644 > >>>> --- a/libavutil/cuda_check.h > >>>> +++ b/libavutil/cuda_check.h > >>>> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >>>> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >>>> err_string); > >>>> av_log(avctx, AV_LOG_ERROR, "\n"); > >>>> + // Not recoverable > >>>> + if (err == CUDA_ERROR_UNKNOWN) > >>>> + return AVERROR_UNRECOVERABLE; > >>> > >>> Why does specifically CUDA_ERROR_UNKNOWN get mapped to > >> unrecoverable? > >> > >> It's the code that Soft Works found out was returned repeatedly no > >> matter how many packets you fed to the encoder, which meant it was > >> stuck > >> in an unrecoverable state. See > >> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > >> > >> If you know of cases where this error could be returned in valid > >> recoverable scenarios that are not already handled in some other > way, > >> what do you suggest could be done? > > > > Thanks James, for picking this up! > > > > All I can say is that my original patch is deployed to a quite a > > number of systems and there hasn't been any case where this > > would have had an adverse effect. > > > > I hadn't reported this to Nvidia because a solution was needed > > and it was an erroneous file, so the best they could > > have probably done was to return a different error code ;-) > > > > softworkz > > Can you be more specific about what kind of erroneous file it was? > Are > we talking about a completely broken stream where no packet was valid > and even the software decoder would fail, or something that had one > invalid packet that somehow chocked the nvdec... I was able to find the conversations where this had been reported. There were three cases, two were investigated, both of which quite similar. The first case was about an mpegts "recording" from some online stream where the "recorder" was simply reconnecting on connection failure and then continued writing to the same mpegts file. It seems the server had disconnected after 30 min and the streams have changed from then on: 11:35:35.096 frame=107726 fps=371 q=29.0 size= 588032kB time=00:29:57.59 bitrate=2682.9kbits/s throttle=off speed=6.18x 11:35:35.596 frame=107907 fps=371 q=28.0 size= 589312kB time=00:30:00.62 bitrate=2684.2kbits/s throttle=off speed=6.18x 11:35:35.995 [mpeg2_cuvid @ 0x699a40] AVHWFramesContext is already initialized with incompatible parameters 11:35:35.995 [mpeg2_cuvid @ 0x699a40] ctx->cvdl->cuvidParseVideoData(ctx->cuparser, ) failed -> CUDA_ERROR_UNKNOWN: unknown error 11:35:35.995 Error while decoding stream #0:0: Generic error in an external library 11:35:35.998 [mpeg2_cuvid @ 0x699a40] ctx->cvdl->cuvidParseVideoData(ctx->cuparser, ) failed -> CUDA_ERROR_UNKNOWN: unknown error 11:35:35.998 Error while decoding stream #0:0: Generic error in an external library 11:35:36.003 [mpeg2_cuvid @ 0x699a40] ctx->cvdl->cuvidParseVideoData(ctx->cuparser, ) failed -> CUDA_ERROR_UNKNOWN: unknown error We can't know what "incompatible parameters" actually means. It could be the frame size, but it could also be a different codec (like H264 instead of MPEG2) or both, or interlaced/non-interlaced. The other case was similar. The user had eventually admitted: &qu
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > Timo Rothenpieler > Sent: Tuesday, November 22, 2022 4:16 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > > > On 22/11/2022 14:31, James Almer wrote: > > On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > >> On 22/11/2022 14:07, James Almer wrote: > >>> Based on a patch by Soft Works. > >>> > >>> Signed-off-by: James Almer > >>> --- > >>> libavutil/cuda_check.h | 4 > >>> 1 file changed, 4 insertions(+) > >>> > >>> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >>> index f5a9234eaf..33aaf9c098 100644 > >>> --- a/libavutil/cuda_check.h > >>> +++ b/libavutil/cuda_check.h > >>> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >>> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >>> err_string); > >>> av_log(avctx, AV_LOG_ERROR, "\n"); > >>> + // Not recoverable > >>> + if (err == CUDA_ERROR_UNKNOWN) > >>> + return AVERROR_UNRECOVERABLE; > >> > >> Why does specifically CUDA_ERROR_UNKNOWN get mapped to > unrecoverable? > > > > It's the code that Soft Works found out was returned repeatedly no > > matter how many packets you fed to the encoder, which meant it was > stuck > > in an unrecoverable state. See > > http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > > > > If you know of cases where this error could be returned in valid > > recoverable scenarios that are not already handled in some other > way, > > what do you suggest could be done? > > It's more that that very much depends on the context the function is > used in. > In the context of an active de/encoding loop, every non-success > return > would be unrecoverable. Or even during init. > > While in other places a failure just breaks a single frame and if it > somehow recovery, it can continue on. Though that's usually the > exception. Can you give an example for this in ffmpeg where CUDA returns an error and ffmpeg continues? At the time when I had submitted the patch, I had replied this to James after he had argued that the behavior would be correct as is and a user who would want ffmpeg to exit in such case, must specify the -xerror parameter. softworkz: > When there's an error in a filter => ffmpeg exits > When there's an error in an encoder => ffmpeg exits > When there's an error initializing a decoder or an encoder > => ffmpeg exits > > So why shouldn't ffmpeg exit when there's an unrecoverable error > in a decoder? AFAIR, in case of filtering and encoding, any "normal" CUDA error is already sufficient to cause ffmpeg to exit. > In any case, this function does not seem the right place to map an > arbitrary return code to such a result. That depends on whether you consider CUDA_ERROR_UNKNOWN as generally unrecoverable. When you do - like I did - then it's the right place IMO. If you doubt it, then yes - it wouldn't be the right place. Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Tuesday, November 22, 2022 3:48 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Tuesday, November 22, 2022 2:31 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: > propagate > >> AVERROR_UNRECOVERABLE when needed > >> > >> On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > >>> On 22/11/2022 14:07, James Almer wrote: > >>>> Based on a patch by Soft Works. > >>>> > >>>> Signed-off-by: James Almer > >>>> --- > >>>> libavutil/cuda_check.h | 4 > >>>> 1 file changed, 4 insertions(+) > >>>> > >>>> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >>>> index f5a9234eaf..33aaf9c098 100644 > >>>> --- a/libavutil/cuda_check.h > >>>> +++ b/libavutil/cuda_check.h > >>>> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >>>> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >>>> err_string); > >>>> av_log(avctx, AV_LOG_ERROR, "\n"); > >>>> + // Not recoverable > >>>> + if (err == CUDA_ERROR_UNKNOWN) > >>>> + return AVERROR_UNRECOVERABLE; > >>> > >>> Why does specifically CUDA_ERROR_UNKNOWN get mapped to > >> unrecoverable? > >> > >> It's the code that Soft Works found out was returned repeatedly no > >> matter how many packets you fed to the encoder, which meant it was > >> stuck > >> in an unrecoverable state. See > >> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > >> > >> If you know of cases where this error could be returned in valid > >> recoverable scenarios that are not already handled in some other > way, > >> what do you suggest could be done? > > > > Thanks James, for picking this up! > > > > All I can say is that my original patch is deployed to a quite a > > number of systems and there hasn't been any case where this > > would have had an adverse effect. > > > > I hadn't reported this to Nvidia because a solution was needed > > and it was an erroneous file, so the best they could > > have probably done was to return a different error code ;-) > > > > If this was an erroneous file, then wouldn't it be possible for > future > packets to be non-erroneous and therefore decodable? > (An alternative fix for this would be for fftools to calculate how > many > errors there were in a row and stop trying to decode a stream that > returns too many errors based upon some option (one could also use > the > ratio of successful to non-successful decode calls or something like > that).) That error isn't a normal decode error. Those exist as well and it can recover from these. But from all experiments I did, my conclusion was that this is an error of final death. AFAIR. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Tuesday, November 22, 2022 3:41 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > On 11/22/2022 11:33 AM, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> James Almer > >> Sent: Tuesday, November 22, 2022 2:31 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: > propagate > >> AVERROR_UNRECOVERABLE when needed > >> > >> On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > >>> On 22/11/2022 14:07, James Almer wrote: > >>>> Based on a patch by Soft Works. > >>>> > >>>> Signed-off-by: James Almer > >>>> --- > >>>> libavutil/cuda_check.h | 4 > >>>> 1 file changed, 4 insertions(+) > >>>> > >>>> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >>>> index f5a9234eaf..33aaf9c098 100644 > >>>> --- a/libavutil/cuda_check.h > >>>> +++ b/libavutil/cuda_check.h > >>>> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >>>> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >>>> err_string); > >>>> av_log(avctx, AV_LOG_ERROR, "\n"); > >>>> + // Not recoverable > >>>> + if (err == CUDA_ERROR_UNKNOWN) > >>>> + return AVERROR_UNRECOVERABLE; > >>> > >>> Why does specifically CUDA_ERROR_UNKNOWN get mapped to > >> unrecoverable? > >> > >> It's the code that Soft Works found out was returned repeatedly no > >> matter how many packets you fed to the encoder, which meant it was > >> stuck > >> in an unrecoverable state. See > >> http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > >> > >> If you know of cases where this error could be returned in valid > >> recoverable scenarios that are not already handled in some other > way, > >> what do you suggest could be done? > > > > Thanks James, for picking this up! > > > > All I can say is that my original patch is deployed to a quite a > > number of systems and there hasn't been any case where this > > would have had an adverse effect. > > > > I hadn't reported this to Nvidia because a solution was needed > > and it was an erroneous file, so the best they could > > have probably done was to return a different error code ;-) > > > > softworkz > > Can you be more specific about what kind of erroneous file it was? It's been a while, so I would need to look up what it was. > Are > we talking about a completely broken stream where no packet was valid > and even the software decoder would fail, or something that had one > invalid packet that somehow chocked the nvdec to the point not even > an > IDR picture triggering a refresh would fix it? > > If this is the former, then what you encountered was not the decoder > entering an unrecoverable state, but just properly rejecting bad > input, > and then this patch would probably not be correct. What I remember is that other decoders could read over it and recover. But not the Nvidia decoder - no matter what it was being fed. It was a point of no return. It kept getting fed packet after packet but it never recovered, and by "never" I mean even as long as that the ffmpeg grew to 50 GB. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate AVERROR_UNRECOVERABLE when needed
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Tuesday, November 22, 2022 2:31 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH 3/4] avutil/cuda_check: propagate > AVERROR_UNRECOVERABLE when needed > > On 11/22/2022 10:21 AM, Timo Rothenpieler wrote: > > On 22/11/2022 14:07, James Almer wrote: > >> Based on a patch by Soft Works. > >> > >> Signed-off-by: James Almer > >> --- > >> libavutil/cuda_check.h | 4 > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/libavutil/cuda_check.h b/libavutil/cuda_check.h > >> index f5a9234eaf..33aaf9c098 100644 > >> --- a/libavutil/cuda_check.h > >> +++ b/libavutil/cuda_check.h > >> @@ -49,6 +49,10 @@ static inline int ff_cuda_check(void *avctx, > >> av_log(avctx, AV_LOG_ERROR, " -> %s: %s", err_name, > >> err_string); > >> av_log(avctx, AV_LOG_ERROR, "\n"); > >> + // Not recoverable > >> + if (err == CUDA_ERROR_UNKNOWN) > >> + return AVERROR_UNRECOVERABLE; > > > > Why does specifically CUDA_ERROR_UNKNOWN get mapped to > unrecoverable? > > It's the code that Soft Works found out was returned repeatedly no > matter how many packets you fed to the encoder, which meant it was > stuck > in an unrecoverable state. See > http://ffmpeg.org/pipermail/ffmpeg-devel/2021-October/287153.html > > If you know of cases where this error could be returned in valid > recoverable scenarios that are not already handled in some other way, > what do you suggest could be done? Thanks James, for picking this up! All I can say is that my original patch is deployed to a quite a number of systems and there hasn't been any case where this would have had an adverse effect. I hadn't reported this to Nvidia because a solution was needed and it was an erroneous file, so the best they could have probably done was to return a different error code ;-) softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v6 3/3] avcodec/qsvdec: Implement SEI parsing for QSV decoders
> -Original Message-
> From: Xiang, Haihao
> Sent: Monday, November 21, 2022 3:45 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: softwo...@hotmail.com; kier...@obe.tv; haihao.xiang-at-
> intel@ffmpeg.org; andreas.rheinha...@outlook.com
> Subject: Re: [FFmpeg-devel] [PATCH v6 3/3] avcodec/qsvdec: Implement
> SEI parsing for QSV decoders
>
> On Tue, 2022-10-25 at 04:03 +, softworkz wrote:
> > From: softworkz
> >
> > Signed-off-by: softworkz
> > ---
> > libavcodec/Makefile | 2 +-
> > libavcodec/qsvdec.c | 321
>
> > 2 files changed, 322 insertions(+), 1 deletion(-)
> >
> > diff --git a/libavcodec/Makefile b/libavcodec/Makefile
> > index 90c7f113a3..cbddbb0ace 100644
> > --- a/libavcodec/Makefile
> > +++ b/libavcodec/Makefile
> > @@ -146,7 +146,7 @@ OBJS-$(CONFIG_MSS34DSP)+=
> mss34dsp.o
> > OBJS-$(CONFIG_PIXBLOCKDSP) += pixblockdsp.o
> > OBJS-$(CONFIG_QPELDSP) += qpeldsp.o
> > OBJS-$(CONFIG_QSV) += qsv.o
> > -OBJS-$(CONFIG_QSVDEC) += qsvdec.o
> > +OBJS-$(CONFIG_QSVDEC) += qsvdec.o h264_sei.o
> hevc_sei.o
> > OBJS-$(CONFIG_QSVENC) += qsvenc.o
> > OBJS-$(CONFIG_RANGECODER) += rangecoder.o
> > OBJS-$(CONFIG_RDFT)+= rdft.o
> > diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> > index 73405b5747..467a248224 100644
> > --- a/libavcodec/qsvdec.c
> > +++ b/libavcodec/qsvdec.c
> > @@ -41,6 +41,7 @@
> > #include "libavutil/time.h"
> > #include "libavutil/imgutils.h"
> > #include "libavutil/film_grain_params.h"
> > +#include
> >
> > #include "avcodec.h"
> > #include "codec_internal.h"
> > @@ -49,6 +50,9 @@
> > #include "hwconfig.h"
> > #include "qsv.h"
> > #include "qsv_internal.h"
> > +#include "h264_sei.h"
> > +#include "hevc_ps.h"
> > +#include "hevc_sei.h"
> >
> > #if QSV_ONEVPL
> > #include
> > @@ -66,6 +70,8 @@ static const AVRational mfx_tb = { 1, 9 };
> > AV_NOPTS_VALUE : pts_tb.num ? \
> > av_rescale_q(mfx_pts, mfx_tb, pts_tb) : mfx_pts)
> >
> > +#define PAYLOAD_BUFFER_SIZE 65535
> > +
> > typedef struct QSVAsyncFrame {
> > mfxSyncPoint *sync;
> > QSVFrame *frame;
> > @@ -107,6 +113,9 @@ typedef struct QSVContext {
> >
> > mfxExtBuffer **ext_buffers;
> > int nb_ext_buffers;
> > +
> > +mfxU8 payload_buffer[PAYLOAD_BUFFER_SIZE];
> > +AVBufferRef *a53_buf_ref;
> > } QSVContext;
> >
> > static const AVCodecHWConfigInternal *const qsv_hw_configs[] = {
> > @@ -628,6 +637,299 @@ static int
> qsv_export_film_grain(AVCodecContext *avctx,
> > mfxExtAV1FilmGrainParam
> > }
> > #endif
> >
> > +static int find_start_offset(mfxU8 data[4])
> > +{
> > +if (data[0] == 0 && data[1] == 0 && data[2] == 1)
> > +return 3;
> > +
> > +if (data[0] == 0 && data[1] == 0 && data[2] == 0 && data[3] ==
> 1)
> > +return 4;
> > +
> > +return 0;
> > +}
> > +
> > +static int parse_sei_h264(AVCodecContext* avctx, QSVContext* q,
> AVFrame* out)
> > +{
> > +H264SEIContext sei = { 0 };
> > +GetBitContext gb = { 0 };
> > +mfxPayload payload = { 0, .Data = >payload_buffer[0],
> .BufSize =
> > sizeof(q->payload_buffer) - AV_INPUT_BUFFER_PADDING_SIZE };
> > +mfxU64 ts;
> > +int ret;
> > +
> > +while (1) {
> > +int start;
> > +memset(payload.Data, 0, payload.BufSize);
> > +
> > +ret = MFXVideoDECODE_GetPayload(q->session, ,
> );
> > +if (ret == MFX_ERR_NOT_ENOUGH_BUFFER) {
> > +av_log(avctx, AV_LOG_WARNING, "Warning: Insufficient
> buffer on
> > GetPayload(). Size: %"PRIu64" Needed: %d\n", sizeof(q-
> >payload_buffer),
> > payload.BufSize);
> > +return 0;
> > +}
> > +if (ret != MFX_ERR_NONE)
> > +return ret;
> > +
> > +if (payload.NumBit == 0 || payload.NumBit >=
> payload.BufSize * 8)
> > +break;
> > +
> > +start = find_start_offset(payload.Data);
> > +
> > +switch (payload.Type) {
> > +case SEI_TYPE_BUFFERING_PERIOD:
> > +case SEI_TYPE_PIC_TIMING:
> > +continue;
> > +}
> > +
> > +if (init_get_bits(, [start],
> payload.NumBit - start *
> > 8) < 0)
> > +av_log(avctx, AV_LOG_ERROR, "Error initializing
> bitstream reader
> > SEI type: %d Numbits %d error: %d\n", payload.Type,
> payload.NumBit, ret);
> > +else {
> > +ret = ff_h264_sei_decode(, , NULL, avctx);
> > +
> > +if (ret < 0)
> > +av_log(avctx, AV_LOG_WARNING, "Failed to parse SEI
> type:
> > %d Numbits %d error: %d\n", payload.Type, payload.NumBit, ret);
> > +else
> > +av_log(avctx, AV_LOG_DEBUG, "mfxPayload Type: %d
> Numbits
> > %d\n", payload.Type, payload.NumBit);
> > +}
> > +}
> > +
> > +if (out)
> > +return ff_h264_set_sei_to_frame(avctx, , out, NULL,
> 0);
> > +
>
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Monday, November 14, 2022 11:06 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > On Mon, Nov 14, 2022 at 05:13:40PM +0100, Anton Khirnov wrote: > > Quoting Soft Works (2022-11-14 16:13:29) > > > > I did read your document, and my takeaway message from it is > "doing > > > > it > > > > properly is too hard". As long as that continues to be your > position, > > > > you might as well not bother. > > > > > > This is ridiculous, and you know that. Or at least you would know > > > if you would have really tried to understand the problem. > > > > > > And that unfortunately applies to some others as well. Nobody is > > > willing to go deep enough to the point where it becomes clear > > > that a "perfect" solution would only be possible by making > fundamental > > > changes to libavfilter, which are complex, risky and something > > > that would never be accepted from me, even when it would be > > > the most excellent solution. > > > > Stop with the drama, please. You are not a persecuted misunderstood > > genius. Nobody here has a personal grudge against you. The reason > > people, including me, are objecting to your patches is that they > are not > > fundamental enough. You want to redo the subtitle API in a major > way, > > but keep it saddled with legacy hacks right from the start. We have > > enough of those already to know we don't want any more. > > Maybe people should take a step back and together discuss with > softworks > what changes need to be done. > > This thread is a bit odd becasue from reading just it its very clear > there > are objections but what exactly needs to be done to move this forward > is > not so clear (to me at least). To me neither. Until today. > I think a patch review should result in a clear list of things that > need > to be done. > Then these things can be gone through one by one. What can be done > what not > where there is consensus, where not and why ... > > If a request to a fundamental redesign is there then thats what it is > and maybe noone will do it but IMHO it should be clear so everyone > understands > what is requested > > As it is, this thread simply makes me sad because its deadlocked, > there is > some will and effort on one side and that isnt directed into anything > that > goes into the ffmpeg codebase ... I had a friendly chat with Anton on IRC about it. Bottom line is that he insists that I re-work libavfilter filtering from the ground up to support non-monotonous pts values in graph processing. The reason is that the single time_64t field that I want to add as a member to AVFrame would not be acceptable because it would make timestamp handling "too messy". Other objections haven't been made. I asked multiple times whether he would be serious about this, demanding me to take on a possibly multi- month work for the reason that having a single additional field in AVFrame would be messy, which he confirmed. I said it would be a huge amount of work and too much for me to take. He doubted, saying it can't be that much, I asked what he would expect me to change and how: Nov 14 19:08:02i can't say what to change exactly without actually doing it Nov 14 19:08:22i didn't write that code, I only have a very rought idea what it does Full transcript on request. I got no more words. Except about the irony that reveals when looking at the subject and who wrote it. Thanks, Michael! softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/8] fftools/ffmpeg: simplify ost_iter()
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Anton Khirnov
> Sent: Monday, November 14, 2022 4:14 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH 1/8] fftools/ffmpeg: simplify
> ost_iter()
>
> The inner loop never goes through more than 1 iteration, and so can
> be
> replaced by an if().
>
> Found-by: Andreas Rheinhardt
> ---
> fftools/ffmpeg.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
> index e6f6773f6a..0fa2fe8c52 100644
> --- a/fftools/ffmpeg.c
> +++ b/fftools/ffmpeg.c
> @@ -612,7 +612,7 @@ static OutputStream *ost_iter(OutputStream *prev)
>
> for (; of_idx < nb_output_files; of_idx++) {
> OutputFile *of = output_files[of_idx];
> -for (; ost_idx < of->nb_streams; ost_idx++)
> +if (ost_idx < of->nb_streams)
> return of->streams[ost_idx];
>
> ost_idx = 0;
> --
> 2.35.1
I'm objecting the whole patchset.
I think it's not fundamental enough.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Monday, November 14, 2022 5:40 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > Soft Works (12022-11-14): > > What are your points? Which are your objections? Please show the > > code that you think is wrong and say how it should be done instead. > > For the record, I have, multiple times. You’ve been destructive from the very first moment when you realized that I was going to do something that you had been planning for a long time. You haven't missed a single occasion to discredit me or my work, keeping it vague just like Anton does right now, to keep things at a level where credibility weighs - instead than a technical facts. Ah, wait - there's one occasion where you never jumped on, never mentioned, talked about or objected: The separate subtitle timing fields. You know exactly why I have them and why these are needed, but you would rather bite your tongue than saying it, and so you kept quiet each time when others were criticizing it. Now you'll have to say that I'm wrong. I'm curious how you'll say it. PS: I'm sorry that I got in your way, I understood that much later only. It was just that I needed this functionality, not that I would have had a choice. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Monday, November 14, 2022 5:14 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > Quoting Soft Works (2022-11-14 16:13:29) > > > I did read your document, and my takeaway message from it is > "doing > > > it > > > properly is too hard". As long as that continues to be your > position, > > > you might as well not bother. > > > > This is ridiculous, and you know that. Or at least you would know > > if you would have really tried to understand the problem. > > > > And that unfortunately applies to some others as well. Nobody is > > willing to go deep enough to the point where it becomes clear > > that a "perfect" solution would only be possible by making > fundamental > > changes to libavfilter, which are complex, risky and something > > that would never be accepted from me, even when it would be > > the most excellent solution. > > Stop with the drama, please. You are not a persecuted misunderstood > genius. Nobody here has a personal grudge against you. The reason > people, including me, are objecting to your patches is that they are > not > fundamental enough. You want to redo the subtitle API in a major way, > but keep it saddled with legacy hacks right from the start. We have > enough of those already to know we don't want any more. This is so disgusting! Why can't you just point at those "legacy hacks" and tell how you want to have it done instead? "not fundamental enough"? Where why and how? And why do you keep bullshitting me with such nonsense statements? What are your points? Which are your objections? Please show the code that you think is wrong and say how it should be done instead. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Monday, November 14, 2022 4:18 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > On 11/14/22, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Anton Khirnov > >> Sent: Monday, November 14, 2022 3:35 PM > >> To: FFmpeg development discussions and patches >> de...@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > >> the argument for submitting patches > >> > >> Quoting Soft Works (2022-11-14 12:20:00) > >> > > >> > > >> > > -Original Message- > >> > > From: ffmpeg-devel On Behalf > Of > >> > > Anton Khirnov > >> > > Sent: Monday, November 14, 2022 12:08 PM > >> > > To: FFmpeg development discussions and patches >> > > de...@ffmpeg.org> > >> > > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: > >> extend > >> > > the argument for submitting patches > >> > > > >> > > Quoting Soft Works (2022-11-14 11:46:49) > >> > > > > Sorry, but you problems are entirely self-inflicted. You > have > >> > > been > >> > > > > told what changes need to happen right from the beginning, > >> > > > > repeatedly, and by several developers independently. > >> > > > > >> > > > And those are completed and settled, like I had state > multiple > >> > > times. > >> > > > It's ready for review for months already. > >> > > > >> > > Your stating something does not make it true, no matter how > many > >> > > times > >> > > you do it. > >> > > > >> > > My objections were not addressed. > >> > > > >> > > In your last resend, Hendrik yet again raised the start_pts > >> question. > >> > > As > >> > > far as I can tell, your explanation for why it's supposedly > >> needed > >> > > did > >> > > not convince ANYONE. > >> > > >> > What means "as far as I can tell" here? Do you have something to > >> > say about it, then please do? > >> > >> It means that I am not aware of anyone who changed their stance > based > >> on > >> your arguments, but cannot prove that no such person exists. > > > > I'm afraid, but everything you are writing is making references to > > others and what they would think or what you are assuming that they > > might think. > > > >> I did read your document, and my takeaway message from it is > "doing > >> it > >> properly is too hard". As long as that continues to be your > position, > >> you might as well not bother. > > > > This is ridiculous, and you know that. Or at least you would know > > if you would have really tried to understand the problem. > > > > And that unfortunately applies to some others as well. Nobody is > > willing to go deep enough to the point where it becomes clear > > that a "perfect" solution would only be possible by making > fundamental > > changes to libavfilter, which are complex, risky and something > > that would never be accepted from me, even when it would be > > the most excellent solution. I think this is pretty clear to > > everybody here, and trying to present this in a light as if > > I would just be too lazy to go for it, is just despicable, > > I'm afraid. > > > > I wish you could stop referring to others potential opinions > > and get yourself as much into the subject as it is required to > > understand the actual problem and talk for yourself. > > > > Because I would happily discuss alternatives > > with you and follow your advice, no matter when it takes > > a little more effort - as long as it will still be possible > > to handle all cases like with the current patchset. > > But I mean substantial and detailed advice based on an > > understanding of the problems, not the kind of "no, that's > > bad, I don't believe you that it couldn't be done like I > > think it's gotta be". > > > > I will happily, gladly and friendly work and converse with > > anybody who would be so kind to leave one's peripheral > > spectator position and get down with me to the core > > problem and discuss potential solutions. > > They can not admit they have zero understanding why and how code > works > Instead they propose some nonsense that hardly can be implemented. Nobody has actually proposed anything. I wish somebody had. "No, not like this, I don't care whether and how it can be done otherwise" - is not a proposal. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Monday, November 14, 2022 4:18 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > On 11/14/22, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Anton Khirnov > >> Sent: Monday, November 14, 2022 3:35 PM > >> To: FFmpeg development discussions and patches >> de...@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > >> the argument for submitting patches > >> > >> Quoting Soft Works (2022-11-14 12:20:00) > >> > > >> > > >> > > -Original Message- > >> > > From: ffmpeg-devel On Behalf > Of > >> > > Anton Khirnov > >> > > Sent: Monday, November 14, 2022 12:08 PM > >> > > To: FFmpeg development discussions and patches >> > > de...@ffmpeg.org> > >> > > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: > >> extend > >> > > the argument for submitting patches > >> > > > >> > > Quoting Soft Works (2022-11-14 11:46:49) > >> > > > > Sorry, but you problems are entirely self-inflicted. You > have > >> > > been > >> > > > > told what changes need to happen right from the beginning, > >> > > > > repeatedly, and by several developers independently. > >> > > > > >> > > > And those are completed and settled, like I had state > multiple > >> > > times. > >> > > > It's ready for review for months already. > >> > > > >> > > Your stating something does not make it true, no matter how > many > >> > > times > >> > > you do it. > >> > > > >> > > My objections were not addressed. > >> > > > >> > > In your last resend, Hendrik yet again raised the start_pts > >> question. > >> > > As > >> > > far as I can tell, your explanation for why it's supposedly > >> needed > >> > > did > >> > > not convince ANYONE. > >> > > >> > What means "as far as I can tell" here? Do you have something to > >> > say about it, then please do? > >> > >> It means that I am not aware of anyone who changed their stance > based > >> on > >> your arguments, but cannot prove that no such person exists. > > > > I'm afraid, but everything you are writing is making references to > > others and what they would think or what you are assuming that they > > might think. > > > >> I did read your document, and my takeaway message from it is > "doing > >> it > >> properly is too hard". As long as that continues to be your > position, > >> you might as well not bother. > > > > This is ridiculous, and you know that. Or at least you would know > > if you would have really tried to understand the problem. > > > > And that unfortunately applies to some others as well. Nobody is > > willing to go deep enough to the point where it becomes clear > > that a "perfect" solution would only be possible by making > fundamental > > changes to libavfilter, which are complex, risky and something > > that would never be accepted from me, even when it would be > > the most excellent solution. I think this is pretty clear to > > everybody here, and trying to present this in a light as if > > I would just be too lazy to go for it, is just despicable, > > I'm afraid. > > > > I wish you could stop referring to others potential opinions > > and get yourself as much into the subject as it is required to > > understand the actual problem and talk for yourself. > > > > Because I would happily discuss alternatives > > with you and follow your advice, no matter when it takes > > a little more effort - as long as it will still be possible > > to handle all cases like with the current patchset. > > But I mean substantial and detailed advice based on an > > understanding of the problems, not the kind of "no, that's > > bad, I don't believe you that it couldn't be done like I > > think it's gotta be". > > > > I will happily, gladly and friendly work and converse with > > anybody who would be so kind to leave one's peripheral > > spectator position and get down with me to the core > > problem and discuss potential solutions. > > They can not admit they have zero understanding why and how code > works > Instead they propose some nonsense that hardly can be implemented. I'm not saying that and I have no doubt they could.. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Monday, November 14, 2022 3:35 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > Quoting Soft Works (2022-11-14 12:20:00) > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Anton Khirnov > > > Sent: Monday, November 14, 2022 12:08 PM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: > extend > > > the argument for submitting patches > > > > > > Quoting Soft Works (2022-11-14 11:46:49) > > > > > Sorry, but you problems are entirely self-inflicted. You have > > > been > > > > > told what changes need to happen right from the beginning, > > > > > repeatedly, and by several developers independently. > > > > > > > > And those are completed and settled, like I had state multiple > > > times. > > > > It's ready for review for months already. > > > > > > Your stating something does not make it true, no matter how many > > > times > > > you do it. > > > > > > My objections were not addressed. > > > > > > In your last resend, Hendrik yet again raised the start_pts > question. > > > As > > > far as I can tell, your explanation for why it's supposedly > needed > > > did > > > not convince ANYONE. > > > > What means "as far as I can tell" here? Do you have something to > > say about it, then please do? > > It means that I am not aware of anyone who changed their stance based > on > your arguments, but cannot prove that no such person exists. I'm afraid, but everything you are writing is making references to others and what they would think or what you are assuming that they might think. > I did read your document, and my takeaway message from it is "doing > it > properly is too hard". As long as that continues to be your position, > you might as well not bother. This is ridiculous, and you know that. Or at least you would know if you would have really tried to understand the problem. And that unfortunately applies to some others as well. Nobody is willing to go deep enough to the point where it becomes clear that a "perfect" solution would only be possible by making fundamental changes to libavfilter, which are complex, risky and something that would never be accepted from me, even when it would be the most excellent solution. I think this is pretty clear to everybody here, and trying to present this in a light as if I would just be too lazy to go for it, is just despicable, I'm afraid. I wish you could stop referring to others potential opinions and get yourself as much into the subject as it is required to understand the actual problem and talk for yourself. Because I would happily discuss alternatives with you and follow your advice, no matter when it takes a little more effort - as long as it will still be possible to handle all cases like with the current patchset. But I mean substantial and detailed advice based on an understanding of the problems, not the kind of "no, that's bad, I don't believe you that it couldn't be done like I think it's gotta be". I will happily, gladly and friendly work and converse with anybody who would be so kind to leave one's peripheral spectator position and get down with me to the core problem and discuss potential solutions. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Monday, November 14, 2022 12:08 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > Quoting Soft Works (2022-11-14 11:46:49) > > > Sorry, but you problems are entirely self-inflicted. You have > been > > > told what changes need to happen right from the beginning, > > > repeatedly, and by several developers independently. > > > > And those are completed and settled, like I had state multiple > times. > > It's ready for review for months already. [...] > > Some random quotes from IRC: > 2022-09-01 00:25:21 @Lynne elenril: I never retracted my > insistence on using the native frame fields for subtitles > 2022-09-01 00:25:27 @Lynne not sure how softworks got that idea I got that "idea" from this discussion: Jan 14 02:46:02 can't you really not hide away everything to do with repetition, subtitle pts, and subtitle duration all into the private opqaue field, and give the user what they expect when the frames go out of lavfi? Jan 14 02:46:55 worth a thought, but I'm not sure Jan 14 02:49:32 I think it's better to make it more transparent. the heartbeat mechanism has been a hidden thing and that's why it wasn't an ideal solution Jan 14 02:50:10 when you already need to understand when and why you need to insert a subfeed filter, then there's no point in hiding the effect imo Jan 14 02:50:53 I think that's worth a really good look Jan 14 02:50:56 the good thing is, that often none of that is needed at all Jan 14 02:51:15 (say sometimes..) Jan 14 02:52:04 if you could hide all of that into the opaque field, I'd be happy with the patch This IS a retraction from the "insistence on using the native frame fields for subtitles." (because the actual subtitle timings would be in that opaque field) Regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Monday, November 14, 2022 12:08 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend > the argument for submitting patches > > Quoting Soft Works (2022-11-14 11:46:49) > > > Sorry, but you problems are entirely self-inflicted. You have > been > > > told what changes need to happen right from the beginning, > > > repeatedly, and by several developers independently. > > > > And those are completed and settled, like I had state multiple > times. > > It's ready for review for months already. > > Your stating something does not make it true, no matter how many > times > you do it. > > My objections were not addressed. > > In your last resend, Hendrik yet again raised the start_pts question. > As > far as I can tell, your explanation for why it's supposedly needed > did > not convince ANYONE. What means "as far as I can tell" here? Do you have something to say about it, then please do? I cannot talk and respond to any "ghost" statements which I'm not aware of. And regarding the IRC snippet: Well, interesting, but I can neither know not respond to anything I'm not aware of. I had taken the effort to explain the reasons this in this article: https://github.com/softworkz/SubtitleFilteringDemos/issues/1 and Hendrik didn't respond. So whom should I talk to, in order to find a solution, especially, considering the fact, that nobody has really taken the effort to ACTUALLY understand what the problem is? Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, August 31, 2022 6:02 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Anton Khirnov
> > Sent: Wednesday, August 31, 2022 3:40 AM
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering
> 2022
> >
> > Quoting Soft Works (2022-08-27 00:48:01)
> > > 2. "There's no reason why this cannot be handled using the buffer
> > > and data fields"
> > >
> > > I had explained the reasons and in conversation on IRC, Lynne was
> > > no longer insisting on this AFAIR.
> >
> > I did not see this explanation, can you restate it here?
>
You had asked me to restate the explanation.
I did that (below) but you never responded.
Thanks,
softworkz
> Sure. Let's look at the AVSubtitleArea struct first:
>
>
> typedef struct AVSubtitleArea {
>
> enum AVSubtitleType type;
> int flags;
>
> int x; ///< top left corner of area.
> int y; ///< top left corner of area.
> int w; ///< widthof area.
> int h; ///< height of area.
> int nb_colors; ///< number of colors in bitmap palette (@ref
> pal).
>
> /**
> * Buffers and line sizes for the bitmap of this subtitle.
> *
> * @{
> */
> AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
> int linesize[AV_NUM_BUFFER_POINTERS];
> /**
> * @}
> */
>
> uint32_t pal[256]; ///< RGBA palette for the bitmap.
>
> char *text;///< 0-terminated plain UTF-8 text
> char *ass; ///< 0-terminated ASS/SSA compatible event
> line.
>
> } AVSubtitleArea;
>
>
>
> These structs are stored in AVFrame like this:
>
>
> /**
> * Number of items in the @ref subtitle_areas array.
> */
> unsigned num_subtitle_areas;
>
> /**
> * Array of subtitle areas, may be empty.
> */
> AVSubtitleArea **subtitle_areas;
>
>
>
> The question was "why this cannot be handled using the buffer
> and data fields?" - there are multiple reasons:
>
> 1. Area Count
>
> In ASS subtitles it's not uncommon that animations are defined
> through subtitle events (regular ass events). This can go as
> far as that dust particles are flying around on the screen and
> each particle has its own subtitle event/line which defines
> how it flies from x to y and how fast, etc.
> Not yet, but in a future update, the ass decoder should be
> improved in a way that it doesn't emit an individual AVFrame
> for each event line but bundles subsequent events together
> when these would have the same start time and duration.
> As a result, we can have AVFrames with dozens or even a hundred
> AVSubtitleArea items in extreme cases.
>
> The buffer and data fields are an array of fixed size (currently
> 8). Increasing to 16 or 32 would not help: there will still be
> cases where this does not suffice.
>
> 2. What exactly should be stored in frame->buf[]?
>
> Should we store the AVSubtitleArea structs as AVBuffer there?
>
> Or should we only store data in those buffers? If yes, which
> data? The subtitle bitmap probably. How about the subtitle
> text - maybe and area has even both. And should we also
> store the palette in some frame->buf[n]?
> If yes, how to keep track of which data is stored in which
> buffer?
> Further, there's a documented convention that requires that
> non-zero buffers are contiguous, i.e. there must not be
> an empty buffer pointer between two other a non-empty buffer
> pointers. This would require to re-arrange the buffers to
> close any gap when some subtitle area data is removed, zeroed
> out or has been converted to another format.
> Closing such gap also require to update any mapping information
> ("which buffer is related to which area?")
>
> That's a lot of tedious work and every API user (or codec/filter
> developer) would need to do it in the right way.
>
>
>
> 3. AVFrame Code Logic Mistmatch
>
> A subtitle frame can have 0 to N areas, which means that a
> frame without any area is still valid. Opposed to that, existing
> (code all over the place in ffmpeg) is treating an AVFrame
> as invalid when the first data buffer is empty,
> i.e. frame->buf[0] != NULL
>
&g
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Anton Khirnov
> Sent: Monday, November 14, 2022 10:53 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend
> the argument for submitting patches
>
> Quoting Soft Works (2022-11-14 10:35:40)
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Anton Khirnov
> > > Sent: Monday, November 14, 2022 10:13 AM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend
> the
> > > argument for submitting patches
> > >
> > > Stop talking about commercial programs, since this applies to any
> > > downstream user.
> > > ---
> > > doc/developer.texi | 20 +++-
> > > 1 file changed, 15 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/doc/developer.texi b/doc/developer.texi
> > > index 5cf3b19ee0..2f0d2b7daa 100644
> > > --- a/doc/developer.texi
> > > +++ b/doc/developer.texi
> > > @@ -24,11 +24,21 @@ generated from the headers
> > > the examples under @file{doc/examples}
> > > @end itemize
> > >
> > > -You can use the FFmpeg libraries in your commercial program, but
> you
> > > -are encouraged to @emph{publish any patch you make}. In this
> case
> > > the
> > > -best way to proceed is to send your patches to the ffmpeg-devel
> > > -mailing list following the guidelines illustrated in the
> remainder
> > > of
> > > -this document.
> > > +If you modify FFmpeg code for your own use case, you are highly
> > > encouraged to
> > > +@emph{submit your changes back to us}, using this document as a
> > > guide. There are
> > > +both pragmatic and ideological reasons to do so:
> > > +@itemize @bullet
> > > +@item
> > > +Maintaining external changes to keep up with upstream
> development is
> > > +time-consuming and error-prone. With your code in the main tree,
> it
> > > will be
> > > +maintained by FFmpeg developers.
> >
> > You should mention that sometimes it's not really worth to take the
> effort,
> > because waiting for reviews and permanent rebasing and re-
> submitting,
> > explaining, defending, getting insulted or ignored and whatsoever,
> > might end up taking much more time than just to keep and maintain
> your
> > changes privately. Eventually you might regret that you have even
> > started going that way.
>
> Sorry, but you problems are entirely self-inflicted. You have been
> told
> what changes need to happen right from the beginning, repeatedly, and
> by
> several developers independently.
And those are completed and settled, like I had state multiple times.
It's ready for review for months already.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the argument for submitting patches
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Anton Khirnov
> Sent: Monday, November 14, 2022 10:13 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH 2/4] doc/developer.texi: extend the
> argument for submitting patches
>
> Stop talking about commercial programs, since this applies to any
> downstream user.
> ---
> doc/developer.texi | 20 +++-
> 1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/doc/developer.texi b/doc/developer.texi
> index 5cf3b19ee0..2f0d2b7daa 100644
> --- a/doc/developer.texi
> +++ b/doc/developer.texi
> @@ -24,11 +24,21 @@ generated from the headers
> the examples under @file{doc/examples}
> @end itemize
>
> -You can use the FFmpeg libraries in your commercial program, but you
> -are encouraged to @emph{publish any patch you make}. In this case
> the
> -best way to proceed is to send your patches to the ffmpeg-devel
> -mailing list following the guidelines illustrated in the remainder
> of
> -this document.
> +If you modify FFmpeg code for your own use case, you are highly
> encouraged to
> +@emph{submit your changes back to us}, using this document as a
> guide. There are
> +both pragmatic and ideological reasons to do so:
> +@itemize @bullet
> +@item
> +Maintaining external changes to keep up with upstream development is
> +time-consuming and error-prone. With your code in the main tree, it
> will be
> +maintained by FFmpeg developers.
You should mention that sometimes it's not really worth to take the effort,
because waiting for reviews and permanent rebasing and re-submitting,
explaining, defending, getting insulted or ignored and whatsoever,
might end up taking much more time than just to keep and maintain your
changes privately. Eventually you might regret that you have even
started going that way.
Regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion fixes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Oneric > Sent: Sunday, November 13, 2022 10:00 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion > fixes > > On Sun, Nov 13, 2022 at 21:27:11 +0100, Oneric wrote: > > [...] You can easily verify emails are capable of it by > > applying patch 2 yourself or using patchwork’s mbox file. > > A correction: it seems since last time we discussed this, some > patchwork > update has expanded the line-mangling to also affect "mbox" and > "series" > downloads. (Or I’m misremembering and no patchwork download ever > worked) > So to apply v2 patch 2/3, using the received mail itself is the only > option. (Applying v1 patch 2/3 also does the job) Did you notice that FATE has failed, even with the v1 patches? softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion fixes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Oneric > Sent: Sunday, November 13, 2022 9:27 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion > fixes > > On Sun, Nov 13, 2022 at 20:15:23 +, Soft Works wrote: > > > > [...] it's not a Patchwork bug and it cannot be "fixed" > > at all in any other way than using binary diffs. The problem lies > in > > the fact that it's being sent by e-mail where the different line > > endings get lost (as the spec mandates CRLF and doesn't allow LF > only). > > As last time: No, emails are perfectly capable of preserving CRLF > line > endings via e.g. "quoted-printable" or "base64" transfer-encoding and > that’s what git send-email automatically uses. > > As a case in point of it not being "just" patchwork, there’s the > troubles with patch 3 of this series already as sent out by the list. > This is most likely to blame on the list itself and its message- > editing to > add a footer though, and not on git send-email or email in general. > > > Caveats when testing: Your local generated .patch files still work > > and depending on your e-mail client, even the patch in your sent > items > > folder might not work. But as soon as it's been sent via SMTP the > > different line ending get lost. > > And again like last, time I do ofc check the sent emails and > not > local patch files. You can easily verify emails are capable of it by > applying patch 2 yourself or using patchwork’s mbox file. > > Please actually focus on the patch this time. I have no need for > repeating > the same long tangent about email and transfer encodings as last > time, so > I’ll ignore all further messages from you not about the code changes. I am unable to apply any of those patches. Neither v1 nor v2 neither when downloading the series mbox or [2/3] from patchwork nor when saving [2/3] from the e-mail. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion fixes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Oneric > Sent: Sunday, November 13, 2022 8:57 PM > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH v2 0/3] Some small ASS conversion > fixes > > This fixes colours and font selection for files converted to ASS > > v2 > == > > On Nicolas George’s request resent with plain text diff for the > updated > refernce files. There is no other change and applying the diff of v1 > yields equal results to v2. > > Those files do intentionally contain CRLF line endings, > but apart from tests/ref/fate/sub-scc they are not marked as > "diff eol=crlf" in .gitattributes, so the diff (intentionally) > contains > some CRLF lines as well. Patchwork has or at least used to have bugs > dealing with CRLF diffs. In particular it may wrongly fail to apply > the > patch (I believe this is fixed by now though) and the buttons > for downloading the diff or pasting it into the clipboard corrupt > line > endings making the diff useless. > Instead directly apply the email in your inbox or use patchwork’s > mbox > download. If your git is configured strictly, you may also need to > use > git am --keep-cr ... . (Or just apply v1) Multiple issues were seen in this area. For the one part that is about mixed line endings, it's not a Patchwork bug and it cannot be "fixed" at all in any other way than using binary diffs. The problem lies in the fact that it's being sent by e-mail where the different line endings get lost (as the spec mandates CRLF and doesn't allow LF only). Caveats when testing: Your local generated .patch files still work and depending on your e-mail client, even the patch in your sent items folder might not work. But as soon as it's been sent via SMTP the different line ending get lost. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 2/3] avcodec/ass: accurately preserve colours
> -Original Message- > From: ffmpeg-devel On Behalf Of > Oneric > Sent: Sunday, November 13, 2022 8:22 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH 2/3] avcodec/ass: accurately > preserve colours > > Thanks for taking a look! > > On Sun, Nov 13, 2022 at 20:12:46 +0100, Nicolas George wrote: > > Oneric (12022-11-13): > > > [...] > > > tests/ref/fate/sub-webvtt2 | Bin 1648 -> 1668 bytes > > > > These are text files. Please fix this so that we can review the > patch. > > As explained in the cover letter, those files are intentionally using > CRLF > line endings and patchwork bugs (or at least used to bug) out if it > receives patches with CRLF line endings. The last few times this > always > caused some debate, so I used binary diffs to workaround it this > time. > It just adds the "YCbCr Matrix: None" line in every file and changes > the Encoding field to "1" instead of "0". > > If you prefer and believe the patchwork bugs won’t cause confusion, > I can of course immediately send a v2 with plain text diffs. IMO, this is not necessary, because those might be better visible but nobody would be able to properly apply them. I would even go one step further and specify those files which are causing trouble in a .gitattributes file, because this will allow this (creating patches for those files as binary) to be handled automatically by git. The .gitattributes solution worked successfully as tested here: https://patchwork.ffmpeg.org/project/ffmpeg/patch/bn0p223mb0358900908a4b2a6a37c3296ba...@bn0p223mb0358.namp223.prod.outlook.com/ PS: Thanks for your patch, will look at it later softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Rework color quantization in palette{gen,use}
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Clément Bœsch
> Sent: Tuesday, November 8, 2022 10:08 PM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Rework color quantization in
> palette{gen,use}
>
> On Sun, Nov 06, 2022 at 07:46:38PM +, Soft Works wrote:
> >
> >
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Clément Bœsch
> > > Sent: Saturday, November 5, 2022 4:26 PM
> > > To: ffmpeg-devel@ffmpeg.org
> > > Subject: [FFmpeg-devel] Rework color quantization in
> palette{gen,use}
> > >
> > > Hi,
> > >
> > > This patchset essentially fixes a few core problems in these
> filters
> > > and
> > > switches to a perceptual model.
> > >
> > > I've generated a report for each key commit on this (temporary)
> page:
> > > http://big.pkh.me/pal/ (warning: heavy page, ~500M; I did try to
> add
> > > some lazy
> > > loading of the images but I'm not sure it's actually working as
> > > expected).
> >
> > Comparing the results for the known and simple "rainbow O" example
> reveals
> > that the proposed implementation seems to be even inferior to the
> current
> > code and even farther away from what is possible to achieve:
> >
> > https://gist.github.com/softworkz/e310e3c84a338f98977d70b09e3e3f4f
>
> The pngquant file on this page has 373 unique colors, and the
> transparency
> is fake (the checkerboard is opaque white & grey). I think there is a
> mistake here.
Hi Clement,
I'm sorry about the confusion. The files in both Gists were created
in the same way: Opened the result image in PhotoShop, set the view
size to 400% and then created a screenshot and pasted into the Gist.
The reason I did it that way was that GitHub seemed to do its own
image "optimization" and I wanted to rule out any such effects and
just let others see what I see.
I couldn't find the original result from pngquant, but I have attached
the result from the elbg filter which is almost of the same quality.
For completeness, I'm also including the recent comparison, but it
seems you're already on track in this regard.
> WRT the regression after the patch, I confirm that there is a problem
> related to the dithering. If you try with dither=none or even
> dither=bayer, you'll observe that the colors are much better. I will
> update the results page at some point to include that file.
That would be great. Maybe you could also find another "simple" example
like with large-scale gradients rather than being so strongly colored
like the others?
Then I'd have a question about your file07 example. Is this the
original file or did I mix something up?
http://big.pkh.me/pal/output/0-current/file07/cfg00/0-ref.png
I'm wondering because the image is full or weird artifacts at the
edges of the green (and other) leafes.
> Now indeed the sierra dithering (and probably the other of the same
> type)
> are somehow spreading way too strongly, it's unclear to me yet but
> that
> might be a bug I introduced. I'll investigate, thanks.
Yup, okay, thanks.
PS: I'd be curious what you think about the elbg image...
Thanks,
softworkz
<>
<>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v9 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpegagent > Sent: Tuesday, October 25, 2022 11:13 AM > To: ffmpeg-devel@ffmpeg.org > Cc: softworkz > Subject: [PATCH v9 00/25] Subtitle Filtering 2022 > > > Subtitle Filtering 2022 > === > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as > possible on > one side and to provide more clarity and control over the way things > are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides > different > modes for arbitrating subtitle frames in a filter graph. Finally, > each > subtitle filter's documentation has been amended by a section > describing the > filter's timeline behavior (in v3 update). Another PING. I'm not aware of any unanswered questions or pending objections. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Rework color quantization in palette{gen,use}
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Clément Bœsch
> Sent: Saturday, November 5, 2022 4:26 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] Rework color quantization in palette{gen,use}
>
> Hi,
>
> This patchset essentially fixes a few core problems in these filters
> and
> switches to a perceptual model.
>
> I've generated a report for each key commit on this (temporary) page:
> http://big.pkh.me/pal/ (warning: heavy page, ~500M; I did try to add
> some lazy
> loading of the images but I'm not sure it's actually working as
> expected).
Comparing the results for the known and simple "rainbow O" example reveals
that the proposed implementation seems to be even inferior to the current
code and even farther away from what is possible to achieve:
https://gist.github.com/softworkz/e310e3c84a338f98977d70b09e3e3f4f
Regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Rework color quantization in palette{gen,use}
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Clément Bœsch
> Sent: Saturday, November 5, 2022 4:26 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] Rework color quantization in palette{gen,use}
>
> Hi,
>
> This patchset essentially fixes a few core problems in these filters
> and
> switches to a perceptual model.
>
> I've generated a report for each key commit on this (temporary) page:
> http://big.pkh.me/pal/ (warning: heavy page, ~500M; I did try to add
> some lazy
> loading of the images but I'm not sure it's actually working as
> expected).
>
> It is easy for me to add specific samples and re-run the whole thing,
> so feel
> free to suggest one.
The "rainbow Q" image would be nice to see in comparison.
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH, v3] MAINTAINERS: add myself as amfenc* maintainer
> -Original Message- > From: ffmpeg-devel On Behalf Of > OvchinnikovDmitrii > Sent: Friday, November 4, 2022 7:58 PM > To: ffmpeg-devel@ffmpeg.org > Cc: OvchinnikovDmitrii > Subject: [FFmpeg-devel] [PATCH, v3] MAINTAINERS: add myself as > amfenc* maintainer > > Due to the lack of an active AMF maintainer at the moment, as well > as plans to add the av1 encoder and other improvements of AMF, > I added myself to the maintainers. Timely review and merging > patches targeting AMF integration should improve support > of AMD GPUs and APUs in FFmpeg. > For the last couple of years I have been working on AMF related > patches to ffmpeg and other open source projects. Considering the lack of ambition in the past, I think there's no point in waiting for AMD to come around, and having a maintainer would surely be beneficial, hence LGTM. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] avfilter/avfilter: add avfilter_print_config_formats()
> -Original Message-
> From: Paul B Mahol
> Sent: Thursday, November 3, 2022 10:58 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Cc: softworkz
> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avfilter/avfilter: add
> avfilter_print_config_formats()
>
> On 10/11/22, softworkz wrote:
> > From: softworkz
> >
> > Prints the following to AVBPrint:
> >
> > For pass-through filter links:
> >
> > "All (passthrough)"
> >
> > For filters using query_formats:
> >
> > "Dynamic"
> >
> > For filters using query_formats where a call to query_formats
> > succeeds (example):
> >
> > "Dynamic, Defaults: [yuv420p, yuvj420p, yuva420p, nv12, nv21]"
> >
> > For all other filters (example):
> >
> > "[s16p, s32p, fltp, dblp]"
> >
> > Except in case when the number of formats equals the number of
> > available formats:
> >
> > "All"
> >
> > Signed-off-by: softworkz
> > ---
> > doc/APIchanges | 3 ++
> > libavfilter/avfilter.c | 102
> +++-
> > libavfilter/avfilter.h | 12 +
> > libavfilter/avfiltergraph.c | 14 +++--
> > libavfilter/internal.h | 9
> > libavfilter/version.h | 4 +-
> > 6 files changed, 136 insertions(+), 8 deletions(-)
> >
> > diff --git a/doc/APIchanges b/doc/APIchanges
> > index cbb579612e..6e2a528b04 100644
> > --- a/doc/APIchanges
> > +++ b/doc/APIchanges
> > @@ -14,6 +14,9 @@ libavutil: 2021-04-27
> >
> > API changes, most recent first:
> >
> > +2022-10-11 - xx - lavf 59.50.100 - avfilter.h
> > + Add add avfilter_print_config_formats().
> > +
> > 2022-10-05 - 37d5ddc317 - lavu 57.39.100 - cpu.h
> >Add AV_CPU_FLAG_RVB_BASIC.
> >
> > diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
> > index cc5505e65b..8cc665e19c 100644
> > --- a/libavfilter/avfilter.c
> > +++ b/libavfilter/avfilter.c
> > @@ -196,6 +196,104 @@ void avfilter_link_free(AVFilterLink **link)
> > av_freep(link);
> > }
> >
> > +static unsigned get_nb_pix_fmts()
> > +{
> > +unsigned i = 0;
> > +while (av_pix_fmt_desc_get(i++)) {}
> > +return i - 1;
> > +}
> > +
> > +static unsigned get_nb_sample_fmts()
> > +{
> > +unsigned i = 0;
> > +while (av_get_sample_fmt_name(i++)) {}
> > +return i - 1;
> > +}
> > +
> > +int avfilter_print_config_formats(AVBPrint *bp, const struct
> AVFilter
> > *filter, int for_output, unsigned pad_index)
> > +{
> > +AVFilterGraph *graph;
> > +AVFilterContext *filter_context;
> > +AVFilterFormatsConfig *config;
> > +enum AVMediaType media_type;
> > +int ret = 0;
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_PASSTHROUGH) {
> > +av_bprintf(bp, "All (passthrough)");
> > +return 0;
> > +}
> > +
> > +graph = avfilter_graph_alloc();
> > +if (!graph) {
> > +av_log(NULL, AV_LOG_ERROR, "Failed to create
> filtergraph\n");
> > +ret = AVERROR(ENOMEM);
> > +goto cleanup;
> > +}
> > +
> > +filter_context = avfilter_graph_alloc_filter(graph, filter,
> "filter");
> > +if (!filter_context) {
> > +av_log(NULL, AV_LOG_ERROR, "Failed to create filter\n");
> > +ret = AVERROR(ENOMEM);
> > +goto cleanup;
> > +}
> > +
> > +avfilter_init_str(filter_context, NULL);
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC)
> > +av_bprintf(bp, "Dynamic");
> > +
> > +if (!for_output && pad_index >= filter_context->nb_inputs
> > +|| for_output && pad_index >= filter_context->nb_outputs)
> > +goto cleanup;
> > +
> > +avfilter_graph_config(graph, graph);
> > +
> > +for (unsigned i = 0; i < filter_context->nb_inputs; i++)
> > +filter_context->inputs[i] = (AVFilterLink
> > *)av_mallocz(sizeof(AVFilterLink));
> > +
> > +for (unsigned i = 0; i < filter_context->nb_outputs; i++)
> > +filter_context->outputs[i] = (AVFilterLink
> > *)av_mallocz(sizeof(AVFilterLink));
> > +
> > +ff_filter_query_formats(filter_context);
> > +
> > +config = for_output ? _context->outputs[pad_index]-
> >incfg :
> > _context->inputs[pad_index]->outcfg;
> > +
> > +if (!config || !config->formats)
> > +goto cleanup;
> > +
> > +media_type= for_output ? filter->outputs[pad_index].type :
> > filter->inputs[pad_index].type;
> > +
> > +if (filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC) {
> > +if (config->formats && config->formats->nb_formats)
> > +av_bprintf(bp, ", Default: ");
> > +}
> > +
> > +if (config->formats == NULL)
> > +av_bprintf(bp, "unknown");
> > +else if (media_type == AVMEDIA_TYPE_VIDEO &&
> > config->formats->nb_formats == get_nb_pix_fmts() ||
> > + media_type == AVMEDIA_TYPE_AUDIO &&
> > config->formats->nb_formats == get_nb_sample_fmts())
> > +av_bprintf(bp, "All");
> > +else {
> > +for (unsigned i = 0; i < config->formats->nb_formats; i++)
> {
> > +if (i ==
Re: [FFmpeg-devel] [PATCH 2/2] avcodec/vpp_qsv: Copy side data from input to output frame
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> "zhilizhao(???)"
> Sent: Tuesday, November 1, 2022 10:18 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH 2/2] avcodec/vpp_qsv: Copy side
> data from input to output frame
>
>
>
> > On Nov 1, 2022, at 12:58, Xiang, Haihao intel@ffmpeg.org> wrote:
> >
> > On Mon, 2022-10-24 at 23:04 +, softworkz wrote:
> >> From: softworkz
> >>
> >> Signed-off-by: softworkz
> >> ---
> >> libavfilter/qsvvpp.c | 6 ++
> >> libavfilter/vf_overlay_qsv.c | 19 +++
> >> 2 files changed, 21 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c
> >> index 8428ee89ab..ae9766d12f 100644
> >> --- a/libavfilter/qsvvpp.c
> >> +++ b/libavfilter/qsvvpp.c
> >> @@ -880,6 +880,12 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s,
> AVFilterLink
> >> *inlink, AVFrame *picr
> >> return AVERROR(EAGAIN);
> >> break;
> >> }
> >> +
> >> +av_frame_remove_all_side_data(out_frame->frame);
> >> +ret = av_frame_copy_side_data(out_frame->frame, in_frame-
> >frame, 0);
> >> +if (ret < 0)
> >> +return ret;
> >> +
> >> out_frame->frame->pts = av_rescale_q(out_frame-
> >>> surface.Data.TimeStamp,
> >> default_tb, outlink-
> >time_base);
> >>
> >> diff --git a/libavfilter/vf_overlay_qsv.c
> b/libavfilter/vf_overlay_qsv.c
> >> index d947a1faa1..04fd284b92 100644
> >> --- a/libavfilter/vf_overlay_qsv.c
> >> +++ b/libavfilter/vf_overlay_qsv.c
> >> @@ -231,13 +231,24 @@ static int process_frame(FFFrameSync *fs)
> >> {
> >> AVFilterContext *ctx = fs->parent;
> >> QSVOverlayContext *s = fs->opaque;
> >> +AVFrame *frame0 = NULL;
> >> AVFrame*frame = NULL;
> >> -int ret = 0, i;
> >> +int ret = 0;
> >>
> >> -for (i = 0; i < ctx->nb_inputs; i++) {
> >> +for (unsigned i = 0; i < ctx->nb_inputs; i++) {
> >> ret = ff_framesync_get_frame(fs, i, , 0);
> >> -if (ret == 0)
> >> -ret = ff_qsvvpp_filter_frame(s->qsv, ctx->inputs[i],
> frame);
> >> +
> >> +if (ret == 0) {
> >> +if (i == 0)
> >> +frame0 = frame;
> >> +else {
> >> +av_frame_remove_all_side_data(frame);
> >> +ret = av_frame_copy_side_data(frame, frame0, 0);
> >> +}
> >> +
> >> +ret = ret < 0 ? ret : ff_qsvvpp_filter_frame(s->qsv,
> ctx-
> >>> inputs[i], frame);
> >> +}
> >> +
> >> if (ret < 0 && ret != AVERROR(EAGAIN))
> >> break;
> >> }
> >
> > Patchset LGTM, I'll push this patchset if no more comment or
> objection.
>
> avcodec/vpp_qsv: Copy side data from input to output frame
> ^^^
>
> avcodec -> avfilter
>
Good catch!
Thank you,
sw
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Monday, October 31, 2022 10:59 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Mon, Oct 31, 2022 at 11:57:16AM +0100, Clément Bœsch wrote: > > On Mon, Oct 31, 2022 at 01:43:11AM +, Soft Works wrote: > > [...] > > > > > > The patch I had submitted doesn't change the previous > behavior > > > > > > without the use_alpha parameter. > > > > > > > > Yes I noticed, but unfortunately I'm reworking the color > distance to > > > > work > > > > in perceptual color space, and the way that alpha is mixed up > in the > > > > equation just doesn't make any sense at all and prevents me > from > > > > doing > > > > these changes. > > > > > > If you want to implement a new color distance algorithm, it > should > > > be either a new filter or a new (switchable) mode for the > existing > > > filter. > > > > Why? > > > > > Photoshop has these different modes as well and it would > > > surely be useful, but I don't think it should be replacing the > > > existing behavior. > > > > > > > There is no point in keeping a ton of complexity exposed as user > options > > for something implementation specific. We offer no guarantee over > how the > > quantization is expected to run. > > > > > When it turns out that the use_alpha implementation doesn't fit > > > with your new color distance calculation and you add it as > > > an additional mode, then it would be fine IMO when the filter > > > errors in case it would be attempted to use that mode in > > > combination with use_alpha. > > > > IMO The use_alpha option shouldn't exist in the first place, it > should be > > the default behaviour because honoring the alpha is the correct > thing to > > do. That's not what the option is currently doing though. > > > > > > > Do you think it might make sense to put more weight on the > > > > > alpha value by tripling it? So it would be weighted equally > to the > > > > > RGB value? > > > > > > > > You cannot mix alpha with colors at all, they are separate > domains > > > > and you > > > > need to treat them as such. > > > > > > What's interesting is that I've followed the same (simplified) > > > way for adding a use_alpha option to vf_elbg and it provides > excellent > > > results without treating alpha separately. > > > > I don't know how the filter works and what it's supposed to do, but > if > > it's indeed using the same approach as the palette ones, it cannot > work. > > > > > > From paletteuse perspective what you need to do is first choose > the > > > > colors > > > > in the palette that match exactly the alpha (or at least the > closest > > > > if > > > > and only there is no exact match). Then within that set, and > only > > > > within > > > > that one, you'd pick the closest color. > > > > > > > > From palettegen perspective, you need to split the colors in > > > > different > > > > transparency domain (a first dimensional quantization), then > quantize > > > > the > > > > colors in each quantized alpha dimension. And when you have all > your > > > > quantized palettes for each level of alpha, you find an > algorithm to > > > > reduce the number of transparency dimensions or the number of > colors > > > > per > > > > dimension to make it fit inside a single palette. But you can't > just > > > > do > > > > the alpha and the colors at the same time, it cannot work, > whatever > > > > weights you choose. > > > > > > I would be curious to see how well that would work, especially > > > in cases when the target palettes have just a few number of > colors. > > > > > > > You have to make a call between whether you want to preserve the > > transparency or the color while constructing the palette, but when > > choosing a color you must absolutely not choose a color with a > different > > transparency, you must pick amongst the closest alpha, with a > particular > > attention to extreme alphas: an opaque colors m
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Monday, October 31, 2022 7:51 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Mon, Oct 31, 2022 at 03:11:13PM +, Soft Works wrote: > [...] > > And pngquant doing the impossible as well: > > > > > Interestingly, pngquant which is supposed to have the best open > source > > > quantization algorithms > > Says who? That's the result of my research I did at that time. Feel free to do your own research. > https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da > 02d86925cc0167831205/pam.h#L163-L182 > > > https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da > 02d86925cc0167831205/mediancut.c#L29-L49 > > > https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da > 02d86925cc0167831205/mediancut.c#L449-L476 > > > > I'd rather not look too much into that code. Do they mess up the > alpha > channel as well? When working on this, you should at least be familiar with the quality of results it provides, even when you don't want to look at the code. > > If you want to improve the way it works that's another story. > > > > But at this point, we're talking about removal. And I disagree to > that. > > You may disagree but: > - the option causes many transparency artifacts > - the option is fundamentally flawed and need to be rewritten > differently > - handling the alpha should be by default if such a feature was > existing > - the option is preventing improvements to the code > > I will send a patchset in the coming days, which depends on its > removal. > You'll be free to propose again a patch to support alpha quantization I'll take that as a joke.. > properly, but I'll ask for it to be reliable enough so that it's > enabled > by default. That's not acceptable either, because the filter may also be used to create animated GIFs where only a single palette entry may exist which indicates full transparency and all others are fully opaque. I don't want to be just negative and rejective and I'm really welcoming patches that provide improvement. But I think that a prerequisite for working on this subject is to be familiar with state-of-the art implementations of image color quantization. The best implementations I'm aware of can be found in PhotoShop (closed source) and pngquant (open source). The quantization that vf_elbg (with use_alpha) does get very close to the aforementioned, but it comes at the price of relatively high computational cost. If you would come up with an implementation for palettegen/-use that provides comparable results in an efficient way like pngquant, that would be really awesome! > but I'll ask for it to be reliable enough so that it's I'll ask for your patch to support alpha in the same optional way like it is supported right now. (actually something that should go without saying) Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Monday, October 31, 2022 11:57 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Mon, Oct 31, 2022 at 01:43:11AM +, Soft Works wrote: > [...] > > > > > The patch I had submitted doesn't change the previous > behavior > > > > > without the use_alpha parameter. > > > > > > Yes I noticed, but unfortunately I'm reworking the color distance > to > > > work > > > in perceptual color space, and the way that alpha is mixed up in > the > > > equation just doesn't make any sense at all and prevents me from > > > doing > > > these changes. > > > > If you want to implement a new color distance algorithm, it should > > be either a new filter or a new (switchable) mode for the existing > > filter. > > Why? > > > Photoshop has these different modes as well and it would > > surely be useful, but I don't think it should be replacing the > > existing behavior. > > > > There is no point in keeping a ton of complexity exposed as user > options > for something implementation specific. We offer no guarantee over how > the > quantization is expected to run. Says who? > > When it turns out that the use_alpha implementation doesn't fit > > with your new color distance calculation and you add it as > > an additional mode, then it would be fine IMO when the filter > > errors in case it would be attempted to use that mode in > > combination with use_alpha. > > IMO The use_alpha option shouldn't exist in the first place, it > should be > the default behaviour because honoring the alpha is the correct thing > to > do. That's not what the option is currently doing though. > > > > > Do you think it might make sense to put more weight on the > > > > alpha value by tripling it? So it would be weighted equally to > the > > > > RGB value? > > > > > > You cannot mix alpha with colors at all, they are separate > domains > > > and you > > > need to treat them as such. > > > > What's interesting is that I've followed the same (simplified) > > way for adding a use_alpha option to vf_elbg and it provides > excellent > > results without treating alpha separately. > > I don't know how the filter works and what it's supposed to do, but > if > it's indeed using the same approach as the palette ones, it cannot > work. Then it's magic, I guess. The commands and results are on the same page I have posted. And pngquant doing the impossible as well: > Interestingly, pngquant which is supposed to have the best open source > quantization algorithms seems to be using weights (albeit in a more > sophisticated way) and does not handle alpha separately for calculating > color distance, variance and averaging: https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/pam.h#L163-L182 https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/mediancut.c#L29-L49 https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/mediancut.c#L449-L476 > That's not how it's going to work, sorry; I'm not going to increase > complexity and maintenance effort for no gain. Implementing a correct > support for the alpha will likely involve a revert of that commit > anyway. If you want to improve the way it works that's another story. But at this point, we're talking about removal. And I disagree to that. Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 09/11] avfilter/overlay_vaapi: enable expressions for overlay parameters
> -Original Message-
> From: Xiang, Haihao
> Sent: Monday, October 31, 2022 6:44 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: softwo...@hotmail.com
> Subject: Re: [FFmpeg-devel] [PATCH 09/11] avfilter/overlay_vaapi:
> enable expressions for overlay parameters
>
> On Mon, 2022-10-10 at 10:54 +, softworkz wrote:
> > From: softworkz
> >
> > Signed-off-by: softworkz
> > ---
> > libavfilter/vf_overlay_vaapi.c | 141
> +
> > 1 file changed, 127 insertions(+), 14 deletions(-)
> >
> > diff --git a/libavfilter/vf_overlay_vaapi.c
> b/libavfilter/vf_overlay_vaapi.c
> > index b2c254d9dd..7be7d52589 100644
> > --- a/libavfilter/vf_overlay_vaapi.c
> > +++ b/libavfilter/vf_overlay_vaapi.c
> > @@ -27,19 +27,106 @@
> > #include "formats.h"
> > #include "internal.h"
> > #include "vaapi_vpp.h"
> > +#include "libavutil/eval.h"
> > +
> > +enum var_name {
> > +VAR_MAIN_iW, VAR_MW,
> > +VAR_MAIN_iH, VAR_MH,
> > +VAR_OVERLAY_iW,
> > +VAR_OVERLAY_iH,
>
> It is better not to mix capital and lower-case letters here, (I have
> a patch to
> change the var_name in qsv)
Yea - I had done it equal to overlay_qsv.
I'll change it, no problem.
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Monday, October 31, 2022 1:30 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Sun, Oct 30, 2022 at 10:55:31PM +, Soft Works wrote: > [...] > > > Do you think it might make sense to put more weight on the > > alpha value by tripling it? So it would be weighted equally to the > > RGB value? > > You cannot mix alpha with colors at all, they are separate domains > and you > need to treat them as such. > > From paletteuse perspective what you need to do is first choose the > colors > in the palette that match exactly the alpha (or at least the closest > if > and only there is no exact match). Then within that set, and only > within > that one, you'd pick the closest color. > > From palettegen perspective, you need to split the colors in > different > transparency domain (a first dimensional quantization), then quantize > the > colors in each quantized alpha dimension. And when you have all your > quantized palettes for each level of alpha, you find an algorithm to > reduce the number of transparency dimensions or the number of colors > per > dimension to make it fit inside a single palette. But you can't just > do > the alpha and the colors at the same time, it cannot work, whatever > weights you choose. Interestingly, pngquant which is supposed to have the best open source quantization algorithms seems to be using weights (albeit in a more sophisticated way) and does not handle alpha separately for calculating color distance, variance and averaging: https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/pam.h#L163-L182 https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/mediancut.c#L29-L49 https://github.com/ImageOptim/libimagequant/blob/a16c9ca66a24158496da02d86925cc0167831205/mediancut.c#L449-L476 softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Monday, October 31, 2022 1:30 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Sun, Oct 30, 2022 at 10:55:31PM +, Soft Works wrote: > [...] > > > I understand why. I know that it's not perfect. But it's the best > > > what's achievable within the way the filter is working. > > > > > > But I wouldn't go that far as saying it would be "broken". I > think > > > the result is quite acceptable with regards to the method being > > > using. > > It's broken because the alpha channel in the output is really > completely > random. If we blend the output of that "O" PNG somewhere, it's going > to be > a real mess. > > Here is a more concrete example: if your input has some red fully > transparent (00ff), and some red fully opaque () which > end up > in the same box, they will be averaged to a red with an alpha of > 0x80, and > I'm not even accounting for the weight and other colors with > different > transparency. That non-opaque average alpha might end up being used > in > area that are expected to be opaque, and in some area where it's > supposed > to be transparent. That's exactly what I showed with the "O" png. > > In addition to that problem, since you're also accounting the alpha > as a > weight to determine the proximity of 2 colors, you're going to select > the > wrong colors. For example if we want to find the closest color to an > opaque green (ff00ff00), and the palette has a slightly transparent > green > (fa00ff00) and an opaque blue (ffff), then now the algorithm will > prefer the blue over the green. > > That explains why in addition to the alpha being random in the "O" > png, > the colors are also messed up. > > > > The patch I had submitted doesn't change the previous behavior > > > without the use_alpha parameter. > > Yes I noticed, but unfortunately I'm reworking the color distance to > work > in perceptual color space, and the way that alpha is mixed up in the > equation just doesn't make any sense at all and prevents me from > doing > these changes. If you want to implement a new color distance algorithm, it should be either a new filter or a new (switchable) mode for the existing filter. Photoshop has these different modes as well and it would surely be useful, but I don't think it should be replacing the existing behavior. When it turns out that the use_alpha implementation doesn't fit with your new color distance calculation and you add it as an additional mode, then it would be fine IMO when the filter errors in case it would be attempted to use that mode in combination with use_alpha. > > Do you think it might make sense to put more weight on the > > alpha value by tripling it? So it would be weighted equally to the > > RGB value? > > You cannot mix alpha with colors at all, they are separate domains > and you > need to treat them as such. What's interesting is that I've followed the same (simplified) way for adding a use_alpha option to vf_elbg and it provides excellent results without treating alpha separately. > From paletteuse perspective what you need to do is first choose the > colors > in the palette that match exactly the alpha (or at least the closest > if > and only there is no exact match). Then within that set, and only > within > that one, you'd pick the closest color. > > From palettegen perspective, you need to split the colors in > different > transparency domain (a first dimensional quantization), then quantize > the > colors in each quantized alpha dimension. And when you have all your > quantized palettes for each level of alpha, you find an algorithm to > reduce the number of transparency dimensions or the number of colors > per > dimension to make it fit inside a single palette. But you can't just > do > the alpha and the colors at the same time, it cannot work, whatever > weights you choose. I would be curious to see how well that would work, especially in cases when the target palettes have just a few number of colors. But to return to the proposal of removal: If everything from ffmpeg would be removed which is not perfect, then it would be lacking quite a number of features I suppose :-) In the same way, one could say that palettegen/-use should be removed because its results are wrong and colors are randomly mixed and misplaced while the vf_elbg filter does it right. When you look at the result under the heading "Paletteuse/
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Sunday, October 30, 2022 10:41 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Clément Bœsch > > Sent: Sunday, October 30, 2022 10:30 PM > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Subject: Re: [FFmpeg-devel] [PATCH] Revert > > "avfilter/vf_palette(gen|use): support palettes with alpha" > > > > On Sun, Oct 30, 2022 at 09:19:05PM +, Soft Works wrote: > > [...] > > > At the time of submission I did a lot of experiments and the > > results > > > seemed to be very useful: > > > > > > > https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f > > > > > > > On that page, the result of the command using palette* filters for > > the > > colored "O" gives a broken output, and it's not a simple bug, it's > > more > > fundamental. You cannot just just average the colors with different > > levels > > of alpha, it makes no sense. > > > > Do you understand why or do you need me to elaborate? > > I understand why. I know that it's not perfect. But it's the best > what's achievable within the way the filter is working. > > But I wouldn't go that far as saying it would be "broken". I think > the result is quite acceptable with regards to the method being > using. > > The patch I had submitted doesn't change the previous behavior > without the use_alpha parameter. > And when using the use_alpha parameter, the results are still > useful in many cases - maybe not always. > > But it don't see any reason to remove this just because it's not > perfect. If you would have a patch to improve the results - that > would be a different story and of course very welcome. I'm really not an expert in image processing algorithms, and I haven't tried the following yet: Do you think it might make sense to put more weight on the alpha value by tripling it? So it would be weighted equally to the RGB value? softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Sunday, October 30, 2022 10:30 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] Revert > "avfilter/vf_palette(gen|use): support palettes with alpha" > > On Sun, Oct 30, 2022 at 09:19:05PM +, Soft Works wrote: > [...] > > At the time of submission I did a lot of experiments and the > results > > seemed to be very useful: > > > > https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f > > > > On that page, the result of the command using palette* filters for > the > colored "O" gives a broken output, and it's not a simple bug, it's > more > fundamental. You cannot just just average the colors with different > levels > of alpha, it makes no sense. > > Do you understand why or do you need me to elaborate? I understand why. I know that it's not perfect. But it's the best what's achievable within the way the filter is working. But I wouldn't go that far as saying it would be "broken". I think the result is quite acceptable with regards to the method being using. The patch I had submitted doesn't change the previous behavior without the use_alpha parameter. And when using the use_alpha parameter, the results are still useful in many cases - maybe not always. But it don't see any reason to remove this just because it's not perfect. If you would have a patch to improve the results - that would be a different story and of course very welcome. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"
> -Original Message- > From: ffmpeg-devel On Behalf Of > Clément Bœsch > Sent: Sunday, October 30, 2022 6:58 PM > To: ffmpeg-devel@ffmpeg.org > Cc: softwo...@hotmail.com; Clément Bœsch > Subject: [FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): > support palettes with alpha" > > This reverts commit dea673d0d548c864ec85f9260d8900d944ef7a2a. > > This change cannot work for several reasons, the most obvious ones > are: > > - the alpha is being part of the scoring of the color difference, > even > though we can not interpret the alpha as part of the perception of > the > color (we don't even know if it's premultiplied or postmultiplied) > - the colors are averaged with their alpha value which simply cannot > work > > The command proposed in the original thread of the patch actually > produces a completely broken file: > > ffmpeg -y -loglevel verbose -i fate-suite/apng/o_sample.png - > filter_complex > "split[split1][split2];[split1]palettegen=max_colors=254:use_alpha=1[ > pal1];[split2][pal1]paletteuse=use_alpha=1" -frames:v 1 out.png > > We can see that many color pixels are off, but more importantly some > colors have a random alpha value: https://imgur.com/eFQ2UK7 > > I don't see any easy fix for this unfortunately, the approach appears > to > be flawed by design. > --- At the time of submission I did a lot of experiments and the results seemed to be very useful: https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Tuesday, October 25, 2022 11:59 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare
> AVFrame for subtitle handling
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Hendrik Leppkes
> > Sent: Tuesday, October 25, 2022 11:38 AM
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare
> > AVFrame for subtitle handling
> >
> > On Tue, Oct 25, 2022 at 11:14 AM softworkz
> > wrote:
> > >
> > > @@ -712,6 +712,53 @@ typedef struct AVFrame {
> > > * Duration of the frame, in the same units as pts. 0 if
> > unknown.
> > > */
> > > int64_t duration;
> > > +
> > > +/**
> > > + * Media type of the frame (audio, video, subtitles..)
> > > + *
> > > + * See AVMEDIA_TYPE_xxx
> > > + */
> > > +enum AVMediaType type;
> > > +
> > > +/**
> > > + * Number of items in the @ref subtitle_areas array.
> > > + */
> > > +unsigned num_subtitle_areas;
> > > +
> > > +/**
> > > + * Array of subtitle areas, may be empty.
> > > + */
> > > +AVSubtitleArea **subtitle_areas;
> > > +
> > > +/**
> > > + * Header containing style information for text subtitles.
> > > + */
> > > +AVBufferRef *subtitle_header;
> > > +
> > > +/**
> > > + * Indicates that a subtitle frame is a repeated frame for
> > arbitrating flow
> > > + * in a filter graph.
> > > + * The field subtitle_timing.start_pts always indicates the
> > original presentation
> > > + * time, while the frame's pts field may be different.
> > > + */
> > > +int repeat_sub;
> > > +
> > > +struct SubtitleTiming
> > > +{
> > > +/**
> > > + * The display start time, in AV_TIME_BASE.
> > > + *
> > > + * For subtitle frames, AVFrame.pts is populated from
> the
> > packet pts value,
> > > + * which is not always the same as this value.
> > > + */
> > > +int64_t start_pts;
> >
> > There is still no explanation here why they are not the same, why
> > they
> > could not just be the same, and which field a user should look at.
> > The cover letter talks about clarity why this is needed is
> important,
> > but then provides none of that.
> >
> > "Its required" is not an argument. So please enlighten us once
> again
> > why we absolutely need two timestamps for subtitles, instead of
> just
> > one. As far as I can see, subtitle frames only have one relevant
> time
> > - when its supposed to be shown on the screen. What would the other
> > time ever be good for to a user?
> > Similarly for the duration, of course. I can even see the
> > AVFrame.duration field in this patch snippet just above the
> additions
> > that would seem to fully replace this one.
> >
> > - Hendrik
>
> Hi Hendrik,
>
> thanks a lot for your reply.
>
> Probably I should have better advertised the article I had written
> specifically to explain the background of this:
>
> https://github.com/softworkz/SubtitleFilteringDemos/issues/1
@Hendrik - did that answer your question?
You are also welcome to contact me directly for discussing details
or going through specific examples; or just to ping me for logging
on to IRC for chat.
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v9 23/25] avcodec/subtitles: Migrate subtitle encoders to frame-based API
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Niedermayer > Sent: Thursday, October 27, 2022 7:54 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v9 23/25] avcodec/subtitles: > Migrate subtitle encoders to frame-based API > > On Tue, Oct 25, 2022 at 09:13:44AM +, softworkz wrote: > > From: softworkz > > > > and provide a compatibility shim for the legacy api > > > > Signed-off-by: softworkz > > --- > > libavcodec/assenc.c | 189 ++-- > > > libavcodec/avcodec.h| 5 +- > > libavcodec/codec_internal.h | 12 --- > > libavcodec/dvbsubenc.c | 96 ++ > > libavcodec/dvdsubenc.c | 103 > > libavcodec/encode.c | 61 +++- > > libavcodec/movtextenc.c | 114 -- > > libavcodec/srtenc.c | 108 ++--- > > libavcodec/tests/avcodec.c | 5 +- > > libavcodec/ttmlenc.c| 101 ++- > > libavcodec/utils.c | 1 - > > libavcodec/webvttenc.c | 86 +++- > > libavcodec/xsubenc.c| 88 ++--- > > 13 files changed, 689 insertions(+), 280 deletions(-) > > Causes this testcase to fail: > ./ffmpeg -i 'bgc.sub.dub.ogm' -vframes 3 -bitexact -y nosubs.webm > > https://samples.ffmpeg.org/ogg/ > > I did not investgate why or if this a bug or expected. Just reporting > a difference > ive seen Hi Michael, thanks a lot for testing my patchset! I have run the command with and without my patchset and I can confirm that the output is different. Though, from analyzing the output files, it appears that the output with my patchset applied seems "more correct" than the output that ffmpeg is currently creating. The details of the investigation can be found here: https://github.com/softworkz/SubtitleFilteringDemos/issues/2 Thanks again, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare AVFrame for subtitle handling
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Hendrik Leppkes
> Sent: Tuesday, October 25, 2022 11:38 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v9 02/25] avutil/frame: Prepare
> AVFrame for subtitle handling
>
> On Tue, Oct 25, 2022 at 11:14 AM softworkz
> wrote:
> >
> > @@ -712,6 +712,53 @@ typedef struct AVFrame {
> > * Duration of the frame, in the same units as pts. 0 if
> unknown.
> > */
> > int64_t duration;
> > +
> > +/**
> > + * Media type of the frame (audio, video, subtitles..)
> > + *
> > + * See AVMEDIA_TYPE_xxx
> > + */
> > +enum AVMediaType type;
> > +
> > +/**
> > + * Number of items in the @ref subtitle_areas array.
> > + */
> > +unsigned num_subtitle_areas;
> > +
> > +/**
> > + * Array of subtitle areas, may be empty.
> > + */
> > +AVSubtitleArea **subtitle_areas;
> > +
> > +/**
> > + * Header containing style information for text subtitles.
> > + */
> > +AVBufferRef *subtitle_header;
> > +
> > +/**
> > + * Indicates that a subtitle frame is a repeated frame for
> arbitrating flow
> > + * in a filter graph.
> > + * The field subtitle_timing.start_pts always indicates the
> original presentation
> > + * time, while the frame's pts field may be different.
> > + */
> > +int repeat_sub;
> > +
> > +struct SubtitleTiming
> > +{
> > +/**
> > + * The display start time, in AV_TIME_BASE.
> > + *
> > + * For subtitle frames, AVFrame.pts is populated from the
> packet pts value,
> > + * which is not always the same as this value.
> > + */
> > +int64_t start_pts;
>
> There is still no explanation here why they are not the same, why
> they
> could not just be the same, and which field a user should look at.
> The cover letter talks about clarity why this is needed is important,
> but then provides none of that.
>
> "Its required" is not an argument. So please enlighten us once again
> why we absolutely need two timestamps for subtitles, instead of just
> one. As far as I can see, subtitle frames only have one relevant time
> - when its supposed to be shown on the screen. What would the other
> time ever be good for to a user?
> Similarly for the duration, of course. I can even see the
> AVFrame.duration field in this patch snippet just above the additions
> that would seem to fully replace this one.
>
> - Hendrik
Hi Hendrik,
thanks a lot for your reply.
Probably I should have better advertised the article I had written
specifically to explain the background of this:
https://github.com/softworkz/SubtitleFilteringDemos/issues/1
I hope it's understandable - please let me know when you have
questions.
Thanks again,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for QSV decoders
> -Original Message- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Thursday, July 21, 2022 11:56 PM > To: Xiang, Haihao ; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for > QSV decoders > > Xiang, Haihao: > > On Fri, 2022-07-01 at 20:48 +, ffmpegagent wrote: > >> Missing SEI information has always been a major drawback when > using the QSV > >> decoders. I used to think that there's no chance to get at the > data without > >> explicit implementation from the MSDK side (or doing something > weird like > >> parsing in parallel). It turned out that there's a hardly known > api method > >> that provides access to all SEI (h264/hevc) or user data > (mpeg2video). > >> > >> This allows to get things like closed captions, frame packing, > display > >> orientation, HDR data (mastering display, content light level, > etc.) without > >> having to rely on those data being provided by the MSDK as > extended buffers. > >> > >> The commit "Implement SEI parsing for QSV decoders" includes some > hard-coded > >> workarounds for MSDK bugs which I reported: > >> > > https://github.com/Intel-Media- > SDK/MediaSDK/issues/2597#issuecomment-1072795311 > >> > >> But that doesn't help. Those bugs exist and I'm sharing my > workarounds, > >> which are empirically determined by testing a range of files. If > someone is > >> interested, I can provide private access to a repository where we > have been > >> testing this. Alternatively, I could also leave those workarounds > out, and > >> just skip those SEI types. > >> > >> In a previous version of this patchset, there was a concern that > payload > >> data might need to be re-ordered. Meanwhile I have researched this > carefully > >> and the conclusion is that this is not required. > >> > >> My detailed analysis can be found here: > >> https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 > >> > >> v4 > >> > >> * add new dependencies in makefile Now, build still works when > someone uses > >>configure --disable-decoder=h264 --disable-decoder=hevc > >>--disable-decoder=mpegvideo --disable-decoder=mpeg1video > >>--disable-decoder=mpeg2video --enable-libmfx > >> > >> v3 > >> > >> * frame.h: clarify doc text for av_frame_copy_side_data() > >> > >> v2 > >> > >> * qsvdec: make error handling consistent and clear > >> * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants > >> * hevcdec: rename function to ff_hevc_set_side_data(), add doc > text > >> > >> v3 > >> > >> * qsvdec: fix c/p error > >> > >> softworkz (6): > >> avutil/frame: Add av_frame_copy_side_data() and > >> av_frame_remove_all_side_data() > >> avcodec/vpp_qsv: Copy side data from input to output frame > >> avcodec/mpeg12dec: make mpeg_decode_user_data() accessible > >> avcodec/hevcdec: make set_side_data() accessible > >> avcodec/h264dec: make h264_export_frame_props() accessible > >> avcodec/qsvdec: Implement SEI parsing for QSV decoders > >> > >> doc/APIchanges | 4 + > >> libavcodec/Makefile | 6 +- > >> libavcodec/h264_slice.c | 98 --- > >> libavcodec/h264dec.h | 2 + > >> libavcodec/hevcdec.c | 117 +- > >> libavcodec/hevcdec.h | 9 ++ > >> libavcodec/hevcdsp.c | 4 + > >> libavcodec/mpeg12.h | 28 + > >> libavcodec/mpeg12dec.c | 40 +- > >> libavcodec/qsvdec.c | 234 > +++ > >> libavfilter/qsvvpp.c | 6 + > >> libavfilter/vf_overlay_qsv.c | 19 ++- > >> libavutil/frame.c| 67 ++ > >> libavutil/frame.h| 32 + > >> libavutil/version.h | 2 +- > >> 15 files changed, 494 insertions(+), 174 deletions(-) > >> > >> > >> base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 > >> Published-As: > >> https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging- > 31%2Fsoftworkz%2Fsubmit_qsv_sei-v5 > >> Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr- > ffstaging- > >> 31/softworkz/submit_qsv_sei-v5 > >> Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 > >> > >> Range-diff vs v4: > >> > >> 1: 7656477360 = 1: 7656477360 avutil/frame: Add > av_frame_copy_side_data() > >> and av_frame_remove_all_side_data() > >> 2: 06976606c5 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data > from input to > >> output frame > >> 3: 320a8a535c = 3: 320a8a535c avcodec/mpeg12dec: make > >> mpeg_decode_user_data() accessible > >> 4: e58ad6564f = 4: e58ad6564f avcodec/hevcdec: make > set_side_data() > >> accessible > >> 5: a57bfaebb9 = 5: 4c0b6eb4cb avcodec/h264dec: make > >> h264_export_frame_props() accessible > >> 6: 3f2588563e ! 6: 19bc00be4d avcodec/qsvdec: Implement SEI > parsing for QSV > >> decoders > >> @@ Commit message > >> > >> Signed-off-by: softworkz > >> > >> + ## libavcodec/Makefile ## > >> +@@ libavcodec/Makefile:
Re: [FFmpeg-devel] [PATCH v3 0/4] Add derive-device function which searches for existing devices in both directions
> -Original Message- > From: ffmpegagent > Sent: Friday, July 22, 2022 1:40 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Mark Thompson ; Soft Works ; > softworkz > Subject: [PATCH v3 0/4] Add derive-device function which searches for > existing devices in both directions > > This is an updated version of: [PATCH v4 1/1] avutils/hwcontext: When > deriving a hwdevice, search for existing device in both directions > > There has been an objection that the earlier patchset would change > API > behavior, and that this change should be limited to ffmpeg cli. > > To achieve this, the API behavior is left unchanged now and a new > function > av_hwdevice_ctx_get_or_create_derived() is added and used by the > hwupload > and hwmap filters. > > v2: Implemented concept for "weak references" to avoid circular > reference > lockup. > > v3: rebased due to conflicts > > softworkz (4): > avutil/buffer: add av_ref_from_buffer() function > avutils/hwcontext: add derive-device function which searches for > existing devices in both directions > lavu: bump minor version and add doc/APIchanges entry for > av_hwdevice_ctx_get_or_create_derived() > avfilter/hwmap,hwupload: use new > av_hwdevice_ctx_get_or_create_derived > method Hi, @Mark Thompson - would you mind to take a look at this? You were the only one who was objecting and I had reworked this in the sense of our discussion (hopefully). Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] Ping on help output, logging and VAAPI overlay
This is a friendly ping on these recent submissions: Print filter input/output formats in help output https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7737 Fixes and Enhancements for VAAPI Overlay https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7722 Add option to log timing https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7290 Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 3/3] ffmpeg: add video heartbeat capability to fix_sub_duration
> -Original Message- > From: ffmpeg-devel On Behalf Of Jan > Ekström > Sent: Monday, October 10, 2022 2:45 PM > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH v5 3/3] ffmpeg: add video heartbeat > capability to fix_sub_duration > > From: Jan Ekström > > Splits the currently handled subtitle at random access point > packets that can be configured to follow a specific output stream. > > This way the subtitle - which is known to be shown at this time > can be split and passed to muxer before its full duration is > yet known. > > Co-authored-by: Andrzej Nadachowski > Co-authored-by: Bernard Boulay > > Signed-off-by: Jan Ekström I hope you don't mind me to mention that my subtitle filtering patchset provides a way more comprehensive and flexible approach to this. Best, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 11/11] doc/filters.texi: update overlay_vaapi documentation
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Gyan Doshi
> Sent: Monday, October 10, 2022 1:08 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 11/11] doc/filters.texi: update
> overlay_vaapi documentation
>
>
>
> On 2022-10-10 04:24 pm, softworkz wrote:
> > From: softworkz
> >
> > Signed-off-by: softworkz
> > ---
> > doc/filters.texi | 49 +--
> -
> > 1 file changed, 38 insertions(+), 11 deletions(-)
> >
> > diff --git a/doc/filters.texi b/doc/filters.texi
> > index 2d0b5db909..5f4604a834 100644
> > --- a/doc/filters.texi
> > +++ b/doc/filters.texi
> > @@ -26271,30 +26271,57 @@ It takes two inputs and has one output.
> The first input is the "main" video on w
> > The filter accepts the following options:
> >
> > @table @option
> > -
> > @item x
> > -Set the x coordinate of the overlaid video on the main video.
> > -Default value is @code{0}.
> > -
> > @item y
> > -Set the y coordinate of the overlaid video on the main video.
> > -Default value is @code{0}.
> > +Set expressions for the x and y coordinates of the overlaid video
> > +on the main video.
> >
> > -@item w
> > -Set the width of the overlaid video on the main video.
> > -Default value is the width of input overlay video.
> > +Default value is "0" for both expressions.
> >
> > +@item w
> > @item h
> > -Set the height of the overlaid video on the main video.
> > -Default value is the height of input overlay video.
> > +Set expressions for the width and height the overlaid video
> > +on the main video.
>
> The default values should be mentioned here. And also what the
> default
> value means, if not trivial.
>
> Regards,
> Gyan
Yea, you are hitting a point that I had left out because
I wasn't sure how detailed this should be explained:
The expression handling is done analog to overlay_qsv and I've taken
the same defaults which are:
{ "w", "Overlay width", OFFSET(overlay_ow), AV_OPT_TYPE_STRING, {
.str="overlay_iw"}, 0, 255, .flags = FLAGS},
{ "h", "Overlay height", OFFSET(overlay_oh), AV_OPT_TYPE_STRING, {
.str="overlay_ih*w/overlay_iw"}, 0, 255, .flags = FLAGS},
Essentially, the values are defaulting to the frame size of the
overlay input. This is because both, w and overlay_iw are initialized
to the overlay size. The default expression allows to set the width
only and have the height be adjusted proportionally.
But it doesn't work the other way round (setting h only), so
I'm not sure whether it's a good default at all - I just wanted
to have it equal to overlay_qsv..
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Wednesday, August 31, 2022 3:40 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-27 00:48:01) > > 2. "There's no reason why this cannot be handled using the buffer > > and data fields" > > > > I had explained the reasons and in conversation on IRC, Lynne was > > no longer insisting on this AFAIR. > > I did not see this explanation, can you restate it here? > > If you claim the other points were addressed I will look at the > patches > again. > > -- > Anton Khirnov > ___ Friendly Ping :-) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, August 31, 2022 6:02 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Anton Khirnov
> > Sent: Wednesday, August 31, 2022 3:40 AM
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering
> 2022
> >
> > Quoting Soft Works (2022-08-27 00:48:01)
> > > 2. "There's no reason why this cannot be handled using the buffer
> > > and data fields"
> > >
> > > I had explained the reasons and in conversation on IRC, Lynne was
> > > no longer insisting on this AFAIR.
> >
> > I did not see this explanation, can you restate it here?
>
> Sure. Let's look at the AVSubtitleArea struct first:
>
>
> typedef struct AVSubtitleArea {
>
> enum AVSubtitleType type;
> int flags;
>
> int x; ///< top left corner of area.
> int y; ///< top left corner of area.
> int w; ///< widthof area.
> int h; ///< height of area.
> int nb_colors; ///< number of colors in bitmap palette (@ref
> pal).
>
> /**
> * Buffers and line sizes for the bitmap of this subtitle.
> *
> * @{
> */
> AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
> int linesize[AV_NUM_BUFFER_POINTERS];
> /**
> * @}
> */
>
> uint32_t pal[256]; ///< RGBA palette for the bitmap.
>
> char *text;///< 0-terminated plain UTF-8 text
> char *ass; ///< 0-terminated ASS/SSA compatible event
> line.
>
> } AVSubtitleArea;
>
>
>
> These structs are stored in AVFrame like this:
>
>
> /**
> * Number of items in the @ref subtitle_areas array.
> */
> unsigned num_subtitle_areas;
>
> /**
> * Array of subtitle areas, may be empty.
> */
> AVSubtitleArea **subtitle_areas;
>
>
>
> The question was "why this cannot be handled using the buffer
> and data fields?" - there are multiple reasons:
>
> 1. Area Count
>
> In ASS subtitles it's not uncommon that animations are defined
> through subtitle events (regular ass events). This can go as
> far as that dust particles are flying around on the screen and
> each particle has its own subtitle event/line which defines
> how it flies from x to y and how fast, etc.
> Not yet, but in a future update, the ass decoder should be
> improved in a way that it doesn't emit an individual AVFrame
> for each event line but bundles subsequent events together
> when these would have the same start time and duration.
> As a result, we can have AVFrames with dozens or even a hundred
> AVSubtitleArea items in extreme cases.
>
> The buffer and data fields are an array of fixed size (currently
> 8). Increasing to 16 or 32 would not help: there will still be
> cases where this does not suffice.
>
> 2. What exactly should be stored in frame->buf[]?
>
> Should we store the AVSubtitleArea structs as AVBuffer there?
>
> Or should we only store data in those buffers? If yes, which
> data? The subtitle bitmap probably. How about the subtitle
> text - maybe and area has even both. And should we also
> store the palette in some frame->buf[n]?
> If yes, how to keep track of which data is stored in which
> buffer?
> Further, there's a documented convention that requires that
> non-zero buffers are contiguous, i.e. there must not be
> an empty buffer pointer between two other a non-empty buffer
> pointers. This would require to re-arrange the buffers to
> close any gap when some subtitle area data is removed, zeroed
> out or has been converted to another format.
> Closing such gap also require to update any mapping information
> ("which buffer is related to which area?")
>
> That's a lot of tedious work and every API user (or codec/filter
> developer) would need to do it in the right way.
>
>
>
> 3. AVFrame Code Logic Mistmatch
>
> A subtitle frame can have 0 to N areas, which means that a
> frame without any area is still valid. Opposed to that, existing
> (code all over the place in ffmpeg) is treating an AVFrame
> as invalid when the first data buffer is empty,
> i.e. frame->buf[0] != NULL
>
> To accommodate to this situation, subtitle frames are always
> creating a 1-byte buffer for buf[0].
&g
Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Monday, September 5, 2022 7:59 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads > > Quoting Soft Works (2022-09-04 09:43:36) > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Anton Khirnov > > > Sent: Sunday, September 4, 2022 8:58 AM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads > > > > > > Quoting Timo Rothenpieler (2022-09-02 01:46:59) > > > > On 02.09.2022 01:32, Michael Niedermayer wrote: > > > > > Hi all > > > > > > > > > > Theres a use after free issue in H.264 Decoding on d3d11va > with > > > multiple threads > > > > > I dont have the hardware/platform nor do i know the hw > decoding > > > code so i made > > > > > no attempt to fix this beyond asking others to ... > > > > > > > > hwaccel with multiple threads being broken is not exactly a > > > surprise. > > > > So we could just disable that, and always have it be one single > > > thread? > > > > > > We are already disabling it in a way - the frame threading code > > > ensures > > > that threads run one at a time when hwaccel is being used. > > > > > > Is there a described way to repro? I would try whether it still > > happens after removing the lock code in hwcontext_d3d11va.c. > > Those locks are not really needed and might prevent release > > of dx11 resources in proper order. It's a guess only but > > easy to try. > > The problem is not in d3d11 locking code, but in the generic code > that > does not have clear enough ownership rules. Steve already tested that > my > patch from Friday fixes this. Oh I see. I was missing the context. The patch makes sense to me. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [RFC] d3dva security hw+threads
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Sunday, September 4, 2022 8:58 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [RFC] d3dva security hw+threads > > Quoting Timo Rothenpieler (2022-09-02 01:46:59) > > On 02.09.2022 01:32, Michael Niedermayer wrote: > > > Hi all > > > > > > Theres a use after free issue in H.264 Decoding on d3d11va with > multiple threads > > > I dont have the hardware/platform nor do i know the hw decoding > code so i made > > > no attempt to fix this beyond asking others to ... > > > > hwaccel with multiple threads being broken is not exactly a > surprise. > > So we could just disable that, and always have it be one single > thread? > > We are already disabling it in a way - the frame threading code > ensures > that threads run one at a time when hwaccel is being used. Is there a described way to repro? I would try whether it still happens after removing the lock code in hwcontext_d3d11va.c. Those locks are not really needed and might prevent release of dx11 resources in proper order. It's a guess only but easy to try. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Anton Khirnov
> Sent: Wednesday, August 31, 2022 3:40 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
> Quoting Soft Works (2022-08-27 00:48:01)
> > 2. "There's no reason why this cannot be handled using the buffer
> > and data fields"
> >
> > I had explained the reasons and in conversation on IRC, Lynne was
> > no longer insisting on this AFAIR.
>
> I did not see this explanation, can you restate it here?
Sure. Let's look at the AVSubtitleArea struct first:
typedef struct AVSubtitleArea {
enum AVSubtitleType type;
int flags;
int x; ///< top left corner of area.
int y; ///< top left corner of area.
int w; ///< widthof area.
int h; ///< height of area.
int nb_colors; ///< number of colors in bitmap palette (@ref pal).
/**
* Buffers and line sizes for the bitmap of this subtitle.
*
* @{
*/
AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
int linesize[AV_NUM_BUFFER_POINTERS];
/**
* @}
*/
uint32_t pal[256]; ///< RGBA palette for the bitmap.
char *text;///< 0-terminated plain UTF-8 text
char *ass; ///< 0-terminated ASS/SSA compatible event line.
} AVSubtitleArea;
These structs are stored in AVFrame like this:
/**
* Number of items in the @ref subtitle_areas array.
*/
unsigned num_subtitle_areas;
/**
* Array of subtitle areas, may be empty.
*/
AVSubtitleArea **subtitle_areas;
The question was "why this cannot be handled using the buffer
and data fields?" - there are multiple reasons:
1. Area Count
In ASS subtitles it's not uncommon that animations are defined
through subtitle events (regular ass events). This can go as
far as that dust particles are flying around on the screen and
each particle has its own subtitle event/line which defines
how it flies from x to y and how fast, etc.
Not yet, but in a future update, the ass decoder should be
improved in a way that it doesn't emit an individual AVFrame
for each event line but bundles subsequent events together
when these would have the same start time and duration.
As a result, we can have AVFrames with dozens or even a hundred
AVSubtitleArea items in extreme cases.
The buffer and data fields are an array of fixed size (currently
8). Increasing to 16 or 32 would not help: there will still be
cases where this does not suffice.
2. What exactly should be stored in frame->buf[]?
Should we store the AVSubtitleArea structs as AVBuffer there?
Or should we only store data in those buffers? If yes, which
data? The subtitle bitmap probably. How about the subtitle
text - maybe and area has even both. And should we also
store the palette in some frame->buf[n]?
If yes, how to keep track of which data is stored in which
buffer?
Further, there's a documented convention that requires that
non-zero buffers are contiguous, i.e. there must not be
an empty buffer pointer between two other a non-empty buffer
pointers. This would require to re-arrange the buffers to
close any gap when some subtitle area data is removed, zeroed
out or has been converted to another format.
Closing such gap also require to update any mapping information
("which buffer is related to which area?")
That's a lot of tedious work and every API user (or codec/filter
developer) would need to do it in the right way.
3. AVFrame Code Logic Mistmatch
A subtitle frame can have 0 to N areas, which means that a
frame without any area is still valid. Opposed to that, existing
(code all over the place in ffmpeg) is treating an AVFrame
as invalid when the first data buffer is empty,
i.e. frame->buf[0] != NULL
To accommodate to this situation, subtitle frames are always
creating a 1-byte buffer for buf[0].
When we would want subtitle data to be stored in those buffers,
every developer would need to be aware about the requirement
to have that dummy buffer in the first array element. When something
is to be stored, the dummy buffer would need to be freed first
before storing the actual data.
And when the last buffer gets deleted, API users would need to
know to create a new dummy buffer.
That fixed size array might be a good fit for image data, but
it's not for subtitle data.
The approach in my patchset is simple, intuitive and everybody
can easily work with it without needing any special knowledge:
unsigned num_subtitle_areas;
AVSubtitleArea **subtitle_areas;
IMHO, this is the most suitable pattern for this situation.
> If you claim the other points were addressed I will look at the
> patches
> again.
Oh cool, that would be great!
Thanks,
softworkz
___
[FFmpeg-devel] External Library Dependencies and Testability
Hi, I don’t want to get involved in the ipfsgateway discussion, but the part about external library version dependencies and testability reminded me of some recent thoughts: I wonder whether it wouldn’t make sense to maintain a version reference for external libraries in a way that at any point in (repo) time there’s a clearly defined set of dependency library versions. This could provide a number of benefits, like: - enable testability where external libs are involved - improve reproducibility of issues with external libs - serve as an orientation for library versions to use for static builds - possibly more It could still be purely informational but provide a defined reference point for dependency library versions whenever needed or useful. The configure script could get an option like 'strict_references' option would cause it to error out when it doesn't find the exact versions of libs. Another use case for this would be for ensuring reproducibility of (static) builds. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Friday, August 26, 2022 10:47 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-25 00:19:33) > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Jean-Baptiste Kempf > > > Sent: Monday, August 22, 2022 4:39 PM > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > > > Almost exactly identical objections to the basic aspects of the > API > > > were > > > > raised independently by me, Lynne, and Hendrik. > > > > IIUC Soft Works still refuses to address them (though it's not > so > > > easy > > > > to tell in a 200-email thread). > > > > > > OK. I lost the lists of objections, then. > > > > > > -- > > > Jean-Baptiste Kempf - President > > > > > > Could everybody who still has any objection PLEASE name it with > reasoning > > and explain in which way it should be resolved? > > Most of the main objections are mentioned in [1]. As far as I can > tell, none of them were adequately addressed. Hi Anton, thanks a lot for your reply. In fact, all of these were addressed long ago. http://lists.ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288894.html That message you referenced is about 4 issues: 1. "Milliseconds?" I have changed fields to int64_t values in AV_TIME_BASE, please see below. 2. "There's no reason why this cannot be handled using the buffer and data fields" I had explained the reasons and in conversation on IRC, Lynne was no longer insisting on this AFAIR. 3. "I'm not going to accept a field which is instantly deprecated." I have reworked this and there is no longer a field that is instantly deprecated. 3. "As we've discussed multiple times, please merge this into the regular frame PTS field. We already have _2_ necessary stard/end fields." I have addressed this as well already. There are no longer 3 but only 2 fields remaining (int64_t in AV_TIME_BASE): AVFrame.subtitle_timing.start_pts AVFrame.subtitle_timing.duration > Frankly, you write too many words. A good argument about something > like > this should fit in a paragraph. Maybe followed by some extended > explanations and clarifications, but the core of it should be quite > short and right at the top, not buried under heaps of introductory > remarks and examples. And if you cannot boil down your argument to a > few words then maybe it's not a very strong one. It's a complicated matter and after I had seen that the situation wasn't understood, I started to get more into detail. I will try to write less text in the future. > Or maybe people just got tired of repeating the same objections to > the > same patches being submitted again and again. Or maybe people didn't realize that the objections were addressed already? > You are the author of this set, it is _your_ job to keep track of > what has and has not been addressed I do that, and the count of unaddressed objections that I'm aware of is zero. That's why I'm asking whether anybody would still have an objection or might not see her/his objection being addressed adequately. Thanks again, softworkz PS: the one thing that was under discussion on IRC was about why the subtitle start_pts cannot be the same as the AVFrame pts and that was the point I was explaining in so much detail. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 3/3] lavf/id3v2dec: support multiple values and TIPL frames
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> rcombs
> Sent: Thursday, August 25, 2022 1:52 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH 3/3] lavf/id3v2dec: support multiple
> values and TIPL frames
>
> Fixes https://trac.ffmpeg.org/ticket/6949
>
> Ordinary text frames in ID3v2 are allowed to have multiple
> (null-separated) values. This technically isn't allowed in TXXX,
> but it's used in practice by Picard, and supporting it is harmless.
>
> TIPL/IPL (Involved People List) and TMCL (Musician Credits List) work
> similarly to TXXX, but alternate key-value-key-value.
> ---
> libavformat/id3v2.c | 49 ++-
> --
> 1 file changed, 28 insertions(+), 21 deletions(-)
>
> diff --git a/libavformat/id3v2.c b/libavformat/id3v2.c
> index 191a305ffb..335a1436b2 100644
> --- a/libavformat/id3v2.c
> +++ b/libavformat/id3v2.c
> @@ -321,8 +321,12 @@ static void read_ttag(AVFormatContext *s,
> AVIOContext *pb, int taglen,
>AVDictionary **metadata, const char *key)
> {
> uint8_t *dst;
> -int encoding, dict_flags = AV_DICT_DONT_OVERWRITE |
> AV_DICT_DONT_STRDUP_VAL;
> +uint8_t *dst_key = NULL;
> +int encoding, dict_flags = AV_DICT_MULTIKEY |
> AV_DICT_DONT_STRDUP_VAL | AV_DICT_DEDUP;
> unsigned genre;
> +int count = 0;
> +int is_tipl = !(strcmp(key, "TIPL") && strcmp(key, "TMCL") &&
> +strcmp(key, "IPL"));
>
> if (taglen < 1)
> return;
> @@ -330,30 +334,33 @@ static void read_ttag(AVFormatContext *s,
> AVIOContext *pb, int taglen,
> encoding = avio_r8(pb);
> taglen--; /* account for encoding type byte */
>
> -if (decode_str(s, pb, encoding, , ) < 0) {
> -av_log(s, AV_LOG_ERROR, "Error reading frame %s, skipped\n",
> key);
> -return;
> -}
> -
> -if (!(strcmp(key, "TCON") && strcmp(key, "TCO"))
> &&
> -(sscanf(dst, "(%d)", ) == 1 || sscanf(dst, "%d",
> ) == 1) &&
> -genre <= ID3v1_GENRE_MAX) {
> -av_freep();
> -dst = av_strdup(ff_id3v1_genre_str[genre]);
> -} else if (!(strcmp(key, "TXXX") && strcmp(key, "TXX"))) {
> -/* dst now contains the key, need to get value */
> -key = dst;
> +while (taglen > 1) {
int n = 0;
> if (decode_str(s, pb, encoding, , ) < 0) {
> av_log(s, AV_LOG_ERROR, "Error reading frame %s,
> skipped\n", key);
> -av_freep();
> return;
> }
> -dict_flags |= AV_DICT_DONT_STRDUP_KEY;
> -} else if (!*dst)
> -av_freep();
>
> -if (dst)
> -av_dict_set(metadata, key, dst, dict_flags);
> +count++;
> +
> +if (!(strcmp(key, "TCON") && strcmp(key, "TCO"))
> &&
> +(sscanf(dst, "(%d)", ) == 1 || sscanf(dst, "%d",
> ) == 1) &&
(sscanf(dst, "(%d)", ) == 1 || (sscanf(dst, "%d%n", , ) == 1 && n
== strlen(dst))) &&
avoids parsing genre strings starting with numbers (like '2step')
as genre id.
Thanks for resubmitting,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Jean-Baptiste Kempf > Sent: Monday, August 22, 2022 4:39 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > Almost exactly identical objections to the basic aspects of the API > were > > raised independently by me, Lynne, and Hendrik. > > IIUC Soft Works still refuses to address them (though it's not so > easy > > to tell in a 200-email thread). > > OK. I lost the lists of objections, then. > > -- > Jean-Baptiste Kempf - President Could everybody who still has any objection PLEASE name it with reasoning and explain in which way it should be resolved? > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: ... > (though it's not so easy to tell in a 200-email thread) Yes that's true. For that reason it is not helpful to talk about unspecified objections from more than half a year ago. This is not further actionable without having a list of specific objections. When nobody responds, we need to assume that there aren't any left. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lavu: header and documentation for AVWriter
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Nicolas George
> Sent: Wednesday, August 24, 2022 5:18 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH] lavu: header and documentation for
> AVWriter
>
> The actual implementation, tests and uses in the rest of
> FFmpeg code will be committed separately once the API is
> settled.
>
> Signed-off-by: Nicolas George
> ---
> doc/avwriter_intro.md | 109 ++
> libavutil/writer.h| 484
> ++
> 2 files changed, 593 insertions(+)
> create mode 100644 doc/avwriter_intro.md
> create mode 100644 libavutil/writer.h
>
>
> As suggested by JB, here is the header and documentation for
> AVWriter,
> to discuss the principle before investing time in polishing the
> implementation. I expect to discuss the API and if no blockign
> objections are raised push it then spend time on the implementation.
>
> This API is a nicer and much more powerful version of BPrint. It can
> be
> used to simplify existing code where BPrint could help, plus places
> where BPrint could not help.
>
> It also is the prerequisite for more ambitious projects, expecially
> universal serialization of FFmpeg objects (side data and such) into
> standardized formats.
>
> The implementation is in most part done, and I am sure I can deliver.
> What remains is the boring part of integrating the tests in FATE,
> polishing various parts, updating the parts where I changed my mind
> midway, etc.
>
> Note: FF_NEW_SZ is a macro defined elsewhere; it is really part of
> the
> implementation more than the API.
>
>
> diff --git a/doc/avwriter_intro.md b/doc/avwriter_intro.md
> new file mode 100644
> index 00..4fd8a5a4ad
> --- /dev/null
> +++ b/doc/avwriter_intro.md
> @@ -0,0 +1,109 @@
> +# Quick start guide for AVWriter
> +
> +AVWriter is an API to unify functions returning strings (or any
> arbitrary
> +binary buffer) and to make building strings from parts easier. Here
> is a
> +quick introduction with pairs of “what I would do without AVWriter”
> /
> +“how to do it with AVWriter” of example code.
> +
> +## I want a `char*` buffer, the function wants an AVWriter
> +
> +Old-style code:
> +
> +```
> +char *buf;
> +ret = av_something_to_string(, something);
> +if (ret < 0)
> +die("Failed");
> +use_string(buf);
> +av_freep();
> +```
> +
> +Equivalent code with AVWriter:
> +
> +```
> +AVWriter wr = av_dynbuf_writer();
> +av_something_write(wr, something, 0);
> +if (av_writer_get_error(wr, 0) < 0)
> +die("Failed");
Will it be possible to do:
av_something_write(wr, something1, 0);
av_something_write(wr, something2, 0);
av_something_write(wr, something3, 0);
if (av_writer_get_error(wr, 0) < 0)
die("Failed");
or would av_writer_get_error() need to be called after each
av_something_write()?
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] avfilter/vf_showinfo: add wallclock option
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Wednesday, August 17, 2022 11:28 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter/vf_showinfo: add > wallclock option > > Andreas Rheinhardt (12022-08-17): > > Basically already exists: > > https://ffmpeg.org/pipermail/ffmpeg-devel/2021-August/283434.html > > Lacks subsecond precision and timezone information, but a good start. > > Regards, > > -- > Nicolas George It does provide subsecond precision, please see the 'millisec' variable here: https://patchwork.ffmpeg.org/project/ffmpeg/patch/mn2pr04mb5981773a7336c9b46d298354ba...@mn2pr04mb5981.namprd04.prod.outlook.com/ Adding the ability to print time zone information is a good idea, I think. The patchset allows to control whether to print date, time or both. In practical use, I ended up printing a line with the date on startup and print only time information, e.g. like this: 01:40:42.467 ffmpeg version 5.1... 01:40:42.467 built with ... 01:40:42.468 Execution Date: 2022-08-16 01:40:42 01:40:42.499 Input #0, matroska,webm, ... If there is interest, I can submit my latest version of this. Best, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] avfilter/vf_showinfo: add wallclock option
> -Original Message- > From: ffmpeg-devel On Behalf Of > Michael Riedl > Sent: Wednesday, August 17, 2022 11:02 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v2] avfilter/vf_showinfo: add > wallclock option > > On 17.08.2022 10:43, Nicolas George wrote: > > Michael Riedl (12022-08-17): > >> Signed-off-by: Michael Riedl > >> --- > >> libavfilter/vf_showinfo.c | 11 +++ > >> 1 file changed, 11 insertions(+) > > > > What is the intended use case? It seems to me very ad-hoc. A > fftools > > option to add a timestamp to each log line seems a more generic and > > cleaner approach. > > This is a good idea, I will look into that. There exists a patch that does this and might be useful for you: https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=4526=%2A=both Best, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
From: ffmpeg-devel on behalf of Soft Works Sent: Tuesday, August 23, 2022 12:08 AM To: FFmpeg development discussions and patches Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 From: ffmpeg-devel on behalf of Anton Khirnov Sent: Monday, August 22, 2022 2:18 PM To: ffmpeg-devel Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread). --- Anton, thanks for the reply. Please correct me if I'm wrong. From my memory and understanding, the one any only remaining point of discussion was the necessity to have a separate field for subtitle start PTS in addition to the AVFrame's PTS field. I wasn't refusing to make a change, but I have taken a lot of effort to explain the reasons for that necessity. I did that in several chats on IRC, on the ML, and recently, I have written an article especially to address that concern and better explain the background: https://github.com/softworkz/SubtitleFilteringDemos/issues/1 It remained unresponded (but maybe unnoticed?). Bottom line is that without having the separate subtitle pts field, the whole patchset cannot work. @Anton: you said the reason for this is because I would have designed it like this. But this not the case. The reason why this field is needed it due to the way how libavfilter is designed to work. The only way to avoid that second field would be to fundamentally rework the scheduling of frames in filter graphs and making changes to a core part like that (which is working pretty well and reliably btw) would impose a huge risk of regressions and incompatibilities (it's more a guarantee for issues rather than an abstract risk only), which doesn't make any sense to do at this time and in this context. My conclusion is that having that one additional field in AVFrame is by far the better option. Please let me know what you think should be done and what kind of concern you have with regards to the additional field in AVFrame. It might not qualify as a top candidate for "API Beauty", but it's none of the worst ones either. Your preceding arguments were based on the assumption that it could easily be avoided. I hope the referenced article helps to understand why it can't (without fundamental changes to libavfilter). Please also let me know whether there are any other objections or desired changes and I'll try to address them. I am in no way refusing to make changes, as long as they are feasible. Thanks, softworkz Other alternatives that were discussed: 1. Move the SubtitleStartTime field to AVSubtitleArea One AVFrame can have multiple AVSubtitleArea instances (bitmaps in case of graphic subs or text events in case of text subs). But all AVSubtitleArea instances of an AVFrame must always have the same start and duration - so it would be semantically incorrect to have those values at the area level (when a frame has multiple areas, with different timings, which value is valid? The ones from area0 and the values from area1 to areaN are irrelevant?) 2. Put the SubtitleStartTime into AVFrame's user/opaque data field This is what Lynne had suggested. It avoids adding an additional field to AVFrame but it blocks the use of the user/opaque field for actual user data in case of subtitle frames. Also, each subtitle decoder, each subtitle encoder and many subtitle filters would need to cast the user/opaque field of AVFrame back and forth to a time value. Neither of the two options does make much sense to me, but when there would be consensus on any of those, then I'd be ok with it and make those changes. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
From: ffmpeg-devel on behalf of Anton Khirnov Sent: Monday, August 22, 2022 2:18 PM To: ffmpeg-devel Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread). --- Anton, thanks for the reply. Please correct me if I'm wrong. From my memory and understanding, the one any only remaining point of discussion was the necessity to have a separate field for subtitle start PTS in addition to the AVFrame's PTS field. I wasn't refusing to make a change, but I have taken a lot of effort to explain the reasons for that necessity. I did that in several chats on IRC, on the ML, and recently, I have written an article especially to address that concern and better explain the background: https://github.com/softworkz/SubtitleFilteringDemos/issues/1 It remained unresponded (but maybe unnoticed?). Bottom line is that without having the separate subtitle pts field, the whole patchset cannot work. @Anton: you said the reason for this is because I would have designed it like this. But this not the case. The reason why this field is needed it due to the way how libavfilter is designed to work. The only way to avoid that second field would be to fundamentally rework the scheduling of frames in filter graphs and making changes to a core part like that (which is working pretty well and reliably btw) would impose a huge risk of regressions and incompatibilities (it's more a guarantee for issues rather than an abstract risk only), which doesn't make any sense to do at this time and in this context. My conclusion is that having that one additional field in AVFrame is by far the better option. Please let me know what you think should be done and what kind of concern you have with regards to the additional field in AVFrame. It might not qualify as a top candidate for "API Beauty", but it's none of the worst ones either. Your preceding arguments were based on the assumption that it could easily be avoided. I hope the referenced article helps to understand why it can't (without fundamental changes to libavfilter). Please also let me know whether there are any other objections or desired changes and I'll try to address them. I am in no way refusing to make changes, as long as they are feasible. Thanks, softworkz PS: Please excuse potentially bad e.mail formatting, I'm off-site ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpegagent > Sent: Saturday, June 25, 2022 11:58 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Michael Niedermayer ; Andreas Rheinhardt > ; Soft Works ; > Andriy Gelman ; softworkz > > Subject: [PATCH v5 00/25] Subtitle Filtering 2022 > > > Subtitle Filtering 2022 > === > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as > possible on > one side and to provide more clarity and control over the way things > are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides > different > modes for arbitrating subtitle frames in a filter graph. Finally, > each > subtitle filter's documentation has been amended by a section > describing the > filter's timeline behavior (in v3 update). > > > Subtitle Filtering Demos > > > I published a demonstration of subtitle filtering capabilities with > OCR, > text and bitmap subtitle manipulation involved: Demo 1: Text- > Manipulation > with Bitmap Subtitles > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo > 1] As there were no objections in any regard, I hope that we could make some progress regarding subtitle filtering soon. The only recent statement that has been made was something like "this may work in a limited number of cases but probably won't be capable to work for the whole range of real-life situations" I understand how easily such statements can induce doubt into the audience, even on those who tend and try to be neutral in their assessments. I also think that replying once another time with something like "no, this isn't true" won't be much impressive. Instead, I'm sharing a video with you, documenting my internal test runs for text subtitle overlay (with and without subtitle filtering, with and without hw overlay, with a range of hw accelerations for encoding and decoding, scaling or no scaling and two different kinds of source files). The diagrams at the right are a rather consumer focused, but still accurate. Just hwupload/download/hwmap are omitted and implied by the data changing the "swim lanes". https://github.com/softworkz/SubtitleFilteringDemos/tree/master/TestRun1 Commands and logs on request. Best wishes, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Enhancement layers in FFmpeg
> -Original Message- > From: ffmpeg-devel On Behalf Of > Niklas Haas > Sent: Monday, August 1, 2022 3:59 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] Enhancement layers in FFmpeg > > On Mon, 01 Aug 2022 13:17:12 + Soft Works > wrote: > > From my (rather limited) angle of view, my thoughts are these: > > > > When decoding these kinds of sources, a user would typically not > only > > want to do the processing in hardware but the decoding as well. > > > > I think we cannot realistically expect that any of the hw decoders > > will add support for this in the near future. As we cannot modify > > those ourselves, the only way to do such processing would be a > > hardware filter. I think, the EL data would need to be attached > > to frames as some kind of side data (or similar) and get uploaded > > by the hw filter (internally) which will apply the EL data. > > If both the BL and the EL are separate fully coded bitstreams, then > could we instantiate two independent HW decoder instances to decode > the > respective planes? Sure. TBH, I didn't know that the EL data is encoded in the same way. I wonder how those frames would look like when viewed standalone.. > > IMO it would be desirable when both of these things would/could be > > done in a single operation. > > For Dolby Vision we have little choice in the matter. The EL > application > needs to happen *after* chroma interpolation, PQ linearization, IPT > matrix application, and poly/MMR reshaping. These are currently all > on-GPU processes in the relevant video output codebases. > > So for Dolby Vision that locks us into the design where we merely > expose > the EL planes as part of the AVFrame and leave it to be the user's > problem If ffmpeg cannot apply it, then I don't think there will be many users being able to make some use of it :-) > (or the problem of filters like `vf_libplacebo`). Something I always wanted to ask you: is it even thinkable to port this to a CPU implementation (with reasonable performance)? > An open question (for me) is whether or not this is required for > SVC-H264, SHVC, AV1-SVC etc. > > > As long as it doesn't have its own format, its own start time, > > resolution, duration, color space/transfer/primaries, etc.. > > I wouldn’t say that it's a frame. > > Indeed, it seems like the EL data is tied directly to the BL data for > the formats I have seen so far. So they are just like extra planes on > the AVFrame - and indeed, we could simply use extra data pointers > here > (we already have room for 8). Hendrik's idea makes sense to me when this is not just some data but real frames, decoded with a regular decoder. Yet I don't know anything about the other enhancement cases either. Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Enhancement layers in FFmpeg
> -Original Message- > From: ffmpeg-devel On Behalf Of > Niklas Haas > Sent: Monday, August 1, 2022 1:25 PM > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] Enhancement layers in FFmpeg > > Hey, > > We need to think about possible ways to implement reasonably- > transparent > support for enhancement layers in FFmpeg. (SVC, Dolby Vision, ...). > There are more open questions than answers here. > > From what I can tell, these are basically separate bitstreams that > carry > some amount of auxiliary information needed to reconstruct the > high-quality bitstream. That is, they are not independent, but need > to > be merged with the original bitstream somehow. > > How do we architecturally fit this into FFmpeg? Do we define a new > codec > ID for each (common/relevant) combination of base codec and > enhancement > layer, e.g. HEVC+DoVi, H.264+SVC, ..., or do we transparently handle > it > for the base codec ID and control it via a flag? Do the enhancement > layer packets already make their way to the codec, and if not, how do > we > ensure that this is the case? > > Can the decoder itself recursively initialize a sub-decoder for the > second bitstream? And if so, does the decoder apply the actual > transformation, or does it merely attach the EL data to the AVFrame > somehow in a way that can be used by further filters or end users? From my (rather limited) angle of view, my thoughts are these: When decoding these kinds of sources, a user would typically not only want to do the processing in hardware but the decoding as well. I think we cannot realistically expect that any of the hw decoders will add support for this in the near future. As we cannot modify those ourselves, the only way to do such processing would be a hardware filter. I think, the EL data would need to be attached to frames as some kind of side data (or similar) and get uploaded by the hw filter (internally) which will apply the EL data. (I have no useful thoughts for sw decoding) > (What about the case of Dolby Vision, which iirc requires handling > the > DoVi RPU metadata before the EL can be applied? What about instances > where the user wants the DoVi/EL application to happen on GPU, e.g. > via > libplacebo in mpv/vlc?) IMO it would be desirable when both of these things would/could be done in a single operation. > How does this metadata need to be attached? A second AVFrame > reference > inside the AVFrame? Raw data in a big side data struct? As long as it doesn't have its own format, its own start time, resolution, duration, color space/transfer/primaries, etc.. I wouldn’t say that it's a frame. Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider HW acceleration method when selecting decoder
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Xiang, Haihao
> Sent: Monday, August 1, 2022 3:53 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider HW
> acceleration method when selecting decoder
>
> On Fri, 2022-07-29 at 09:46 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel On Behalf Of
> > > Ronald S. Bultje
> > > Sent: Friday, July 29, 2022 10:50 AM
> > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org>
> > > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider
> HW
> > > acceleration method when selecting decoder
> > >
> > > Hi,
> > >
> > > On Fri, Jul 29, 2022 at 4:38 PM Xiang, Haihao <
> > > haihao.xiang-at-intel@ffmpeg.org> wrote:
> > >
> > > > libdav1d is the preferred AV1 decoder in FFmpeg, libdav1d is
> always
> > >
> > > used
> > > > when
> > > > running the command below even if user expects vaapi or other
> HW
> > > > acceleration
> > > > methods.
> > > >
> > >
> > > I think that is a pretty serious issue. Don't we always want to
> > > prefer a hw
> > > decoder by default? I agree there should also be ways to force-
> select
> > > specific hw/sw decoders, but shouldn't hw be the default?
> >
> > I don't think that this would be reasonably possible in any way.
> > There are a lot of questions which ffmpeg cannot answer, e.g.:
> >
> > - Which hwaccel to choose?
> > - Which hwaccel is available?
> > - Which parameters are required for selecting a device
> > that is working?
> > - Is the auto-selected device even capable to decode a certain
> > input?
> > (pixel format, bit depth, codec profile, frame size, ...)
> >
> > For the user who is creating the command line, it is important to
> > be able to rely on what is going to happen. If they can't command
> > lines will fail:
> >
> > - The outputs of hw decoders vary. Some output to hw format, some
> > to sw format by default. You may need to specify
> hwaccel_output_format
> > or use hwdownload
> > - Depending on the hwaccel, a totally different set of filters
> > may be required (not to speak of encoders)
> > - How would you specify that you want a sw decoder instead?
> >
> > IMO, predictability is the topmost important behavioral aspect
> > for users (be it humans of machines.. ;-)
>
>
> I agree with you it would be better not to make hw the default if
> user doesn't
> ask for. However '-hwaccel arg' is used to specify hw acceleration
> method in the
> examples. According to doc (
> https://github.com/FFmpeg/FFmpeg/blob/master/doc/ffmpeg.texi#L1260-
> L1262), a hwdecoder is expected.
>
> "@item -hwaccel[:@var{stream_specifier}] @var{hwaccel}
> (@emph{input,per-stream})
> Use hardware acceleration to decode the matching stream(s). "
>
> This patch is to fix the issue that a SW decoder is chosen even if
> user is
> expecting a hw method, not to make hw the default in any case.
>
> Thanks
> Haihao
>
Oh, I'm in no way opposed to your patch, I just talked about the
idea of a general automatic selection of hardware decoders.
I agree that the behavior for AV1 should be the same as for the other
decoders.
Best wishes,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider HW acceleration method when selecting decoder
> -Original Message- > From: ffmpeg-devel On Behalf Of > Ronald S. Bultje > Sent: Friday, July 29, 2022 10:50 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider HW > acceleration method when selecting decoder > > Hi, > > On Fri, Jul 29, 2022 at 4:38 PM Xiang, Haihao < > haihao.xiang-at-intel@ffmpeg.org> wrote: > > > libdav1d is the preferred AV1 decoder in FFmpeg, libdav1d is always > used > > when > > running the command below even if user expects vaapi or other HW > > acceleration > > methods. > > > > I think that is a pretty serious issue. Don't we always want to > prefer a hw > decoder by default? I agree there should also be ways to force-select > specific hw/sw decoders, but shouldn't hw be the default? I don't think that this would be reasonably possible in any way. There are a lot of questions which ffmpeg cannot answer, e.g.: - Which hwaccel to choose? - Which hwaccel is available? - Which parameters are required for selecting a device that is working? - Is the auto-selected device even capable to decode a certain input? (pixel format, bit depth, codec profile, frame size, ...) For the user who is creating the command line, it is important to be able to rely on what is going to happen. If they can't command lines will fail: - The outputs of hw decoders vary. Some output to hw format, some to sw format by default. You may need to specify hwaccel_output_format or use hwdownload - Depending on the hwaccel, a totally different set of filters may be required (not to speak of encoders) - How would you specify that you want a sw decoder instead? IMO, predictability is the topmost important behavioral aspect for users (be it humans of machines.. ;-) Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc 7 and higher
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Thursday, July 28, 2022 3:11 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc
> 7 and higher
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > James Almer
> > Sent: Thursday, July 28, 2022 3:05 AM
> > To: ffmpeg-devel@ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization for
> gcc
> > 7 and higher
> >
> > On 7/27/2022 10:02 PM, Soft Works wrote:
> > >
> > >> -Original Message-
> > >> From: ffmpeg-devel On Behalf
> Of
> > >> Hendrik Leppkes
> > >> Sent: Wednesday, July 27, 2022 10:42 PM
> > >> To: FFmpeg development discussions and patches > >> de...@ffmpeg.org>
> > >> Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization
> for
> > gcc
> > >> 7 and higher
> > >>
> > >> On Wed, Jul 27, 2022 at 7:39 PM James Almer
> > >> wrote:
> > >>> On 7/27/2022 2:34 PM, Swinney, Jonathan wrote:
> > >>>> I recognize that this patch is going to be somewhat
> > >> controversial. I'm submitting it mostly to see what the opinions
> > are
> > >> and evaluate options. I am working on improving performance for
> > >> aarch64. On that architecture, there are fewer hand written
> > assembly
> > >> implementations of hot functions than there are for x86_64 and
> > >> allowing gcc to auto-vectorize yields noticeable improvements.
> > >>>> Gcc vectorization has improved recently and it hasn't been
> > >> evaluated on the mailing list for a few years. This is the
> latest
> > >> discussion I found in my searches:
> > >> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/193977.html
> > >>> Every time this was done, it was inevitably reverted after
> > >> complains and
> > >>> crash reports started piling up because gcc can't really handle
> > all
> > >> the
> > >>> inline code our codebase has, among other things.
> > >>>
> > >> No need to wait for issues, I just tested, and the same issues
> > still
> > >> persist that have existed for years with GCC now. They don't
> seem
> > to
> > >> care to make it compatible with inline asm, which might be fair
> > >> enough, but it means it just can't work here.
> > >>
> > >> In file included from libavcodec/cabac_functions.h:49,
> > >> from libavcodec/h264_cabac.c:36:
> > >> libavcodec/h264_cabac.c: In function 'ff_h264_decode_mb_cabac':
> > >> libavcodec/x86/cabac.h:199:5: error: 'asm' operand has
> impossible
> > >> constraints
> > > I wonder why it doesn't fail when I try the same on MINGW32:
> > >
> > > gcc -I. -Isrc/ -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1 -
> > D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -
> > U__STRICT_ANSI__ -D__USE_MINGW_ANSI_STDIO=1 -
> > D__printf__=__gnu_printf__ -D_POSIX_C_SOURCE=200112 -
> > D_XOPEN_SOURCE=600 -DOPJ_STATIC -DZLIB_CONST -DHAVE_AV_CONFIG_H -
> > DBUILDING_avcodec -mthreads -DLIBTWOLAME_STATIC -std=c11 -
> > IV:/ffbuild/mas/local32/include -
> > IV:/ffbuild/mas/msys64/mingw32/include -I/mingw32/include -
> > IF:/ffbuild/mas/local32/include -DLIBARCHIVE_STATIC -Wdeclaration-
> > after-statement -Wall -Wdisabled-optimization -Wpointer-arith -
> > Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing-
> > prototypes -Wstrict-prototypes -Wempty-body -Wno-parentheses -Wno-
> > switch -Wno-format-zero-length -Wno-pointer-sign -Wno-unused-const-
> > variable -Wno-bool-operation -Wno-char-subscripts -O3 -
> Werror=format-
> > security -Werror=implicit-function-declaration -Werror=missing-
> > prototypes -Werror=return-type -Werror=vla -Wformat -fdiagnostics-
> > color=auto -Wno-maybe-uninitialized
> > -
> > > ftree-vectorize -MMD -MF libavcodec/h264_cabac.d -MT
> > libavcodec/h264_cabac.o -c -o libavcodec/h264_cabac.o
> > src/libavcodec/h264_cabac.c
> >
> > You didn't set CPU to haswell (Which will add -march=haswell to the
> > command line).
>
> Yup, you're right - this way I get the same error as Hendrik. Thanks!
>
> But then, when changing -O3 to -O2, it's compiling without
> error again.
Adding
#pragma GCC optimize("no-tree-vectorize")
to get_cabac_inline_x86() allows compiling even with -O3
(the attribute approach doesn't seem to work).
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc 7 and higher
> -Original Message- > From: ffmpeg-devel On Behalf Of > James Almer > Sent: Thursday, July 28, 2022 3:05 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc > 7 and higher > > On 7/27/2022 10:02 PM, Soft Works wrote: > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Hendrik Leppkes > >> Sent: Wednesday, July 27, 2022 10:42 PM > >> To: FFmpeg development discussions and patches >> de...@ffmpeg.org> > >> Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization for > gcc > >> 7 and higher > >> > >> On Wed, Jul 27, 2022 at 7:39 PM James Almer > >> wrote: > >>> On 7/27/2022 2:34 PM, Swinney, Jonathan wrote: > >>>> I recognize that this patch is going to be somewhat > >> controversial. I'm submitting it mostly to see what the opinions > are > >> and evaluate options. I am working on improving performance for > >> aarch64. On that architecture, there are fewer hand written > assembly > >> implementations of hot functions than there are for x86_64 and > >> allowing gcc to auto-vectorize yields noticeable improvements. > >>>> Gcc vectorization has improved recently and it hasn't been > >> evaluated on the mailing list for a few years. This is the latest > >> discussion I found in my searches: > >> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/193977.html > >>> Every time this was done, it was inevitably reverted after > >> complains and > >>> crash reports started piling up because gcc can't really handle > all > >> the > >>> inline code our codebase has, among other things. > >>> > >> No need to wait for issues, I just tested, and the same issues > still > >> persist that have existed for years with GCC now. They don't seem > to > >> care to make it compatible with inline asm, which might be fair > >> enough, but it means it just can't work here. > >> > >> In file included from libavcodec/cabac_functions.h:49, > >> from libavcodec/h264_cabac.c:36: > >> libavcodec/h264_cabac.c: In function 'ff_h264_decode_mb_cabac': > >> libavcodec/x86/cabac.h:199:5: error: 'asm' operand has impossible > >> constraints > > I wonder why it doesn't fail when I try the same on MINGW32: > > > > gcc -I. -Isrc/ -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1 - > D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE - > U__STRICT_ANSI__ -D__USE_MINGW_ANSI_STDIO=1 - > D__printf__=__gnu_printf__ -D_POSIX_C_SOURCE=200112 - > D_XOPEN_SOURCE=600 -DOPJ_STATIC -DZLIB_CONST -DHAVE_AV_CONFIG_H - > DBUILDING_avcodec -mthreads -DLIBTWOLAME_STATIC -std=c11 - > IV:/ffbuild/mas/local32/include - > IV:/ffbuild/mas/msys64/mingw32/include -I/mingw32/include - > IF:/ffbuild/mas/local32/include -DLIBARCHIVE_STATIC -Wdeclaration- > after-statement -Wall -Wdisabled-optimization -Wpointer-arith - > Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing- > prototypes -Wstrict-prototypes -Wempty-body -Wno-parentheses -Wno- > switch -Wno-format-zero-length -Wno-pointer-sign -Wno-unused-const- > variable -Wno-bool-operation -Wno-char-subscripts -O3 -Werror=format- > security -Werror=implicit-function-declaration -Werror=missing- > prototypes -Werror=return-type -Werror=vla -Wformat -fdiagnostics- > color=auto -Wno-maybe-uninitialized > - > > ftree-vectorize -MMD -MF libavcodec/h264_cabac.d -MT > libavcodec/h264_cabac.o -c -o libavcodec/h264_cabac.o > src/libavcodec/h264_cabac.c > > You didn't set CPU to haswell (Which will add -march=haswell to the > command line). Yup, you're right - this way I get the same error as Hendrik. Thanks! But then, when changing -O3 to -O2, it's compiling without error again. softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc 7 and higher
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Hendrik Leppkes
> Sent: Wednesday, July 27, 2022 10:42 PM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc
> 7 and higher
>
> On Wed, Jul 27, 2022 at 7:39 PM James Almer
> wrote:
> >
> > On 7/27/2022 2:34 PM, Swinney, Jonathan wrote:
> > > I recognize that this patch is going to be somewhat
> controversial. I'm submitting it mostly to see what the opinions are
> and evaluate options. I am working on improving performance for
> aarch64. On that architecture, there are fewer hand written assembly
> implementations of hot functions than there are for x86_64 and
> allowing gcc to auto-vectorize yields noticeable improvements.
> > >
> > > Gcc vectorization has improved recently and it hasn't been
> evaluated on the mailing list for a few years. This is the latest
> discussion I found in my searches:
> http://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/193977.html
> >
> > Every time this was done, it was inevitably reverted after
> complains and
> > crash reports started piling up because gcc can't really handle all
> the
> > inline code our codebase has, among other things.
> >
>
> No need to wait for issues, I just tested, and the same issues still
> persist that have existed for years with GCC now. They don't seem to
> care to make it compatible with inline asm, which might be fair
> enough, but it means it just can't work here.
>
> In file included from libavcodec/cabac_functions.h:49,
> from libavcodec/h264_cabac.c:36:
> libavcodec/h264_cabac.c: In function 'ff_h264_decode_mb_cabac':
> libavcodec/x86/cabac.h:199:5: error: 'asm' operand has impossible
> constraints
I wonder why it doesn't fail when I try the same on MINGW32:
gcc -I. -Isrc/ -D_FORTIFY_SOURCE=0 -D__USE_MINGW_ANSI_STDIO=1 -D_ISOC99_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -U__STRICT_ANSI__
-D__USE_MINGW_ANSI_STDIO=1 -D__printf__=__gnu_printf__ -D_POSIX_C_SOURCE=200112
-D_XOPEN_SOURCE=600 -DOPJ_STATIC -DZLIB_CONST -DHAVE_AV_CONFIG_H
-DBUILDING_avcodec -mthreads -DLIBTWOLAME_STATIC -std=c11
-IV:/ffbuild/mas/local32/include -IV:/ffbuild/mas/msys64/mingw32/include
-I/mingw32/include -IF:/ffbuild/mas/local32/include -DLIBARCHIVE_STATIC
-Wdeclaration-after-statement -Wall -Wdisabled-optimization -Wpointer-arith
-Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing-prototypes
-Wstrict-prototypes -Wempty-body -Wno-parentheses -Wno-switch
-Wno-format-zero-length -Wno-pointer-sign -Wno-unused-const-variable
-Wno-bool-operation -Wno-char-subscripts -O3 -Werror=format-security
-Werror=implicit-function-declaration -Werror=missing-prototypes
-Werror=return-type -Werror=vla -Wformat -fdiagnostics-color=auto
-Wno-maybe-uninitialized -
ftree-vectorize -MMD -MF libavcodec/h264_cabac.d -MT libavcodec/h264_cabac.o
-c -o libavcodec/h264_cabac.o src/libavcodec/h264_cabac.c
When I add garbage to line 199 in cabac.h, it errors, so I'm sure it
gets compiled. Same for the av_always_inline line above.
gcc version is 10.3.0. I wonder whether it's about some of the compiler
flags that it doesn't error here, but I couldn't reproduce with various
combinations. Maybe you can spot a difference?
>From my experience, tree-vectorize can provide quite some improvements
in certain cases, but I often had to rewrite the loops (primarily
simplifying) until these got actually vectorized in the way I wanted.
Another conclusion from that work is that there's hardly any benefit
in using tree-vectorize in combination with O3. When O3 is specified,
gcc preferres loop-unrolling over vectorization in the vast majority
of cases (often slower).
Even worse is that loop-unrolling cannot be disabled individually
(neither globally with -O3 -fno-unroll-loops - nor locally with
function __attribute__ or pragma gcc optimize)
I had done a (small) number of tests doing typical stuff to compare
O2 and O3 and I couldn't notice any relevant advantages of O3.
It wasn't exhaustive and very likely one can find cases where O3
performs better, but the vectorization advantages on the other
side were actually relevant, so I had chosen to change all our builds
to O2.
Looking at my notes I remember that I had tried a number of things
to control gcc optimizations at the function level. It didn't
work for me to activate vectorization optimizations (which are
globally disabled), but maybe it works the other way round.
What you could try is either:
#pragma GCC optimize("no-tree-vectorize")
or
#ifdef __GNUC__
__attribute__((optimize("-fno-tree-vectorize")))
#endif
to decorate the function which errors at your side (or maybe
even at the upstream caller).
Maybe this allows to disable vectorization locally for the
erroring case.
Best regards,
softworkz
PS: The observations I made were for x86_64 code in the
context of ffmpeg compiled with gcc 10 (maybe 9) and analyzed
with Intel
Re: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc 7 and higher
> -Original Message- > From: ffmpeg-devel On Behalf Of > Swinney, Jonathan > Sent: Wednesday, July 27, 2022 7:35 PM > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH] enable auto vectorization for gcc 7 > and higher > > I recognize that this patch is going to be somewhat controversial. > I'm submitting it mostly to see what the opinions are and evaluate > options. I am working on improving performance for aarch64. On that > architecture, there are fewer hand written assembly implementations > of hot functions than there are for x86_64 and allowing gcc to auto- > vectorize yields noticeable improvements. > > Gcc vectorization has improved recently and it hasn't been evaluated > on the mailing list for a few years. This is the latest discussion I > found in my searches: http://ffmpeg.org/pipermail/ffmpeg-devel/2016- > May/193977.html > > If the community is not comfortable accepting a patch like this > outright, would you be willing to accept a new option to the > configure script, something like --enable-auto-vectorization? > > Thanks! > > Signed-off-by: Jonathan Swinney > --- > configure | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/configure b/configure > index 6629d14099..c63c9348ad 100755 > --- a/configure > +++ b/configure > @@ -7173,7 +7173,9 @@ if enabled icc; then > disable aligned_stack > fi > elif enabled gcc; then > -check_optflags -fno-tree-vectorize > +case $gcc_basever in > +2|2.*|3.*|4.*|5.*|6.*) check_optflags -fno-tree-vectorize ;; > +esac > check_cflags -Werror=format-security > check_cflags -Werror=implicit-function-declaration > check_cflags -Werror=missing-prototypes > -- LGTM - basically. I had removed that flag about two years ago and never seen an issue (Win,Linux,BSD x x86_64,armv7,aarch64). But it has always been with quite recent versions of gcc, so I can't say whether it's safe already with 7.x One exception I've seen was with an Android NDK build in gcc compatibility mode, where I got a clang compilation error. But that's nothing of concern I think. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I suspect that if I were to do a full review, I would find a few > other flaws. This is a very good example for the kind of behavior I'd kindly ask you to stop with. Do you think you have found a flaw? === => please name it and explain it, ideally asking before calling it a "flaw" (like your one "flaw" wasn't really one as I had laid out) Do you think you COULD find a flaw? === => Go ahead and find one - no need to tell Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I hesitated a long time before replying, but all considering, at this > point expressing what is weighing on my heart cannot make things > worse. > > > Michael Niedermayer (12022-07-03): > > What is the timeline for the audio+video merge ? > > I cannot give a timeline because I do not work on FFmpeg on a > schedule, > I work on FFmpeg in my free time, for fun. And lately, working on > FFmpeg > has been really un-fun. Recently, my contributions have been met with > indifference at best (see when I proposed a XML parser six months > ago: > nobody cared), outright hostility at worst. Under these > circumstances, > whenever I am considering working on something FFmpeg-related, I > usually > find something else more fun to do. > > I do not recognize the project I started contributing to more than > fifteen years ago. I do not even recognize the project that boasted > the > clever optimization framework that made FFVP9 possible, it has become > increasingly hostile to trying new and more efficient ways of doing > things in favor of a corporate never-take-risks style of coding. I am > more and more often considering giving up and cutting my losses. > > > IIUC this would resolve this deadlock (with extra work adapting the > patchset > > so it would be work for SW adapting it and it would be work for you > finishing > > the merge) > > Also can others help nicolas moving his work forward > > > > What i suggest is to pick a time and then try to finish the merge > before. > > If it succeeds this patchset needs updating and can move forward > without > > this main objection > > OTOH if the time is not hit, we agree that the objection can no > longer be > > used > > My answer as maintainer of the framework of libavfilter is: no. > > Of course, maintainers are not dictators. The members of the project > can > collectively decide otherwise. But I have to warn you about the > consequences. > > First, the issue about negotiation is not he only severe flaw in this > patch series. Negotiation hasn't been implemented for audio+video yet. Neither does that patchset do it for audio+video+subtitles. It is out of scope of this patchset. It can be done later or never, not everybody is a fan of doing so, as comments have shown. Clearly, this is in no way a showstopping reason as you had conceded yourself recently. > I can immediately quote another one: for text > subtitles, > the approach of this proposal to synchronization is to feed > everything > to libass as it comes and see what comes out. It will work on easy > cases, when the subtitles are interleaved with the video or come from > a separate file. - or come from decoded closed captions - or come from graphic subtitles converted with the graphicsub2text filter - or come from the subfeed filter after fixing durations - or come from the subfeed filter ensuring a regular repetition (heartbeat) Subtitle events don't need to come in linear order. Multiple events can have identical start times, subtitle events can overlap. The overlaytextsubs filter is meant to be a direct replacement for the existing subtitles filter, which performs additional opening, parsing and decoding of the source file in parallel, and avoiding that was one of the primary objectives I had for starting development. That's why it was very important for me to preserve the exact same behavior as the overlaytextsubs filter exposes. Other approaches for implementatino are surely possible as well. Traian, who did the text2graphicsub filter had initially an implementation that handled the timing manually instead of letting libass do it, but it turned out that this can quickly become a really complex task, especially when overlapping events or animations are part of the game, so it came down to feeding everything to libass in the end, like the overlaytextsubs filter and the subtitles filters do. The nice thing about having subtitle filtering is that there is no fixed functionality involved where you can argue about right or wrong: anyone is free to contribute another filter which is pursuing a different approach. I would welcome that and there may be cases where an alternative method could be advantageous, but it surely won't be superior in general. > But as soon as a filter will, for example, adjust the > timing to make the subtitles more early, it will just not work. Of > course, it was not tested because this patch series does not even > offer the feature to adjust the time of subtitles, which is frankly > ridiculous, it is one of the most obvious thing people might want to > do. The only reason why there is no timing adjustment filter is that I didn't need one. It is really easy to implement such filter. The
Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for QSV decoders
> -Original Message- > From: Xiang, Haihao > Sent: Tuesday, July 19, 2022 8:55 AM > To: ffmpeg-devel@ffmpeg.org > Cc: andreas.rheinha...@outlook.com; kier...@obe.tv; haihao.xiang-at- > intel@ffmpeg.org; softwo...@hotmail.com > Subject: Re: [FFmpeg-devel] [PATCH v5 0/6] Implement SEI parsing for > QSV decoders > > On Fri, 2022-07-01 at 20:48 +, ffmpegagent wrote: > > Missing SEI information has always been a major drawback when using > the QSV > > decoders. I used to think that there's no chance to get at the data > without > > explicit implementation from the MSDK side (or doing something > weird like > > parsing in parallel). It turned out that there's a hardly known api > method > > that provides access to all SEI (h264/hevc) or user data > (mpeg2video). > > > > This allows to get things like closed captions, frame packing, > display > > orientation, HDR data (mastering display, content light level, > etc.) without > > having to rely on those data being provided by the MSDK as extended > buffers. > > > > The commit "Implement SEI parsing for QSV decoders" includes some > hard-coded > > workarounds for MSDK bugs which I reported: > > > https://github.com/Intel-Media-SDK/MediaSDK/issues/2597#issuecomment- > 1072795311 > > > > But that doesn't help. Those bugs exist and I'm sharing my > workarounds, > > which are empirically determined by testing a range of files. If > someone is > > interested, I can provide private access to a repository where we > have been > > testing this. Alternatively, I could also leave those workarounds > out, and > > just skip those SEI types. > > > > In a previous version of this patchset, there was a concern that > payload > > data might need to be re-ordered. Meanwhile I have researched this > carefully > > and the conclusion is that this is not required. > > > > My detailed analysis can be found here: > > https://gist.github.com/softworkz/36c49586a8610813a32270ee3947a932 > > > > v4 > > > > * add new dependencies in makefile Now, build still works when > someone uses > >configure --disable-decoder=h264 --disable-decoder=hevc > >--disable-decoder=mpegvideo --disable-decoder=mpeg1video > >--disable-decoder=mpeg2video --enable-libmfx > > > > v3 > > > > * frame.h: clarify doc text for av_frame_copy_side_data() > > > > v2 > > > > * qsvdec: make error handling consistent and clear > > * qsvdec: remove AV_CODEC_ID_MPEG1VIDEO constants > > * hevcdec: rename function to ff_hevc_set_side_data(), add doc > text > > > > v3 > > > > * qsvdec: fix c/p error > > > > softworkz (6): > > avutil/frame: Add av_frame_copy_side_data() and > > av_frame_remove_all_side_data() > > avcodec/vpp_qsv: Copy side data from input to output frame > > avcodec/mpeg12dec: make mpeg_decode_user_data() accessible > > avcodec/hevcdec: make set_side_data() accessible > > avcodec/h264dec: make h264_export_frame_props() accessible > > avcodec/qsvdec: Implement SEI parsing for QSV decoders > > > > doc/APIchanges | 4 + > > libavcodec/Makefile | 6 +- > > libavcodec/h264_slice.c | 98 --- > > libavcodec/h264dec.h | 2 + > > libavcodec/hevcdec.c | 117 +- > > libavcodec/hevcdec.h | 9 ++ > > libavcodec/hevcdsp.c | 4 + > > libavcodec/mpeg12.h | 28 + > > libavcodec/mpeg12dec.c | 40 +- > > libavcodec/qsvdec.c | 234 > +++ > > libavfilter/qsvvpp.c | 6 + > > libavfilter/vf_overlay_qsv.c | 19 ++- > > libavutil/frame.c| 67 ++ > > libavutil/frame.h| 32 + > > libavutil/version.h | 2 +- > > 15 files changed, 494 insertions(+), 174 deletions(-) > > > > > > base-commit: 6a82412bf33108111eb3f63076fd5a51349ae114 > > Published-As: > > https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging- > 31%2Fsoftworkz%2Fsubmit_qsv_sei-v5 > > Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr- > ffstaging- > > 31/softworkz/submit_qsv_sei-v5 > > Pull-Request: https://github.com/ffstaging/FFmpeg/pull/31 > > > > Range-diff vs v4: > > > > 1: 7656477360 = 1: 7656477360 avutil/frame: Add > av_frame_copy_side_data() > > and av_frame_remove_all_side_data() > > 2: 06976606c5 = 2: 06976606c5 avcodec/vpp_qsv: Copy side data > from input to > > output frame > > 3: 320a8a535c = 3: 320a8a535c avcodec/mpeg12dec: make > > mpeg_decode_user_data() accessible > > 4: e58ad6564f = 4: e58ad6564f avcodec/hevcdec: make > set_side_data() > > accessible > > 5: a57bfaebb9 = 5: 4c0b6eb4cb avcodec/h264dec: make > > h264_export_frame_props() accessible > > 6: 3f2588563e ! 6: 19bc00be4d avcodec/qsvdec: Implement SEI > parsing for QSV > > decoders > > @@ Commit message > > > > Signed-off-by: softworkz > > > > + ## libavcodec/Makefile ## > > +@@ libavcodec/Makefile: OBJS-$(CONFIG_MSS34DSP) > += > > mss34dsp.o > > +
Re: [FFmpeg-devel] [PATCH v10 10/13] lavu/hwcontext_qsv: make qsv hwdevice works with oneVPL
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Xiang, Haihao
> Sent: Tuesday, July 19, 2022 9:19 AM
> To: an...@khirnov.net; ffmpeg-devel@ffmpeg.org
> Cc: Galin, Artem
> Subject: Re: [FFmpeg-devel] [PATCH v10 10/13] lavu/hwcontext_qsv:
> make qsv hwdevice works with oneVPL
>
> On Mon, 2022-07-18 at 15:02 +0200, Anton Khirnov wrote:
> > Quoting Xiang, Haihao (2022-07-12 08:27:32)
> > > +static int qsv_va_update_config(void *ctx, mfxHDL handle,
> mfxConfig cfg)
> > > +{
> > > +#if CONFIG_VAAPI
> > > +#if VA_CHECK_VERSION(1, 5, 0)
> > > +#define LOCAL_VADISPLAYPCIID VADisplayPCIID
> > > +#else
> > > +#define LOCAL_VADISPLAYPCIID 21
> > > +#endif
> > > +mfxStatus sts;
> > > +VADisplay dpy = handle;
> > > +VAStatus vas;
> > > +VADisplayAttribute attr = {
> > > +.type = LOCAL_VADISPLAYPCIID
> > > +};
> > > +mfxVariant impl_value;
> > > +
> > > +vas = vaGetDisplayAttributes(dpy, , 1);
> > > +if (vas == VA_STATUS_SUCCESS && attr.flags !=
> > > VA_DISPLAY_ATTRIB_NOT_SUPPORTED) {
> > > +impl_value.Type = MFX_VARIANT_TYPE_U16;
> > > +impl_value.Data.U16 = (attr.value & 0x);
> > > +sts = MFXSetConfigFilterProperty(cfg,
> > > + (const mfxU8
> > > *)"mfxExtendedDeviceId.DeviceID", impl_value);
> > > +if (sts != MFX_ERR_NONE) {
> > > +av_log(ctx, AV_LOG_ERROR, "Error adding a MFX
> configuration"
> > > + "DeviceID property: %d.\n", sts);
> > > +goto fail;
> > > +}
> > > +} else
> > > +av_log(ctx, AV_LOG_WARNING, "Cannot get device id from
> the driver,
> > > the default "
> > > + "MFX implementation will be loaded for this
> device. Please
> > > consider to "
> > > + "upgrade the driver to support VAAPI 1.5.0. \n");
> >
> > I would still prefer to fail here. The user requested a specific
> device,
> > disregarding that request is evil.
>
> Thanks for the comment. There is only one available device for most
> users, so
> the default one and the given one from user should be the same,
> otherwise it
> won't work. I don't want to make them in trouble if they don't have a
> driver to
> support the new interface. However I agree with you it is a little
> evil to
> ignore the request. I'll update the patch to return error here.
I'm not a fan of that kind of automagic behavior. Quick success
experiences are surely desirable in general, but we also need to
consider the effects of such behavior - in this case, that would
mean: It doesn't really matter what a user specifies for the parameter,
because it will always work anyway.
In turn, users may start to think that their wrong command with the
wrong ID would be right, and then, in a subsequent command
use that wrong ID again in different context, where it might fail,
while in turn maximizing confusion.
When it is possible to internally retrieve potentially valid
values, why not output something useful like: "XXID failed, you
might want to try A, B or C" (or similar)?
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, lavfi.c wrapped_avframe and dependent changes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Timo Rothenpieler > Sent: Saturday, July 9, 2022 2:15 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, > lavfi.c wrapped_avframe and dependent changes > > On 09.07.2022 02:01, Soft Works wrote: > > I can submit the missing bit of differences as a patch. I thought > > it was no longer needed. The requirements have also changed over > > MSDK versions. The 8bit file mapping that I've shown recently > exists > > in fact because an earlier MSDK version was requesting such > texture. > > > > Anyway, when something is causing trouble, then it should be fixed > > in hwcontext_qsv. > > Settings those flags correctly isn't at all a bad idea though, and > not > fixing any issue I actively experienced. Just an oversight I noticed > while reading the code. > > Fixing the heap overflow is the major issue and can really only be > done > in the d3d11 hwcontext. > Look at the texture_flags array. > It only ever gets initialized to the size of the initial pool size. > With a non-fixed-size pool, that grows over time, that obviously is > an > issue, and will trash whatever comes after the hwcontext on the heap > whenever more frames are requested than initially allocated. > With a initial size of 0, that is... immediately. > > > An entire other issue this does not address at all, but which also > does > not cause any memory corruption at least: > > QSV can't properly deal with a non-fixed-size pool. > What happens if more fresh frames are allocated after the qsv hwctx > has > been derived? > From the looks of it, it iterates and maps all the textures only > once > at init. There's an allocator pattern which filters are using, so when using non-array textures it might be possible to have bigger pools than textures allocated. But that's just a "might". Please give me one or two days to respond with a better and more comprehensive answer. Right now, I'm deeply drowned into something else ;-) Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, lavfi.c wrapped_avframe and dependent changes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Timo Rothenpieler > Sent: Saturday, July 9, 2022 1:49 AM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, > lavfi.c wrapped_avframe and dependent changes > > On 09.07.2022 01:46, Soft Works wrote: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of > >> Timo Rothenpieler > >> Sent: Saturday, July 9, 2022 12:54 AM > >> To: ffmpeg-devel@ffmpeg.org > >> Cc: Timo Rothenpieler > >> Subject: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, > lavfi.c > >> wrapped_avframe and dependent changes > >> > >> Since a bunch small stuff has changed since the last time I sent > >> these > >> one by one, here's the whole collection again. > >> I intend to push the whole lot within the next 48h, so they all > make > > > > 48h? > > > > If I'm not mistaken, the first submission was just two days ago: > > Yeah, and then a release deadline came up. > > > avutil/hwcontext_d3d11va: fix texture_infos writes on non-fixed- > size pools > > > > I hadn't seen that one before: > > > > avutil/hwcontext_d3d11va: update hwctx flags from input texture > > Those were already discussed on IRC, and are basically fixing up a > mess > that was made when adding QSV interop. This was my code which I had done in 2019 and Intel had later adopted it. One Part of the 5% differences where the deviated from my implementation was -- guess what: Setting of those flags... > Right now it just trashes the heap without those patches. > > Just look at all the stuff happening in the qsv hwcontext. > It relies on all of those flags being correct, but only did the bare > minimum to ensure that. > So this is a bunch of missing bits and pieces to at least not make it > crash and burn. I can submit the missing bit of differences as a patch. I thought it was no longer needed. The requirements have also changed over MSDK versions. The 8bit file mapping that I've shown recently exists in fact because an earlier MSDK version was requesting such texture. Anyway, when something is causing trouble, then it should be fixed in hwcontext_qsv. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, lavfi.c wrapped_avframe and dependent changes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Soft Works > Sent: Saturday, July 9, 2022 1:46 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Timo Rothenpieler > Subject: Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, > lavfi.c wrapped_avframe and dependent changes > > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Timo Rothenpieler > > Sent: Saturday, July 9, 2022 12:54 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Timo Rothenpieler > > Subject: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, > lavfi.c > > wrapped_avframe and dependent changes > > > > Since a bunch small stuff has changed since the last time I sent > > these > > one by one, here's the whole collection again. > > I intend to push the whole lot within the next 48h, so they all > make > > 48h? > > If I'm not mistaken, the first submission was just two days ago: > > avutil/hwcontext_d3d11va: fix texture_infos writes on non-fixed-size > pools > > I hadn't seen that one before: > > avutil/hwcontext_d3d11va: update hwctx flags from input texture > > > Please give me some time to take a look. Could you please also explain which situations this is supposed to fix? Thank you, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, lavfi.c wrapped_avframe and dependent changes
> -Original Message- > From: ffmpeg-devel On Behalf Of > Timo Rothenpieler > Sent: Saturday, July 9, 2022 12:54 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Timo Rothenpieler > Subject: [FFmpeg-devel] [PATCH v2 0/8] ddagrab source filter, lavfi.c > wrapped_avframe and dependent changes > > Since a bunch small stuff has changed since the last time I sent > these > one by one, here's the whole collection again. > I intend to push the whole lot within the next 48h, so they all make 48h? If I'm not mistaken, the first submission was just two days ago: avutil/hwcontext_d3d11va: fix texture_infos writes on non-fixed-size pools I hadn't seen that one before: avutil/hwcontext_d3d11va: update hwctx flags from input texture Please give me some time to take a look. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d11va: add BGRA/RGBA10 formats support
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Timo Rothenpieler
> Sent: Tuesday, July 5, 2022 6:15 PM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Timo Rothenpieler
> Subject: [FFmpeg-devel] [PATCH] avutil/hwcontext_d3d11va: add
> BGRA/RGBA10 formats support
>
> Desktop duplication outputs those
> ---
> libavutil/hwcontext_d3d11va.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/libavutil/hwcontext_d3d11va.c
> b/libavutil/hwcontext_d3d11va.c
> index 904d14bbc8..1f3d1b7755 100644
> --- a/libavutil/hwcontext_d3d11va.c
> +++ b/libavutil/hwcontext_d3d11va.c
> @@ -86,6 +86,8 @@ static const struct {
> } supported_formats[] = {
> { DXGI_FORMAT_NV12, AV_PIX_FMT_NV12 },
> { DXGI_FORMAT_P010, AV_PIX_FMT_P010 },
> +{ DXGI_FORMAT_B8G8R8A8_UNORM,AV_PIX_FMT_BGRA },
> +{ DXGI_FORMAT_R10G10B10A2_UNORM, AV_PIX_FMT_X2BGR10 },
> // Special opaque formats. The pix_fmt is merely a place holder,
> as the
> // opaque format cannot be accessed directly.
> { DXGI_FORMAT_420_OPAQUE, AV_PIX_FMT_YUV420P },
> --
LGTM - at least I can say that for the first one, which I have
for many years already. My current list at this place is this:
static const struct {
DXGI_FORMAT d3d_format;
enum AVPixelFormat pix_fmt;
} supported_formats[] = {
{ DXGI_FORMAT_NV12,AV_PIX_FMT_NV12 },
{ DXGI_FORMAT_P010,AV_PIX_FMT_P010 },
{ DXGI_FORMAT_Y210,AV_PIX_FMT_Y210 },
{ DXGI_FORMAT_P8, AV_PIX_FMT_PAL8 },
{ DXGI_FORMAT_B8G8R8A8_UNORM, AV_PIX_FMT_BGRA },
{ DXGI_FORMAT_P016,AV_PIX_FMT_P016 },
{ DXGI_FORMAT_YUY2,AV_PIX_FMT_YUYV422 },
// Special opaque formats. The pix_fmt is merely a place holder, as the
// opaque format cannot be accessed directly.
{ DXGI_FORMAT_420_OPAQUE, AV_PIX_FMT_YUV420P },
};
Best regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper type for NALU type
> -Original Message- > From: ffmpeg-devel On Behalf Of > Andreas Rheinhardt > Sent: Sunday, July 3, 2022 12:21 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Andreas Rheinhardt > Subject: [FFmpeg-devel] [PATCH 01/20] avcodec/hevc_sei: Use proper > type for NALU type > > Signed-off-by: Andreas Rheinhardt > --- Nice! That's helpful for the QSV SEI parsing. The one missing bit is the HDR data (AVMasteringDisplayMetadata and AVContentLightMetadata) assignment which still seems to remain in hevcdec. Would it make sense for factor this out as well? Thank you very much, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Saturday, July 2, 2022 10:40 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; Andriy Gelman > ; Andreas Rheinhardt > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Sat, Jul 2, 2022 at 10:32 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Nicolas George > > > Sent: Saturday, July 2, 2022 9:36 PM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Cc: Michael Niedermayer ; Andriy Gelman > > > ; Andreas Rheinhardt > > > > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > Soft Works (12022-07-02): > > > > This can easily be done AFTER my patchset has been merged. > > > > > > With exponentially more work. Out of question. > > > > Previously it would have been about like: > > > > - Merging audio filter code with the video filter code > > (for the filters in question) > > > > Now it will be > > > > - Merging audio and subtitle filter code with the video filter code > > (for the same filters) > > > > TBH, I can't see any exponent here. I think "double work" would be > > closer to the truth and realistically it will be much less than > that > > because the work for merging audio and subtitle code is very > similar, > > so when you have merged audio code in a filter, merging the > subtitle > > part will be very analogous, so in total it would be less than > > double. > > > > And when we look at the required amount of work in total, that > > calculation would only be valid when you would consider the value > > of MY work that I've already done as zero. > > > > I even think that it's a better approach overall to do the > deduplication > > afterwards, because now - with the subtitle filtering patchset - > the > > specific requirements for subtitle filtering are visible on the > table > > and that way, the deduplication can already provision for those > > specific requirements whereas focusing on audio/video only, might > have > > led to do changes that wouldn't accommodate for the needs of > subtitle > > filtering. > > > > I am convinced that doing deduplication afterwards is a better > order > > for getting this done. I'm also convinced that my patchset is > pretty > > solid in the way it does handle subtitles, and I'm further > convinced > > that you know that very well. During all the process I have watched > > very closely, and in several cases where others had objections > about > > things I had done, you kept quiet, presumably because you were the > only > > other one to know why it had to be done that way. Also, you never > > named any specific detail that would be wrong, and I'm sure you > would > > have done if there had been any significant one. > > My impression is that your primary reason for objection is that my > > patchset interferes with your plans and visions you probably had in > > mind for quite a while and I'm very sorry about that. > > But in the end, my patchset doesn't stand in opposition to your > plans, > > it just requires a bit of adaption regarding the order of doing the > work. > > Neither do I stand in opposition to your plans. I respect the > technical > > architecture of libavfilter, especially regarding its simplicity > and > > effectivity compared to other filtering frameworks (like > DirectShow) > > and my interest in Ffmpeg filtering is not limited to subtitles. > > We don't need to be friends, but when you would manage to act and > > communicate in a friendly way, you might gain somebody to help with > > and support your plans in the future and you would also do a favor > > to all readers of the ML by not having them read through despicable > > conversations. > > > > AFAIK only NIcolas is for this merge of different types of filters > into > single filter, > and filter type negotiation. > > Both ideas are very bad. And Nicolas is (still) not (yet) FFmpeg > dictator. I think it makes sense for certain filters like buffer source, buffer sink, null sink, trim, showinfo, copy, delay, repeat, etc. Yet there is no point in making this a pre-condition for subtitle filtering, it's out of scope. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
