[FFmpeg-devel] request to revert all patches that made cfr decoders vfr

[Paul B Mahol]

This is the first warning to revert all patches that made cfr decoders vfr.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 0/5] Switch mmaldec to receive_frame API

[Ming Shun Ho]

While we are on the topic of mmaldec, would you mind reviewing and
applying this patch as well:

https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210916134652.22781-1-cyph1...@gmail.com/

On Fri, Dec 10, 2021 at 11:48 AM Ming Shun Ho  wrote:
>
> Built and tested successfully on a Rpi 3b+ through mpv.
>
> Thanks for this patchset!
>
> On Thu, Dec 9, 2021 at 9:01 PM Andreas Rheinhardt
>  wrote:
> >
> > This is an updated and extended version of [1] by Ho Ming Shun
> > together with some additions by me. Notice that I am unable
> > to test these patches (not even compilation), so I hope that
> > others (Ho Ming Shun) do so.
> >
> > [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html
> >
> > Andreas Rheinhardt (2):
> >   avcodec/mmaldec: Avoid creating unnecessary reference, simplify code
> >   avcodec/mmaldec: Deduplicate AVClasses
> >
> > Ho Ming Shun (3):
> >   avcodec/mmaldec: use decoupled dataflow
> >   avcodec/mmaldec: re-use AVPacket for extra_data
> >   avcodec/mmaldec: fix pointer type warning
> >
> >  libavcodec/mmaldec.c | 81 +++-
> >  1 file changed, 42 insertions(+), 39 deletions(-)
> >
> > --
> > 2.32.0
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH V3 2/2] libavutil/hwcontext_opencl: fix a bug for mapping qsv frame to opencl

[Wenbin Chen]

From: nyanmisaka 

mfxHDLPair was added to qsv, so modify qsv->opencl map function as well.
Now the following commandline works:

ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128 \
-init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device ocl \
-hwaccel qsv -hwaccel_output_format qsv -hwaccel_device qs -c:v h264_qsv \
-i input.264 -vf "hwmap=derive_device=opencl,format=opencl,avgblur_opencl, \
hwmap=derive_device=qsv:reverse=1:extra_hw_frames=32,format=qsv" \
-c:v h264_qsv output.264

Signed-off-by: nyanmisaka 
Signed-off-by: Wenbin Chen 
---
 libavutil/hwcontext_opencl.c | 3 ++-
 libavutil/hwcontext_qsv.h| 5 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c
index 26a3a24593..ab812999cd 100644
--- a/libavutil/hwcontext_opencl.c
+++ b/libavutil/hwcontext_opencl.c
@@ -48,6 +48,7 @@
 #if HAVE_OPENCL_VAAPI_INTEL_MEDIA
 #if CONFIG_LIBMFX
 #include 
+#include "hwcontext_qsv.h"
 #endif
 #include 
 #include 
@@ -2249,7 +2250,7 @@ static int opencl_map_from_qsv(AVHWFramesContext *dst_fc, 
AVFrame *dst,
 #if CONFIG_LIBMFX
 if (src->format == AV_PIX_FMT_QSV) {
 mfxFrameSurface1 *mfx_surface = (mfxFrameSurface1*)src->data[3];
-va_surface = *(VASurfaceID*)mfx_surface->Data.MemId;
+va_surface = *MFXSURFACEP_TO_VASURFACEP(mfx_surface);
 } else
 #endif
 if (src->format == AV_PIX_FMT_VAAPI) {
diff --git a/libavutil/hwcontext_qsv.h b/libavutil/hwcontext_qsv.h
index b98d611cfc..957df01ef1 100644
--- a/libavutil/hwcontext_qsv.h
+++ b/libavutil/hwcontext_qsv.h
@@ -29,6 +29,11 @@
  * contain AVBufferRefs whose data pointer points to an mfxFrameSurface1 
struct.
  */
 
+#if CONFIG_VAAPI
+#define MFXSURFACEP_TO_VASURFACEP(surf) \
+(VASurfaceID*)(((mfxHDLPair*)surf->Data.MemId)->first)
+#endif
+
 /**
  * This struct is allocated as AVHWDeviceContext.hwctx
  */
-- 
2.25.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH V3 1/2] libavcodec/vaapi_decode: fix the problem that init_pool_size < nb_surface

[Wenbin Chen]

For vaapi if the init_pool_size is not zero, the pool size is fixed.
This means max surfaces is init_pool_size, but when mapping vaapi
frame to qsv frame, the init_pool_size < nb_surface. The cause is that
vaapi_decode_make_config() config the init_pool_size and it is called
twice. The first time is to init frame_context and the second time is to
init codec. On the second time the init_pool_size is changed to original
value so the init_pool_size is lower than the reall size because
pool_size used to initialize frame_context need to plus thread_count and
3 (guarantee 4 base work surfaces). Now add code to make sure
init_pool_size is only set once. Now the following commandline works:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \
-hwaccel_output_format vaapi -i input.264 \
-vf "hwmap=derive_device=qsv,format=qsv" \
-c:v h264_qsv output.264

Signed-off-by: Wenbin Chen 
---
 libavcodec/vaapi_decode.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
index 665af370ed..da0e72b10b 100644
--- a/libavcodec/vaapi_decode.c
+++ b/libavcodec/vaapi_decode.c
@@ -650,7 +650,7 @@ int ff_vaapi_decode_init(AVCodecContext *avctx)
 ctx->hwctx  = ctx->device->hwctx;
 
 err = vaapi_decode_make_config(avctx, ctx->frames->device_ref,
-   &ctx->va_config, avctx->hw_frames_ctx);
+   &ctx->va_config, NULL);
 if (err)
 goto fail;
 
-- 
2.25.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 0/5] Switch mmaldec to receive_frame API

[Ming Shun Ho]

Built and tested successfully on a Rpi 3b+ through mpv.

Thanks for this patchset!

On Thu, Dec 9, 2021 at 9:01 PM Andreas Rheinhardt
 wrote:
>
> This is an updated and extended version of [1] by Ho Ming Shun
> together with some additions by me. Notice that I am unable
> to test these patches (not even compilation), so I hope that
> others (Ho Ming Shun) do so.
>
> [1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html
>
> Andreas Rheinhardt (2):
>   avcodec/mmaldec: Avoid creating unnecessary reference, simplify code
>   avcodec/mmaldec: Deduplicate AVClasses
>
> Ho Ming Shun (3):
>   avcodec/mmaldec: use decoupled dataflow
>   avcodec/mmaldec: re-use AVPacket for extra_data
>   avcodec/mmaldec: fix pointer type warning
>
>  libavcodec/mmaldec.c | 81 +++-
>  1 file changed, 42 insertions(+), 39 deletions(-)
>
> --
> 2.32.0
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH V2 1/3] libavcodec/vaapi_decode: fix the problem that init_pool_size < nb_surface

[Chen, Wenbin]

> Quoting Wenbin Chen (2021-11-30 09:44:04)
> > For vaapi if the init_pool_size is not zero, the pool size is fixed.
> > This means max surfaces is init_pool_size, but when mapping vaapi
> > frame to qsv frame, the init_pool_size < nb_surface. The cause is that
> > vaapi_decode_make_config() config the init_pool_size and it is called
> > twice. The first time is to init frame_context and the second time is to
> > init codec. On the second time the init_pool_size is changed to original
> > value so the init_pool_size is lower than the reall size because
> > pool_size used to initialize frame_context need to plus thread_count and
> > 3 (guarantee 4 base work surfaces). Now add code to make sure
> > init_pool_size is only set once. Now the following commandline works:
> >
> > ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 \
> > -hwaccel_output_format vaapi -i input.264 \
> > -vf "hwmap=derive_device=qsv,format=qsv" \
> > -c:v h264_qsv output.264
> >
> > Signed-off-by: Wenbin Chen 
> > ---
> >  libavcodec/vaapi_decode.c | 34 ++
> >  1 file changed, 18 insertions(+), 16 deletions(-)
> >
> > diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> > index 665af370ed..aab8162989 100644
> > --- a/libavcodec/vaapi_decode.c
> > +++ b/libavcodec/vaapi_decode.c
> > @@ -572,22 +572,24 @@ static int
> vaapi_decode_make_config(AVCodecContext *avctx,
> >  if (err < 0)
> >  goto fail;
> >
> > -frames->initial_pool_size = 1;
> > -// Add per-codec number of surfaces used for storing reference
> frames.
> > -switch (avctx->codec_id) {
> > -case AV_CODEC_ID_H264:
> > -case AV_CODEC_ID_HEVC:
> > -case AV_CODEC_ID_AV1:
> > -frames->initial_pool_size += 16;
> > -break;
> > -case AV_CODEC_ID_VP9:
> > -frames->initial_pool_size += 8;
> > -break;
> > -case AV_CODEC_ID_VP8:
> > -frames->initial_pool_size += 3;
> > -break;
> > -default:
> > -frames->initial_pool_size += 2;
> > +if (!frames->initial_pool_size) {
> > +frames->initial_pool_size = 1;
> > +// Add per-codec number of surfaces used for storing reference
> frames.
> > +switch (avctx->codec_id) {
> > +case AV_CODEC_ID_H264:
> > +case AV_CODEC_ID_HEVC:
> > +case AV_CODEC_ID_AV1:
> > +frames->initial_pool_size += 16;
> > +break;
> > +case AV_CODEC_ID_VP9:
> > +frames->initial_pool_size += 8;
> > +break;
> > +case AV_CODEC_ID_VP8:
> > +frames->initial_pool_size += 3;
> > +break;
> > +default:
> > +frames->initial_pool_size += 2;
> > +}
> 
> Seems to me that ff_vaapi_decode_init() should not pass hw_frames_ctx to
> vaapi_decode_make_config() at all, because it is already initialized and
> must not be modified.
> 
> --
> Anton Khirnov

You are right. I don't need to add an new check. ff_vaapi_decode_init() should 
pass NULL
to vaapi_decode_make_config. Thanks for your review. I will submit a new patch

Regards
Wenbin
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] lavfi/drawtext: Add localtime_ms for millisecond precision

[zhilizhao(赵志立)]

> On Dec 10, 2021, at 3:11 AM, Thilo Borgmann  wrote:
> 
> Hi,
> 
> add %{localtime_ms}, %{gmtime_ms} functions to the drawtext filter. Same as 
> %{localtime}/%{gmtime} but with additional millisecond part.
> 
> sorry for delay, second version including review remarks:
> 
> -get timing once
> -also add gmtime_ms instead of just localtime_ms
> 

> +if (tag == 'M' || tag == 'm') {
> +char ms[5] = {0};
> +int64_t dnow = (unow - ((int64_t)now) * 100) / 1000;
> +snprintf(ms, 5, ".%03d", (int)dnow);
> +av_bprint_append_data(bp, ms, 4);
> +}
> 


How about

av_bprintf(&bp, ".%03d", (int)(unow % 100) / 1000);

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of first input to set output format for overlay_qsv

[Wenbin Chen]

overlay_qsv hard coded to use nv12 as output format. Now use the format
of the first input to set output format. Now the following command
works:

ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v hevc_qsv \
-i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255, \
hwdownload,format=p010le" -f rawvideo -y output_p010.yuv

Signed-off-by: Wenbin Chen 
---
 libavfilter/vf_overlay_qsv.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c
index 7e76b39aa9..d947a1faa1 100644
--- a/libavfilter/vf_overlay_qsv.c
+++ b/libavfilter/vf_overlay_qsv.c
@@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink)
 int ret;
 
 av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n", 
av_get_pix_fmt_name(outlink->format));
+vpp->qsv_param.out_sw_format = in0->format;
 if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) ||
 (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) {
 av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel formats 
is not supported.\n");
@@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink)
 av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV 
devices are forbidden.\n");
 return AVERROR(EINVAL);
 }
+vpp->qsv_param.out_sw_format = hw_frame0->sw_format;
 }
 
 outlink->w  = vpp->var_values[VAR_MW];
-- 
2.25.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Marton Balint]

On Thu, 9 Dec 2021, Anton Khirnov wrote:


Quoting Nicolas George (2021-12-09 11:31:54)

Anton Khirnov (12021-12-09):

I disagree. Technical limitations that were overcome 10 years ago should
not guide new API design.


In the case of amerge, it was not a technical limitation, merging
several streams into one so that they can be handled by single-stream
filters is 100% part of the design.


I fail to see how that is an advantage. You can just as well create
multiple instances of those single-stream filters instead of adding
hacks into core APIs.


I suspect devices that capture several independent channels are
designed that way intentionally too, possibly to reduce the risk of
desynchronization.


"possibly" is not a strong enough argument. I'd like to hear at least
one clearly-defined use case that cannot just as well be handled by
using multiple streams.


I recently worked on the MXF demuxer to recognize channel designations in 
MXF files, and in MXF the designation and grouping of the channels is 
completely separate from the track those channels are muxed in.


So if you have e.g. english stereo sound, and german stereo sound you 
can mux it

- as a single 4 channel track
- as two 2 channel tracks
- as four 1 channel tracks.
Some MXF flavors use the multichannel single track approach, others the 
mono-track-only approach. So the user may not be able to choose the 
optimal muxed track assignment...


So ultimately, if you demux and decode a packet from a track, you will 
have an AVFrame, which might contain a single sound group on its own 
(optimal case), part of a sound group, or multiple sound groups.


To summerize, muxed tracks are not necessarily 1:1 mapping between sound 
groups. And when processing/filtering audio, you typically want sound 
groups, not tracks. And yes, it is very rare to have a soundgroup which 
has channels with the same designation, but for a muxed track, it depends 
on the format.


The goal of the end user is probably to be able to specify sound groups, 
not select muxed tracks. Preferably a demuxer should provide which channel 
is part of which sound group, and you should be able to use a filter or a 
combination of filters select a specific sound group. E.g. amerge all 
tracks, then keep only the channels from all the merged channels which are 
part of a specific sound group.


Regards,
Marton

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[James Almer]

On 12/9/2021 8:31 PM, Marton Balint wrote:



On Fri, 10 Dec 2021, Hendrik Leppkes wrote:


On Fri, Dec 10, 2021 at 12:06 AM Marton Balint  wrote:




On Thu, 9 Dec 2021, Anton Khirnov wrote:


Quoting Lynne (2021-12-08 13:57:45)

8 Dec 2021, 13:16 by an...@khirnov.net:


Quoting Lynne (2021-12-08 10:02:34)


8 Dec 2021, 02:06 by jamr...@gmail.com:


From: Anton Khirnov 

The new API is more extensible and allows for custom layouts.
More accurate information is exported, eg for decoders that do not
set a channel layout, lavc will not make one up for them.

Deprecate the old API working with just uint64_t bitmasks.

Expanded and completed by Vittorio Giovara 


and James Almer .
Signed-off-by: Vittorio Giovara 
Signed-off-by: James Almer 
---
 libavutil/channel_layout.c | 498 
++--
 libavutil/channel_layout.h | 510 
++---

 libavutil/version.h    |   1 +
 3 files changed, 906 insertions(+), 103 deletions(-)

 /**
 * @}
@@ -128,6 +199,157 @@ enum AVMatrixEncoding {
 AV_MATRIX_ENCODING_NB
 };

+/**
+ * @}
+ */
+
+/**
+ * An AVChannelCustom defines a single channel within a custom 
order layout

+ *
+ * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is 
a part of the

+ * public ABI.
+ *
+ * No new fields may be added to it without a major version bump.
+ */
+typedef struct AVChannelCustom {
+    enum AVChannel id;
+} AVChannelCustom;



Consider adding a 25-byte or so of a description field string 
here that

would also be copied if the frame/layout is copied?
That way, users can assign a description label to each channel,
for example labeling each channel in a 255 channel custom layout
Opus stream used in production at a venue somewhere.
Also, consider additionally reserving 16-bytes here, just in case.



That would enlarge the layout size by a significant amount. Keep 
in mind
that this is all per frame. And for something that very few people 
would

want to use.

These descriptors, if anyone really needs them, can live in side 
data.




That's fine with me.
Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
or an int too, or even a single-byte uint8_t.


like this?

diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
index 7b77a74b61..7b2ba12532 100644
--- a/libavutil/channel_layout.h
+++ b/libavutil/channel_layout.h
@@ -252,7 +252,8 @@ enum AVMatrixEncoding {
 * No new fields may be added to it without a major version bump.
 */
typedef struct AVChannelCustom {
-    enum AVChannel id;
+    enum AVChannel id:16;
+    int  reserved:16;
} AVChannelCustom;


A dynamically allocated field is so bad here? E.g.

AVDictionary *metadata

You can store labels. You can store group names. You can store language.
You can store sound positions, GPS coordiantes, whatever.

Yes, it will be slow. But it is rarely used for in normal cases, and it
can be NULL usually.



Pointers add a lot of trouble with ownership and memory management.
Wouldn't be so bad if its just the pointer without all that management
baggage.


Sure, but since you are dynamically allocating the custom map 
AVChannelLayout->u.map, you are already managing memory...

E.g. you could also store an array of AVDictionaries in AVChannelLayout.


That's one per layout, whereas your suggestion is one per channel.



Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Marton Balint]

On Fri, 10 Dec 2021, Hendrik Leppkes wrote:


On Fri, Dec 10, 2021 at 12:06 AM Marton Balint  wrote:




On Thu, 9 Dec 2021, Anton Khirnov wrote:


Quoting Lynne (2021-12-08 13:57:45)

8 Dec 2021, 13:16 by an...@khirnov.net:


Quoting Lynne (2021-12-08 10:02:34)


8 Dec 2021, 02:06 by jamr...@gmail.com:


From: Anton Khirnov 

The new API is more extensible and allows for custom layouts.
More accurate information is exported, eg for decoders that do not
set a channel layout, lavc will not make one up for them.

Deprecate the old API working with just uint64_t bitmasks.

Expanded and completed by Vittorio Giovara 
and James Almer .
Signed-off-by: Vittorio Giovara 
Signed-off-by: James Almer 
---
 libavutil/channel_layout.c | 498 ++--
 libavutil/channel_layout.h | 510 ++---
 libavutil/version.h|   1 +
 3 files changed, 906 insertions(+), 103 deletions(-)

 /**
 * @}
@@ -128,6 +199,157 @@ enum AVMatrixEncoding {
 AV_MATRIX_ENCODING_NB
 };

+/**
+ * @}
+ */
+
+/**
+ * An AVChannelCustom defines a single channel within a custom order layout
+ *
+ * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is a part of the
+ * public ABI.
+ *
+ * No new fields may be added to it without a major version bump.
+ */
+typedef struct AVChannelCustom {
+enum AVChannel id;
+} AVChannelCustom;



Consider adding a 25-byte or so of a description field string here that
would also be copied if the frame/layout is copied?
That way, users can assign a description label to each channel,
for example labeling each channel in a 255 channel custom layout
Opus stream used in production at a venue somewhere.
Also, consider additionally reserving 16-bytes here, just in case.



That would enlarge the layout size by a significant amount. Keep in mind
that this is all per frame. And for something that very few people would
want to use.

These descriptors, if anyone really needs them, can live in side data.



That's fine with me.
Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
or an int too, or even a single-byte uint8_t.


like this?

diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
index 7b77a74b61..7b2ba12532 100644
--- a/libavutil/channel_layout.h
+++ b/libavutil/channel_layout.h
@@ -252,7 +252,8 @@ enum AVMatrixEncoding {
 * No new fields may be added to it without a major version bump.
 */
typedef struct AVChannelCustom {
-enum AVChannel id;
+enum AVChannel id:16;
+int  reserved:16;
} AVChannelCustom;


A dynamically allocated field is so bad here? E.g.

AVDictionary *metadata

You can store labels. You can store group names. You can store language.
You can store sound positions, GPS coordiantes, whatever.

Yes, it will be slow. But it is rarely used for in normal cases, and it
can be NULL usually.



Pointers add a lot of trouble with ownership and memory management.
Wouldn't be so bad if its just the pointer without all that management
baggage.


Sure, but since you are dynamically allocating the 
custom map AVChannelLayout->u.map, you are already managing memory...

E.g. you could also store an array of AVDictionaries in AVChannelLayout.

Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Hendrik Leppkes]

On Fri, Dec 10, 2021 at 12:06 AM Marton Balint  wrote:
>
>
>
> On Thu, 9 Dec 2021, Anton Khirnov wrote:
>
> > Quoting Lynne (2021-12-08 13:57:45)
> >> 8 Dec 2021, 13:16 by an...@khirnov.net:
> >>
> >>> Quoting Lynne (2021-12-08 10:02:34)
> >>>
>  8 Dec 2021, 02:06 by jamr...@gmail.com:
> 
> > From: Anton Khirnov 
> >
> > The new API is more extensible and allows for custom layouts.
> > More accurate information is exported, eg for decoders that do not
> > set a channel layout, lavc will not make one up for them.
> >
> > Deprecate the old API working with just uint64_t bitmasks.
> >
> > Expanded and completed by Vittorio Giovara 
> > and James Almer .
> > Signed-off-by: Vittorio Giovara 
> > Signed-off-by: James Almer 
> > ---
> >  libavutil/channel_layout.c | 498 ++--
> >  libavutil/channel_layout.h | 510 ++---
> >  libavutil/version.h|   1 +
> >  3 files changed, 906 insertions(+), 103 deletions(-)
> >
> >  /**
> >  * @}
> > @@ -128,6 +199,157 @@ enum AVMatrixEncoding {
> >  AV_MATRIX_ENCODING_NB
> >  };
> >
> > +/**
> > + * @}
> > + */
> > +
> > +/**
> > + * An AVChannelCustom defines a single channel within a custom order 
> > layout
> > + *
> > + * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is a part 
> > of the
> > + * public ABI.
> > + *
> > + * No new fields may be added to it without a major version bump.
> > + */
> > +typedef struct AVChannelCustom {
> > +enum AVChannel id;
> > +} AVChannelCustom;
> >
> 
>  Consider adding a 25-byte or so of a description field string here that
>  would also be copied if the frame/layout is copied?
>  That way, users can assign a description label to each channel,
>  for example labeling each channel in a 255 channel custom layout
>  Opus stream used in production at a venue somewhere.
>  Also, consider additionally reserving 16-bytes here, just in case.
> 
> >>>
> >>> That would enlarge the layout size by a significant amount. Keep in mind
> >>> that this is all per frame. And for something that very few people would
> >>> want to use.
> >>>
> >>> These descriptors, if anyone really needs them, can live in side data.
> >>>
> >>
> >> That's fine with me.
> >> Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
> >> or an int too, or even a single-byte uint8_t.
> >
> > like this?
> >
> > diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
> > index 7b77a74b61..7b2ba12532 100644
> > --- a/libavutil/channel_layout.h
> > +++ b/libavutil/channel_layout.h
> > @@ -252,7 +252,8 @@ enum AVMatrixEncoding {
> >  * No new fields may be added to it without a major version bump.
> >  */
> > typedef struct AVChannelCustom {
> > -enum AVChannel id;
> > +enum AVChannel id:16;
> > +int  reserved:16;
> > } AVChannelCustom;
>
> A dynamically allocated field is so bad here? E.g.
>
> AVDictionary *metadata
>
> You can store labels. You can store group names. You can store language.
> You can store sound positions, GPS coordiantes, whatever.
>
> Yes, it will be slow. But it is rarely used for in normal cases, and it
> can be NULL usually.
>

Pointers add a lot of trouble with ownership and memory management.
Wouldn't be so bad if its just the pointer without all that management
baggage.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] avfilter/af_astats: improve options descriptions

[lance . lmwang]

On Thu, Dec 09, 2021 at 10:41:17PM +0530, Gyan Doshi wrote:
> 
> 
> On 2021-12-09 06:49 am, lance.lmw...@gmail.com wrote:
> > From: Limin Wang 
> > 
> > Signed-off-by: Limin Wang 
> > ---
> >   libavfilter/af_astats.c | 6 +++---
> >   1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/libavfilter/af_astats.c b/libavfilter/af_astats.c
> > index d0810b5..776a8aa 100644
> > --- a/libavfilter/af_astats.c
> > +++ b/libavfilter/af_astats.c
> > @@ -114,8 +114,8 @@ typedef struct AudioStatsContext {
> >   static const AVOption astats_options[] = {
> >   { "length", "set the window length", OFFSET(time_constant), 
> > AV_OPT_TYPE_DOUBLE, {.dbl=.05}, 0, 10, FLAGS },
> >   { "metadata", "inject metadata in the filtergraph", OFFSET(metadata), 
> > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS },
> > -{ "reset", "recalculate stats after this many frames", 
> > OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, FLAGS },
> > -{ "measure_perchannel", "only measure_perchannel these per-channel 
> > statistics", OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, 
> > {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },
> > +{ "reset", "Set number of frame after which stats are going to be 
> > recalculated", OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, 
> > FLAGS },
> 
> "Set the number of frames over which cumulative stats are calculated before
> being reset"
> 
> > +{ "measure_perchannel", "Select the entries which need to be measured 
> > per channel", OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, 
> > {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },
> 
> "Select the parameters which are measured per channel"
> 
> > { "none"  , "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_NONE}, 0, 0, FLAGS, "measure" },
> > { "all"   , "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_ALL }, 0, 0, FLAGS, "measure" },
> > { "DC_offset" , "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_DC_OFFSET   }, 0, 0, FLAGS, "measure" },
> > @@ -143,7 +143,7 @@ static const AVOption astats_options[] = {
> > { "Number_of_NaNs", "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_NUMBER_OF_NANS  }, 0, 0, FLAGS, "measure" },
> > { "Number_of_Infs", "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_NUMBER_OF_INFS  }, 0, 0, FLAGS, "measure" },
> > { "Number_of_denormals"   , "", 0, AV_OPT_TYPE_CONST, 
> > {.i64=MEASURE_NUMBER_OF_DENORMALS }, 0, 0, FLAGS, "measure" },
> > -{ "measure_overall", "only measure_perchannel these overall 
> > statistics", OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, 
> > {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },
> > +{ "measure_overall", "Select the entries which need to be measured 
> > overall", OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 
> > 0, UINT_MAX, FLAGS, "measure" },
> 
> "Select the parameters which are measured overall"

Thanks for the comments, I'm fine with it. But I saw the descriptions are not 
clear so copy them from doc, I'm sure whether we need update the doc also?

> 
> Regards,
> Gyan
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Marton Balint]

On Thu, 9 Dec 2021, Anton Khirnov wrote:


Quoting Lynne (2021-12-08 13:57:45)

8 Dec 2021, 13:16 by an...@khirnov.net:


Quoting Lynne (2021-12-08 10:02:34)


8 Dec 2021, 02:06 by jamr...@gmail.com:


From: Anton Khirnov 

The new API is more extensible and allows for custom layouts.
More accurate information is exported, eg for decoders that do not
set a channel layout, lavc will not make one up for them.

Deprecate the old API working with just uint64_t bitmasks.

Expanded and completed by Vittorio Giovara 
and James Almer .
Signed-off-by: Vittorio Giovara 
Signed-off-by: James Almer 
---
 libavutil/channel_layout.c | 498 ++--
 libavutil/channel_layout.h | 510 ++---
 libavutil/version.h|   1 +
 3 files changed, 906 insertions(+), 103 deletions(-)

 /**
 * @}
@@ -128,6 +199,157 @@ enum AVMatrixEncoding {
 AV_MATRIX_ENCODING_NB
 };

+/**
+ * @}
+ */
+
+/**
+ * An AVChannelCustom defines a single channel within a custom order layout
+ *
+ * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is a part of the
+ * public ABI.
+ *
+ * No new fields may be added to it without a major version bump.
+ */
+typedef struct AVChannelCustom {
+enum AVChannel id;
+} AVChannelCustom;



Consider adding a 25-byte or so of a description field string here that
would also be copied if the frame/layout is copied?
That way, users can assign a description label to each channel,
for example labeling each channel in a 255 channel custom layout
Opus stream used in production at a venue somewhere.
Also, consider additionally reserving 16-bytes here, just in case.



That would enlarge the layout size by a significant amount. Keep in mind
that this is all per frame. And for something that very few people would
want to use.

These descriptors, if anyone really needs them, can live in side data.



That's fine with me.
Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
or an int too, or even a single-byte uint8_t.


like this?

diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
index 7b77a74b61..7b2ba12532 100644
--- a/libavutil/channel_layout.h
+++ b/libavutil/channel_layout.h
@@ -252,7 +252,8 @@ enum AVMatrixEncoding {
 * No new fields may be added to it without a major version bump.
 */
typedef struct AVChannelCustom {
-enum AVChannel id;
+enum AVChannel id:16;
+int  reserved:16;
} AVChannelCustom;


A dynamically allocated field is so bad here? E.g.

AVDictionary *metadata

You can store labels. You can store group names. You can store language. 
You can store sound positions, GPS coordiantes, whatever.


Yes, it will be slow. But it is rarely used for in normal cases, and it 
can be NULL usually.


Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/4] lavu/frame: Add Dolby Vision metadata side data type

[Niklas Haas]

On Thu, 09 Dec 2021 17:21:58 + Derek Buitenhuis 
 wrote:
> On 12/8/2021 10:12 AM, Niklas Haas wrote:
> > +/* based on guesswork, see mkvtoolnix and dovi_tool */
> > +int av_dovi_profile(const AVDOVIRpuDataHeader *hdr)
> > +{
> 
> The correct way to find the profile is from the stream level
> DOVI configuration record side data, if available.

So, I changed the API to accept the DOVI configuration record, but
actually getting that record into hevcdec isimpossible given the current
design of FFmpeg where stream-level properties are not mode available to
the decoder. Possibilities:

1. Add patch to propagate stream-level properties to
   AVCodecContext.coded_side_data automatically
2. Add patch to automatically propagate stream-level properties to each
   AVPacket somehow?
3. Have the code read the profile from the AVPacket even though the side
   data doesn't exist for them, and let this just be an open bug.
4. Something else?

Thoughts?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame\n for subtitle handling

[Daniel Cantarín]

Hi there.
This is my first message to this list, so please excuse me if I 
unintendedly break some rule.


I've read the debate between Soft Works and others, and would like to 
add something to it.
I don't have a deep knowledge of the libs as other people here show. My 
knowledge comes from working with live streams for some years now. And I 
do understand the issue about modifying a public API for some use case 
under debate: I believe it's a legit line of questioning to Soft Works 
patches. However, I also feel we live streaming people are often let 
aside as "border case" when it comes to ffmpeg/libav usage, and this 
bias is present in many subtitles/captions debates.


I work with Digital TV signals as input, and several different target 
outputs more related to live streaming (mobiles, PCs, and so on). The 
target location is Latin America, and thus I need subtitles/captions for 
when we use english spoken audio (we speak mostly Spanish in LATAM). TV 
people send you TV subtitle formats: scte-27, dvb subs, and so on. And 
live streaming people uses other subtitles formats, mostly vtt and ttml. 
I've found that CEA-608 captions are the most compatible caption format, 
as it's understood natively by smart tvs and other devices, as well as 
non-natively by any other device using popular player-side libraries. 
So, I've made my own filter for generating CEA-608 captions for live 
streams, using ffmpeg with the previously available OCR filter. Tried 
VTT first, but it was problematic for live-streaming packaging, and with 
CEA-608 I could just ignore that part of the process.


While doing those filters, besides the whole deal of implementing the 
conversion from text to CEA-608, I struggled with stuff like this:
- the sparseness of input subtitles, leading to OOM in servers and 
stalled players.
- the "libavfilter doesn't take subtitle frames" and "it's all ASS 
internally" issues.
- the "captions timings vs video frame timings vs audio timings" 
problems (people talk a lot about syncing subs with video frames, but 
rarely against actual dialogue audio).

- other (meta)data problems, like screen positioning or text encoding.

This are all problems Soft Works seems to have faced as well.

But of all the problems regarding live streaming subtitles with ffmpeg 
(and there are LOTS of it), the most annoying problem is always this: 
almost every time someone talked about implementing subtitles in filters 
(in mail lists, in tickets, in other places like stack overflow, 
etcetera), they always asumed input files. When the people specifically 
talked about live streams, their peers always reasoned with files 
mindset, and stated live streaming subtitles/captions as "border case".


Let me be clear: this are not "border case" issues, but actually appear 
in the most common use cases of live streaming transcoding. They all 
appear *inmediatelly* when you try to use subtitles/captions in live 
streams.


I got here (I mean this thread) while looking for ways to fixing some 
issues in my setup. I was reconsidering VTT/TTML generation instead of 
CEA-608 (as rendering behave significantly different from device to 
device), and thus I was about to generate subtitle type output from some 
filter, was about to create my own standalone "heartbeat" filter to 
normalize the sparseness, and so on and so on: again, all stuff Soft 
Works seems to be handling as well. So I was quite happy to find someone 
working on this again; last time I've seen it in ffmpeg's 
mailing/patchwork 
(https://patchwork.ffmpeg.org/project/ffmpeg/patch/20161102220934.26010-...@pkh.me) 
the code there seemed to die, and I was already late to say anything 
about it. However, reading the other devs reaction to Soft Works work 
was worrying, as it felt as history wanted to repeat itself (take a look 
at discussions back then).


It has been years so far of this situation. This time I wanted to 
annotate this, as this conversation is still warm, in order to help Soft 
Works's code survive. So, dear devs: I love and respect your work, and 
your opinion is very important to me. I do not claim to know better than 
you do ffmpeg's code. I do not claim to know better what to do with 
libavfilter's API. Please understand: I'm not here to be right, but to 
note my point of view. I'm not better than you; quite on the contrary 
most likely. But I also need to solve some very real problems, and can't 
wait until everything else is in wonderful shape to do it. I can't also 
add lots of conditions in order to just fix the most immediate issues; 
like it's the case with sparseness and heartbeat frames, which was a 
heated debate years ago and seems to still be one, while I find it to be 
the most obvious common sense backwards-compatible solution 
implementation. Stuff like "clean" or "well designed" can't be more 
important than actually working use cases while not breaking previously 
implemented ones: because it's far easier to fix little blocks of "bad" 
cod

[FFmpeg-devel] [PATCH v2] lavfi/drawtext: Add localtime_ms for millisecond precision

[Thilo Borgmann]

Hi,

add %{localtime_ms}, %{gmtime_ms} functions to the drawtext filter. Same 
as %{localtime}/%{gmtime} but with additional millisecond part.


sorry for delay, second version including review remarks:

-get timing once
-also add gmtime_ms instead of just localtime_ms

Thanks,
ThiloFrom 07c940d63df392a592a72dc23fb48f6682284c65 Mon Sep 17 00:00:00 2001
From: Thilo Borgmann 
Date: Thu, 9 Dec 2021 19:57:18 +0100
Subject: [PATCH v2] lavfi/drawtext: Add localtime_ms for millisecond precision

Suggested-By: ffm...@fb.com
---
 doc/filters.texi  |  8 
 libavfilter/vf_drawtext.c | 15 +--
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/doc/filters.texi b/doc/filters.texi
index 78faf76..db75632 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -10949,10 +10949,18 @@ It can be used to add padding with zeros from the 
left.
 The time at which the filter is running, expressed in UTC.
 It can accept an argument: a strftime() format string.
 
+@item gmtime_ms
+Same as @code{gmtime} but with millisecond precision.
+It can accept an argument: a strftime() format string.
+
 @item localtime
 The time at which the filter is running, expressed in the local time zone.
 It can accept an argument: a strftime() format string.
 
+@item localtime_ms
+Same as @code{localtime} but with millisecond precision.
+It can accept an argument: a strftime() format string.
+
 @item metadata
 Frame metadata. Takes one or two arguments.
 
diff --git a/libavfilter/vf_drawtext.c b/libavfilter/vf_drawtext.c
index 382d589..65da1ec 100644
--- a/libavfilter/vf_drawtext.c
+++ b/libavfilter/vf_drawtext.c
@@ -1045,15 +1045,24 @@ static int func_strftime(AVFilterContext *ctx, AVBPrint 
*bp,
  char *fct, unsigned argc, char **argv, int tag)
 {
 const char *fmt = argc ? argv[0] : "%Y-%m-%d %H:%M:%S";
+int64_t unow;
 time_t now;
 struct tm tm;
 
-time(&now);
-if (tag == 'L')
+unow = av_gettime();
+now  = unow / 100;
+if (tag == 'L' || tag == 'm')
 localtime_r(&now, &tm);
 else
 tm = *gmtime_r(&now, &tm);
 av_bprint_strftime(bp, fmt, &tm);
+
+if (tag == 'M' || tag == 'm') {
+char ms[5] = {0};
+int64_t dnow = (unow - ((int64_t)now) * 100) / 1000;
+snprintf(ms, 5, ".%03d", (int)dnow);
+av_bprint_append_data(bp, ms, 4);
+}
 return 0;
 }
 
@@ -1152,7 +1161,9 @@ static const struct drawtext_function {
 { "pict_type", 0, 0, 0,   func_pict_type },
 { "pts",   0, 3, 0,   func_pts  },
 { "gmtime",0, 1, 'G', func_strftime },
+{ "gmtime_ms", 0, 1, 'M', func_strftime },
 { "localtime", 0, 1, 'L', func_strftime },
+{ "localtime_ms", 0, 1, 'm', func_strftime },
 { "frame_num", 0, 0, 0,   func_frame_num },
 { "n", 0, 0, 0,   func_frame_num },
 { "metadata",  1, 2, 0,   func_metadata },
-- 
1.8.3.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Paul B Mahol]

On Thu, Dec 9, 2021 at 5:26 PM Nicolas George  wrote:

> Hendrik Leppkes (12021-12-09):
> > It sounds like thats object audio, and should be driven separately,
> > and not by the legacy channel identifiers.
>
> This is not exclusive. We have code that works with channel layouts
> right now, and it is important that the extensions work gracefully.
>
> For example, if I add a filter to just extract the pair of channels
> related to the strings section, we want that its output is properly
> marked as left and right.
>
> Why would it stop work with new API in place?


> Regards,
>
> --
>   Nicolas George
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 4/4] avcodec/targa: Do not return images when there is no image in the tga

[Paul B Mahol]

NACK

On Thu, Dec 9, 2021 at 1:23 PM Michael Niedermayer 
wrote:

> On Mon, Dec 06, 2021 at 10:58:31AM +0100, Anton Khirnov wrote:
> > Quoting Michael Niedermayer (2021-12-03 15:04:47)
> > > On Sat, Sep 18, 2021 at 06:27:15PM +0200, Michael Niedermayer wrote:
> > > > On Fri, Sep 17, 2021 at 08:06:48PM +0200, Paul B Mahol wrote:
> > > > > On Fri, Sep 17, 2021 at 8:00 PM Michael Niedermayer <
> mich...@niedermayer.cc>
> > > > > wrote:
> > > > >
> > > > > > On Fri, Sep 17, 2021 at 07:35:32PM +0200, Paul B Mahol wrote:
> > > > > > > Please do not apply, This actually changes output to nothing.
> > > > > >
> > > > > > Do you have such a TGA file without an image in it?
> > > > > >
> > > > > > Why would that be relevant, comply with TGA specifications
> please.
> > > > >
> > > > > thx
> > > >
> > > > It is relevant as it would allow to check how such a file is handled
> > > > by various implementations
> > > > The specification does not specify that. It just says
> > > > "No image Data included"
> > > > and our decoder returns a black image, the specification doesnt say
> that
> > > > means "black image" or i missed it when reading
> > > > after the patch the decoder returns nothing instead of a black image
> > > > The other type this might reach is type 8, the specification i have
> doesnt
> > > > say anything about type 8 except that its reserved
> > > >
> > > > Truevision TGAa
> > > > FILE FORMAT SPECIFICATION
> > > > Version 2.0
> > > > ...
> > > > Image Type - Field 3 (1 byte):
> > > > The TGA File Format can be used to store Pseudo-Color, True-Color
> and Direct-Color images of various
> > > > pixel depths. Truevision has currently defined seven image types:
> > > > image   Description
> > > > Type0   No image Data included
> > >
> > > ping
> > > what do you prefer for this issue ?
> > > Id like to do something here not just ignore it even if the conclusion
> is
> > > that this is "not a bug but intended behavior"
> >
> > A brief look at other opensource decoders suggest they don't support
> > type==0 at all. image-rs does, but it's not clear to be what its output
> > is. I'd say not returning a frame is reasonable.
>
> ok, will apply
>
> thx
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The greatest way to live with honor in this world is to be what we pretend
> to be. -- Socrates
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] MAINTAINERS: Add Haihao Xiang for qsv

[Eoff, Ullysses A]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of Zhong Li
> Sent: Tuesday, June 08, 2021 7:12 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Zhong Li 
> Subject: [FFmpeg-devel] [PATCH] MAINTAINERS: Add Haihao Xiang for qsv
> 
> Signed-off-by: Zhong Li 
> ---
>  MAINTAINERS | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index dcac46003e..39ce91b755 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -226,7 +226,7 @@ Codecs:
>ptx.c Ivo van Poorten
>qcelp*Reynaldo H. Verdejo Pinochet
>qdm2.c, qdm2data.hRoberto Togni
> -  qsv*  Mark Thompson, Zhong Li
> +  qsv*  Mark Thompson, Zhong Li, Haihao Xiang
>qtrle.c   Mike Melanson
>ra144.c, ra144.h, ra288.c, ra288.hRoberto Togni
>resample2.c   Michael Niedermayer
> --
> 2.17.1
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[UAE] +1, LGTM


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/4] lavu/frame: Add Dolby Vision metadata side data type

[Derek Buitenhuis]

On 12/8/2021 10:12 AM, Niklas Haas wrote:
> +/* based on guesswork, see mkvtoolnix and dovi_tool */
> +int av_dovi_profile(const AVDOVIRpuDataHeader *hdr)
> +{

The correct way to find the profile is from the stream level
DOVI configuration record side data, if available.

- Derek
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/5] avfilter/af_astats: improve options descriptions

[Gyan Doshi]

On 2021-12-09 06:49 am, lance.lmw...@gmail.com wrote:

From: Limin Wang 

Signed-off-by: Limin Wang 
---
  libavfilter/af_astats.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavfilter/af_astats.c b/libavfilter/af_astats.c
index d0810b5..776a8aa 100644
--- a/libavfilter/af_astats.c
+++ b/libavfilter/af_astats.c
@@ -114,8 +114,8 @@ typedef struct AudioStatsContext {
  static const AVOption astats_options[] = {
  { "length", "set the window length", OFFSET(time_constant), 
AV_OPT_TYPE_DOUBLE, {.dbl=.05}, 0, 10, FLAGS },
  { "metadata", "inject metadata in the filtergraph", OFFSET(metadata), 
AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS },
-{ "reset", "recalculate stats after this many frames", 
OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, FLAGS },
-{ "measure_perchannel", "only measure_perchannel these per-channel statistics", 
OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },
+{ "reset", "Set number of frame after which stats are going to be 
recalculated", OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, FLAGS },


"Set the number of frames over which cumulative stats are calculated 
before being reset"



+{ "measure_perchannel", "Select the entries which need to be measured per channel", 
OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },


"Select the parameters which are measured per channel"


{ "none"  , "", 0, AV_OPT_TYPE_CONST, {.i64=MEASURE_NONE  
  }, 0, 0, FLAGS, "measure" },
{ "all"   , "", 0, AV_OPT_TYPE_CONST, {.i64=MEASURE_ALL   
  }, 0, 0, FLAGS, "measure" },
{ "DC_offset" , "", 0, AV_OPT_TYPE_CONST, {.i64=MEASURE_DC_OFFSET 
  }, 0, 0, FLAGS, "measure" },
@@ -143,7 +143,7 @@ static const AVOption astats_options[] = {
{ "Number_of_NaNs", "", 0, AV_OPT_TYPE_CONST, 
{.i64=MEASURE_NUMBER_OF_NANS  }, 0, 0, FLAGS, "measure" },
{ "Number_of_Infs", "", 0, AV_OPT_TYPE_CONST, 
{.i64=MEASURE_NUMBER_OF_INFS  }, 0, 0, FLAGS, "measure" },
{ "Number_of_denormals"   , "", 0, AV_OPT_TYPE_CONST, 
{.i64=MEASURE_NUMBER_OF_DENORMALS }, 0, 0, FLAGS, "measure" },
-{ "measure_overall", "only measure_perchannel these overall statistics", 
OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },
+{ "measure_overall", "Select the entries which need to be measured overall", 
OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX, FLAGS, "measure" },


"Select the parameters which are measured overall"

Regards,
Gyan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Hendrik Leppkes (12021-12-09):
> It sounds like thats object audio, and should be driven separately,
> and not by the legacy channel identifiers.

This is not exclusive. We have code that works with channel layouts
right now, and it is important that the extensions work gracefully.

For example, if I add a filter to just extract the pair of channels
related to the strings section, we want that its output is properly
marked as left and right.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Hendrik Leppkes]

On Thu, Dec 9, 2021 at 3:57 PM Nicolas George  wrote:
>
> Hendrik Leppkes (12021-12-09):
> > What kind of sense does a frame make that contains the same channel
> > twice?
>
> Imagine an orchestra recording:
>
> Orchestra, front left
> Orchestra, front center
> Orchestra, front right
> Orchestra, low freq
> Orchestra, rear left
> Orchestra, rear center
> Winds, left
> Winds, right
> Percussions, left
> Percussions, right
> Strings, left
> Strings, right
>
> We cannot have enums for winds, percussions and strings, but we should
> be able to label them properly left and right, and attach a string label
> to each.
>

It sounds like thats object audio, and should be driven separately,
and not by the legacy channel identifiers.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Lynne (12021-12-09):
> So I'm fine with your proposal to have 16-bit enum for the channel
> ID and a 16-bit opaque. Though I'd like the opaque to be an
> uint16_t instead of int opaque : 16.
> And 16-bits does sound like enough for many channels and quite a
> few flags, though the silent flag should be moved to 1 << 15 instead
> of 64, and any new flags could be added beneath so as to not conflict
> with channels.

I insist: a tiny field like that is not enough, let us make it a whole
string.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Lynne]

9 Dec 2021, 15:57 by an...@khirnov.net:

> Quoting Lynne (2021-12-09 15:42:42)
>
>> 9 Dec 2021, 15:24 by geo...@nsup.org:
>>
>> > Anton Khirnov (12021-12-09):
>> >
>> >> I see you repeating the same two arguments:
>> >> - it was implemented like this in the past and therefore must keep
>> >>  working exactly the same
>> >> - it might be useful under some vaguely specified conditions
>> >>
>> >> Neither of these strikes me as a good enough reason to make major
>> >> changes to the API design.
>> >>
>> >
>> > I will turn this argument back on you: you have designed your API so
>> > that it is too limited, it does not strike me as a good reason to make
>> > major changes in existing filters and devices that have given
>> > satisfaction to users for years.
>> >
>>
>> As a compromise, could we specify that while having multple
>> channels with the same ID in a single frame can happen and
>> can be generated by decoders, we would also specify that they
>> possibly won't be treated correctly by encoders and filters, and
>> could be outright dropped with a warning if unsupported.
>>
>
> That is pretty much already the case. I know there are files in the wild
> that have duplicated channels and the proposed API supports exporting
> this information.
>
> What I do _not_ want is treating such streams as first-class citizens
> that have to be fully supported by everything. They are a pathology and
> should be treated as such -- that is supported on input, but not output.
>
>>
>> I can see why having multiple channels with the
>> same ID can happen, an in fact, will, for custom user
>> layouts with more channels than there are IDs.
>> For example, an Opus stream containing a hundred or
>> so channels from multiple overlapping locations from a venue.
>> Each of those channels would have to have an ID of NONE,
>> because the codec mapping family doesn't carry such information
>> for such a configuration.
>>
>
> NONE is intended to be an invalid value, but we can add AV_CHAN_UNKNOWN
> with a high id for such a case. Or we can reserve a range of ids for
> application-specific usage.
>

I'm fine with this. I think an AV_CHAN_UNKNOWN ID is needed
pretty much anyway, and as for user IDs, maybe a single
AV_CHAN_CUSTOM or AV_CHAN_USER_DEFINED.
The user could then use the channel opaque field to store some
info, such as an index into their frame->opaque_ref data with
which they could store channel-specific offset.

So I'm fine with your proposal to have 16-bit enum for the channel
ID and a 16-bit opaque. Though I'd like the opaque to be an
uint16_t instead of int opaque : 16.
And 16-bits does sound like enough for many channels and quite a
few flags, though the silent flag should be moved to 1 << 15 instead
of 64, and any new flags could be added beneath so as to not conflict
with channels.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Anton Khirnov]

Quoting Lynne (2021-12-09 15:42:42)
> 9 Dec 2021, 15:24 by geo...@nsup.org:
> 
> > Anton Khirnov (12021-12-09):
> >
> >> I see you repeating the same two arguments:
> >> - it was implemented like this in the past and therefore must keep
> >>  working exactly the same
> >> - it might be useful under some vaguely specified conditions
> >>
> >> Neither of these strikes me as a good enough reason to make major
> >> changes to the API design.
> >>
> >
> > I will turn this argument back on you: you have designed your API so
> > that it is too limited, it does not strike me as a good reason to make
> > major changes in existing filters and devices that have given
> > satisfaction to users for years.
> >
> 
> As a compromise, could we specify that while having multple
> channels with the same ID in a single frame can happen and
> can be generated by decoders, we would also specify that they
> possibly won't be treated correctly by encoders and filters, and
> could be outright dropped with a warning if unsupported.

That is pretty much already the case. I know there are files in the wild
that have duplicated channels and the proposed API supports exporting
this information.

What I do _not_ want is treating such streams as first-class citizens
that have to be fully supported by everything. They are a pathology and
should be treated as such -- that is supported on input, but not output.

> 
> I can see why having multiple channels with the
> same ID can happen, an in fact, will, for custom user
> layouts with more channels than there are IDs.
> For example, an Opus stream containing a hundred or
> so channels from multiple overlapping locations from a venue.
> Each of those channels would have to have an ID of NONE,
> because the codec mapping family doesn't carry such information
> for such a configuration.

NONE is intended to be an invalid value, but we can add AV_CHAN_UNKNOWN
with a high id for such a case. Or we can reserve a range of ids for
application-specific usage.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Hendrik Leppkes (12021-12-09):
> What kind of sense does a frame make that contains the same channel
> twice?

Imagine an orchestra recording:

Orchestra, front left
Orchestra, front center
Orchestra, front right
Orchestra, low freq
Orchestra, rear left
Orchestra, rear center
Winds, left
Winds, right
Percussions, left
Percussions, right
Strings, left
Strings, right

We cannot have enums for winds, percussions and strings, but we should
be able to label them properly left and right, and attach a string label
to each.

> What am I ever supposed to do with that?

Apply remixing matrices on channels from different parts of the
orchestra.

Let a very smart codec detect correlations, and therefore solutions for
better compression, between one section and another.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Hendrik Leppkes]

On Thu, Dec 9, 2021 at 3:42 PM Lynne  wrote:
>
> 9 Dec 2021, 15:24 by geo...@nsup.org:
>
> > Anton Khirnov (12021-12-09):
> >
> >> I see you repeating the same two arguments:
> >> - it was implemented like this in the past and therefore must keep
> >>  working exactly the same
> >> - it might be useful under some vaguely specified conditions
> >>
> >> Neither of these strikes me as a good enough reason to make major
> >> changes to the API design.
> >>
> >
> > I will turn this argument back on you: you have designed your API so
> > that it is too limited, it does not strike me as a good reason to make
> > major changes in existing filters and devices that have given
> > satisfaction to users for years.
> >
>
> As a compromise, could we specify that while having multple
> channels with the same ID in a single frame can happen and
> can be generated by decoders, we would also specify that they
> possibly won't be treated correctly by encoders and filters, and
> could be outright dropped with a warning if unsupported.
>

Actually thats the worst part of it, and I would be happy to not have
to think about that, as an API user.
What kind of sense does a frame make that contains the same channel
twice? What am I ever supposed to do with that?

It sounds to me like some kind of theoretical design flaw is trying to
be solved at the wrong point, instead of clearly separating streams,
they are supposed to live together but somehow still be separate? That
just sounds like a hack to me. Two streams are two streams, not one
stream with somehow duplicated channels.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Lynne]

9 Dec 2021, 15:24 by geo...@nsup.org:

> Anton Khirnov (12021-12-09):
>
>> I see you repeating the same two arguments:
>> - it was implemented like this in the past and therefore must keep
>>  working exactly the same
>> - it might be useful under some vaguely specified conditions
>>
>> Neither of these strikes me as a good enough reason to make major
>> changes to the API design.
>>
>
> I will turn this argument back on you: you have designed your API so
> that it is too limited, it does not strike me as a good reason to make
> major changes in existing filters and devices that have given
> satisfaction to users for years.
>

As a compromise, could we specify that while having multple
channels with the same ID in a single frame can happen and
can be generated by decoders, we would also specify that they
possibly won't be treated correctly by encoders and filters, and
could be outright dropped with a warning if unsupported.

I can see why having multiple channels with the
same ID can happen, an in fact, will, for custom user
layouts with more channels than there are IDs.
For example, an Opus stream containing a hundred or
so channels from multiple overlapping locations from a venue.
Each of those channels would have to have an ID of NONE,
because the codec mapping family doesn't carry such information
for such a configuration.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to pointer conversion warning

[lance . lmwang]

From: Limin Wang 

---
 libavfilter/dnn/dnn_backend_common.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavfilter/dnn/dnn_backend_common.c 
b/libavfilter/dnn/dnn_backend_common.c
index 6a9c4cc..dd7bdf4 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -83,10 +83,10 @@ static void *async_thread_routine(void *args)
 void *request = async_module->args;
 
 if (async_module->start_inference(request) != DNN_SUCCESS) {
-return DNN_ASYNC_FAIL;
+return (void*)DNN_ASYNC_FAIL;
 }
 async_module->callback(request);
-return DNN_ASYNC_SUCCESS;
+return (void*)DNN_ASYNC_SUCCESS;
 }
 
 DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to pointer conversion warning

[lance . lmwang]

From: Limin Wang 

---
 libavfilter/dnn/dnn_backend_common.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavfilter/dnn/dnn_backend_common.c 
b/libavfilter/dnn/dnn_backend_common.c
index 6a9c4cc..dd7bdf4 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -83,10 +83,10 @@ static void *async_thread_routine(void *args)
 void *request = async_module->args;
 
 if (async_module->start_inference(request) != DNN_SUCCESS) {
-return DNN_ASYNC_FAIL;
+return (void*)DNN_ASYNC_FAIL;
 }
 async_module->callback(request);
-return DNN_ASYNC_SUCCESS;
+return(void*)DNN_ASYNC_SUCCESS;
 }
 
 DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Anton Khirnov (12021-12-09):
> I see you repeating the same two arguments:
> - it was implemented like this in the past and therefore must keep
>   working exactly the same
> - it might be useful under some vaguely specified conditions
> 
> Neither of these strikes me as a good enough reason to make major
> changes to the API design.

I will turn this argument back on you: you have designed your API so
that it is too limited, it does not strike me as a good reason to make
major changes in existing filters and devices that have given
satisfaction to users for years.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Anton Khirnov]

Quoting Nicolas George (2021-12-09 14:52:40)
> Anton Khirnov (12021-12-09):
> > I fail to see how that is an advantage. You can just as well create
> > multiple instances of those single-stream filters instead of adding
> > hacks into core APIs.
> 
> Please think a little further: multiple instances of the single-stream
> filters would not have access to all the channels.
> 
> > "possibly" is not a strong enough argument. I'd like to hear at least
> > one clearly-defined use case that cannot just as well be handled by
> > using multiple streams.
> 
> This was a discussion for when the device was implemented. Now, it works
> that way, the new API has to accommodate it.
> 
> Anyway, these are just two example. I am sure we could find other
> examples easily if we tried. It would be pretty stupid of us to add a
> new API that is barely better than the current one and that we know is
> too limited for likely use cases.

I see you repeating the same two arguments:
- it was implemented like this in the past and therefore must keep
  working exactly the same
- it might be useful under some vaguely specified conditions

Neither of these strikes me as a good enough reason to make major
changes to the API design. So again - can you describe a clearly-defined
use case that cannot just as well be handled by using multiple streams?
Empasis on "clearly defined", so not "I am sure we can find examples".
I would like to hear some of those examples. So far you have not
provided any.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of async_thread_routine

[lance . lmwang]

On Thu, Dec 09, 2021 at 03:00:06PM +0100, Michael Niedermayer wrote:
> On Thu, Dec 09, 2021 at 09:19:41AM +0800, lance.lmw...@gmail.com wrote:
> > From: Limin Wang 
> > 
> > Signed-off-by: Limin Wang 
> > ---
> >  libavfilter/dnn/dnn_backend_common.c | 7 +--
> >  1 file changed, 5 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavfilter/dnn/dnn_backend_common.c 
> > b/libavfilter/dnn/dnn_backend_common.c
> > index 6a9c4cc..8c020e5 100644
> > --- a/libavfilter/dnn/dnn_backend_common.c
> > +++ b/libavfilter/dnn/dnn_backend_common.c
> > @@ -83,10 +83,13 @@ static void *async_thread_routine(void *args)
> >  void *request = async_module->args;
> >  
> >  if (async_module->start_inference(request) != DNN_SUCCESS) {
> > -return DNN_ASYNC_FAIL;
> > +pthread_exit((void*)DNN_ASYNC_FAIL);
> > +return NULL;
> >  }
> >  async_module->callback(request);
> > -return DNN_ASYNC_SUCCESS;
> > +
> > +pthread_exit((void*)DNN_ASYNC_SUCCESS);
> > +return NULL;
> >  }
> 
> seems to break build on mingw64
> 
> 

It's strange, for I saw pthread_exit() is used by vf_libvmaf.c. Anyway,
I realized it should be OK to add (void*) before the error code. 

> src/libavfilter/dnn/dnn_backend_common.c: In function ‘async_thread_routine’:
> src/libavfilter/dnn/dnn_backend_common.c:86:9: error: implicit declaration of 
> function ‘pthread_exit’; did you mean ‘pthread_join’? 
> [-Werror=implicit-function-declaration]
>  pthread_exit((void*)DNN_ASYNC_FAIL);
>  ^~~~
>  pthread_join
> src/libavfilter/dnn/dnn_backend_common.c: In function 
> ‘ff_dnn_async_module_cleanup’:
> src/libavfilter/dnn/dnn_backend_common.c:97:11: warning: unused variable 
> ‘status’ [-Wunused-variable]
>  void *status = 0;
>^~
> src/libavfilter/dnn/dnn_backend_common.c: In function 
> ‘ff_dnn_start_inference_async’:
> src/libavfilter/dnn/dnn_backend_common.c:117:11: warning: unused variable 
> ‘status’ [-Wunused-variable]
>  void *status = 0;
>^~
> src/libavfilter/dnn/dnn_backend_common.c:116:9: warning: unused variable 
> ‘ret’ [-Wunused-variable]
>  int ret;
>  ^~~
> At top level:
> src/libavfilter/dnn/dnn_backend_common.c:80:14: warning: 
> ‘async_thread_routine’ defined but not used [-Wunused-function]
>  static void *async_thread_routine(void *args)
>   ^~~~
> 
> [...]
> -- 
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> The greatest way to live with honor in this world is to be what we pretend
> to be. -- Socrates



> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of async_thread_routine

[Michael Niedermayer]

On Thu, Dec 09, 2021 at 09:19:41AM +0800, lance.lmw...@gmail.com wrote:
> From: Limin Wang 
> 
> Signed-off-by: Limin Wang 
> ---
>  libavfilter/dnn/dnn_backend_common.c | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/libavfilter/dnn/dnn_backend_common.c 
> b/libavfilter/dnn/dnn_backend_common.c
> index 6a9c4cc..8c020e5 100644
> --- a/libavfilter/dnn/dnn_backend_common.c
> +++ b/libavfilter/dnn/dnn_backend_common.c
> @@ -83,10 +83,13 @@ static void *async_thread_routine(void *args)
>  void *request = async_module->args;
>  
>  if (async_module->start_inference(request) != DNN_SUCCESS) {
> -return DNN_ASYNC_FAIL;
> +pthread_exit((void*)DNN_ASYNC_FAIL);
> +return NULL;
>  }
>  async_module->callback(request);
> -return DNN_ASYNC_SUCCESS;
> +
> +pthread_exit((void*)DNN_ASYNC_SUCCESS);
> +return NULL;
>  }

seems to break build on mingw64


src/libavfilter/dnn/dnn_backend_common.c: In function ‘async_thread_routine’:
src/libavfilter/dnn/dnn_backend_common.c:86:9: error: implicit declaration of 
function ‘pthread_exit’; did you mean ‘pthread_join’? 
[-Werror=implicit-function-declaration]
 pthread_exit((void*)DNN_ASYNC_FAIL);
 ^~~~
 pthread_join
src/libavfilter/dnn/dnn_backend_common.c: In function 
‘ff_dnn_async_module_cleanup’:
src/libavfilter/dnn/dnn_backend_common.c:97:11: warning: unused variable 
‘status’ [-Wunused-variable]
 void *status = 0;
   ^~
src/libavfilter/dnn/dnn_backend_common.c: In function 
‘ff_dnn_start_inference_async’:
src/libavfilter/dnn/dnn_backend_common.c:117:11: warning: unused variable 
‘status’ [-Wunused-variable]
 void *status = 0;
   ^~
src/libavfilter/dnn/dnn_backend_common.c:116:9: warning: unused variable ‘ret’ 
[-Wunused-variable]
 int ret;
 ^~~
At top level:
src/libavfilter/dnn/dnn_backend_common.c:80:14: warning: ‘async_thread_routine’ 
defined but not used [-Wunused-function]
 static void *async_thread_routine(void *args)
  ^~~~

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2] libswresample/swresamplec: Err num(negative-size) was used as a function parameter

[Yy]

> 2021年12月9日 下午8:10，Michael Niedermayer  写道：
> 
> On Wed, Dec 08, 2021 at 11:17:13AM +0800, Yu Yang wrote:
>> If cannot allocate memory, ERROR(ENOMEM) '-12' as a parameter will be 
>> constantly being returned.
>> When run resample() firstly, negative size param would cause buffer-overflow 
>> and SEGV in swri_rematrix(). 
>> When run swri_rematrix() firstly, resample() would not cause error but Err 
>> num as a wrong parameter passing.
>> Err num should be returned immediately. And remove assert to ensure the 
>> return of the error code.
>> 
>> coredump info:
>>#0 0x499517 in posix_memalign (/home/r1/ffmpeg/ffmpeg_4.4.1+0x499517)
>>#1 0x6c1f0b4 in av_malloc 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:86:9
>>#2 0x6c208fe in av_mallocz 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:239:17
>>#3 0x6c207ad in av_mallocz_array 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:195:12
>>#4 0x654b2e5 in swri_realloc_audio 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:418:14
>>#5 0x654f9a1 in swr_convert_internal 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:601:17
>>#6 0x654d2c0 in swr_convert 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:766:19
>>#7 0x186cf56 in flush_frame 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/af_aresample.c:251:13
>>#8 0x186a454 in request_frame 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/af_aresample.c:288:20
>>#9 0x787d9c in ff_request_frame_to_filter 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:459:15
>>#10 0x7877f1 in forward_status_change 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1257:19
>>#11 0x77ed7e in ff_filter_activate_default 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1288:20
>>#12 0x77e4e1 in ff_filter_activate 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1441:11
>>#13 0x793b3f in ff_filter_graph_run_once 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfiltergraph.c:1403:12
>>#14 0x7a7bee in get_frame_internal 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/buffersink.c:131:19
>>#15 0x7a7287 in av_buffersink_get_frame_flags 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/buffersink.c:142:12
>>#16 0x792888 in avfilter_graph_request_oldest 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfiltergraph.c:1356:17
>>#17 0x5d07df in transcode_from_filter 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4639:11
>>#18 0x59e557 in transcode_step 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4729:20
>>#19 0x593970 in transcode 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4805:15
>>#20 0x58f7a4 in main 
>> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:5010:9
>>#21 0x7f6fd2dee0b2 in __libc_start_main 
>> /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
>> 
>> SUMMARY: AddressSanitizer: negative-size-param 
>> (/home/r1/ffmpeg/ffmpeg_4.4.1+0x497e67) in __asan_memcpy
>> 
>> Reported-by: TOTE Robot 
>> Signed-off-by: Yu Yang 
>> ---
>> libswresample/swresample.c | 8 +---
>> 1 file changed, 5 insertions(+), 3 deletions(-)
> 
> will apply
> 
> thx
Thank you.
This is the first time my patch has been adopted. 
Have a nice day! :)
> 
> [...]
> -- 
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> It is a danger to trust the dream we wish for rather than
> the science we have, -- Dr. Kenneth Brown
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Anton Khirnov (12021-12-09):
> I fail to see how that is an advantage. You can just as well create
> multiple instances of those single-stream filters instead of adding
> hacks into core APIs.

Please think a little further: multiple instances of the single-stream
filters would not have access to all the channels.

> "possibly" is not a strong enough argument. I'd like to hear at least
> one clearly-defined use case that cannot just as well be handled by
> using multiple streams.

This was a discussion for when the device was implemented. Now, it works
that way, the new API has to accommodate it.

Anyway, these are just two example. I am sure we could find other
examples easily if we tried. It would be pretty stupid of us to add a
new API that is barely better than the current one and that we know is
too limited for likely use cases.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Anton Khirnov]

Quoting Nicolas George (2021-12-09 11:31:54)
> Anton Khirnov (12021-12-09):
> > I disagree. Technical limitations that were overcome 10 years ago should
> > not guide new API design.
> 
> In the case of amerge, it was not a technical limitation, merging
> several streams into one so that they can be handled by single-stream
> filters is 100% part of the design.

I fail to see how that is an advantage. You can just as well create
multiple instances of those single-stream filters instead of adding
hacks into core APIs.

> I suspect devices that capture several independent channels are
> designed that way intentionally too, possibly to reduce the risk of
> desynchronization.

"possibly" is not a strong enough argument. I'd like to hear at least
one clearly-defined use case that cannot just as well be handled by
using multiple streams.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Nicolas George]

Lynne (12021-12-09):
> "reserved" rather than "opaque"?
> 16 bits for both? Could you keep the id as a plain (32-bit, on
> a sane compiler) enum, since we may need to add flags, one
> never knows, and make the opaque field a plain uint32_t?
> 64-bits per channel overall isn't too bad.

You wanted a real string, please do not beg for 16 measly bits! I
strongly support your demand of a real string.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Anton Khirnov]

Quoting Lynne (2021-12-09 12:21:14)
> 9 Dec 2021, 11:14 by an...@khirnov.net:
> 
> > Quoting Lynne (2021-12-08 13:53:52)
> >
> >> That's not a goal, it's anti-goal, and a cause for hysterical raisin
> >> picking in the future.
> >> Having an instantly debuggable structure rather than one that
> >>
> >
> > Instantly debuggable structure?
> >
> > Again: very very little code needs to store actual AVChannels. In the
> > submitted tree this is just af_join. So the gain from adding an offset
> > would simplify very little code, while complicating all the callers
> > dealing with masks, of which there are quite many.
> >
> 
> It would make debugging much less hazardous,

I fail to see how, since almost no code stores AVChannel directly.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Lynne]

9 Dec 2021, 14:12 by an...@khirnov.net:

> Quoting Lynne (2021-12-08 13:57:45)
>
>> 8 Dec 2021, 13:16 by an...@khirnov.net:
>>
>> > Quoting Lynne (2021-12-08 10:02:34)
>> >
>> >> 8 Dec 2021, 02:06 by jamr...@gmail.com:
>> >>
>> >> > From: Anton Khirnov 
>> >> >
>> >> > The new API is more extensible and allows for custom layouts.
>> >> > More accurate information is exported, eg for decoders that do not
>> >> > set a channel layout, lavc will not make one up for them.
>> >> >
>> >> > Deprecate the old API working with just uint64_t bitmasks.
>> >> >
>> >> > Expanded and completed by Vittorio Giovara 
>> >> > and James Almer .
>> >> > Signed-off-by: Vittorio Giovara 
>> >> > Signed-off-by: James Almer 
>> >> > ---
>> >> >  libavutil/channel_layout.c | 498 ++--
>> >> >  libavutil/channel_layout.h | 510 ++---
>> >> >  libavutil/version.h|   1 +
>> >> >  3 files changed, 906 insertions(+), 103 deletions(-)
>> >> > 
>> >> >  /**
>> >> >  * @}
>> >> > @@ -128,6 +199,157 @@ enum AVMatrixEncoding {
>> >> >  AV_MATRIX_ENCODING_NB
>> >> >  };
>> >> > 
>> >> > +/**
>> >> > + * @}
>> >> > + */
>> >> > +
>> >> > +/**
>> >> > + * An AVChannelCustom defines a single channel within a custom order 
>> >> > layout
>> >> > + *
>> >> > + * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is a part 
>> >> > of the
>> >> > + * public ABI.
>> >> > + *
>> >> > + * No new fields may be added to it without a major version bump.
>> >> > + */
>> >> > +typedef struct AVChannelCustom {
>> >> > +enum AVChannel id;
>> >> > +} AVChannelCustom;
>> >> >
>> >>
>> >> Consider adding a 25-byte or so of a description field string here that
>> >> would also be copied if the frame/layout is copied?
>> >> That way, users can assign a description label to each channel,
>> >> for example labeling each channel in a 255 channel custom layout
>> >> Opus stream used in production at a venue somewhere.
>> >> Also, consider additionally reserving 16-bytes here, just in case.
>> >>
>> >
>> > That would enlarge the layout size by a significant amount. Keep in mind
>> > that this is all per frame. And for something that very few people would
>> > want to use.
>> >
>> > These descriptors, if anyone really needs them, can live in side data.
>> >
>>
>> That's fine with me.
>> Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
>> or an int too, or even a single-byte uint8_t.
>>
>
> like this?
>
> diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
> index 7b77a74b61..7b2ba12532 100644
> --- a/libavutil/channel_layout.h
> +++ b/libavutil/channel_layout.h
> @@ -252,7 +252,8 @@ enum AVMatrixEncoding {
>  * No new fields may be added to it without a major version bump.
>  */
>  typedef struct AVChannelCustom {
> -enum AVChannel id;
> +enum AVChannel id:16;
> +int  reserved:16;
>  } AVChannelCustom; 
>

"reserved" rather than "opaque"?
16 bits for both? Could you keep the id as a plain (32-bit, on
a sane compiler) enum, since we may need to add flags, one
never knows, and make the opaque field a plain uint32_t?
64-bits per channel overall isn't too bad.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of async_thread_routine

[lance . lmwang]

On Thu, Dec 09, 2021 at 11:55:14AM +, Guo, Yejun wrote:
> 
> 
> -Original Message-
> From: ffmpeg-devel  On Behalf Of 
> lance.lmw...@gmail.com
> Sent: 2021年12月9日 9:20
> To: ffmpeg-devel@ffmpeg.org
> Cc: Limin Wang 
> Subject: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of 
> async_thread_routine
> 
> From: Limin Wang 
> 
> Signed-off-by: Limin Wang 
> ---
>  libavfilter/dnn/dnn_backend_common.c | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/libavfilter/dnn/dnn_backend_common.c 
> b/libavfilter/dnn/dnn_backend_common.c
> index 6a9c4cc..8c020e5 100644
> --- a/libavfilter/dnn/dnn_backend_common.c
> +++ b/libavfilter/dnn/dnn_backend_common.c
> @@ -83,10 +83,13 @@ static void *async_thread_routine(void *args)
>  void *request = async_module->args;
>  
>  if (async_module->start_inference(request) != DNN_SUCCESS) {
> -return DNN_ASYNC_FAIL;
> +pthread_exit((void*)DNN_ASYNC_FAIL);
> +return NULL;
> 
> Could you share the reason for this change?
> From man pthread_exit:
> Performing a return from the start function of any thread other than the main 
>  thread  results  in  an  implicit  call  to
> pthread_exit(), using the function's return value as the thread's exit status.

I just realized it should be OK to add (void*) to the error code directly.

> 
>  }
>  async_module->callback(request);
> -return DNN_ASYNC_SUCCESS;
> +
> +pthread_exit((void*)DNN_ASYNC_SUCCESS);
> +return NULL;
>  }
>  
>  DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
> -- 
> 1.8.3.1
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Anton Khirnov]

Quoting Lynne (2021-12-08 13:57:45)
> 8 Dec 2021, 13:16 by an...@khirnov.net:
> 
> > Quoting Lynne (2021-12-08 10:02:34)
> >
> >> 8 Dec 2021, 02:06 by jamr...@gmail.com:
> >>
> >> > From: Anton Khirnov 
> >> >
> >> > The new API is more extensible and allows for custom layouts.
> >> > More accurate information is exported, eg for decoders that do not
> >> > set a channel layout, lavc will not make one up for them.
> >> >
> >> > Deprecate the old API working with just uint64_t bitmasks.
> >> >
> >> > Expanded and completed by Vittorio Giovara 
> >> > and James Almer .
> >> > Signed-off-by: Vittorio Giovara 
> >> > Signed-off-by: James Almer 
> >> > ---
> >> >  libavutil/channel_layout.c | 498 ++--
> >> >  libavutil/channel_layout.h | 510 ++---
> >> >  libavutil/version.h|   1 +
> >> >  3 files changed, 906 insertions(+), 103 deletions(-)
> >> > 
> >> >  /**
> >> >  * @}
> >> > @@ -128,6 +199,157 @@ enum AVMatrixEncoding {
> >> >  AV_MATRIX_ENCODING_NB
> >> >  };
> >> > 
> >> > +/**
> >> > + * @}
> >> > + */
> >> > +
> >> > +/**
> >> > + * An AVChannelCustom defines a single channel within a custom order 
> >> > layout
> >> > + *
> >> > + * Unlike most structures in FFmpeg, sizeof(AVChannelCustom) is a part 
> >> > of the
> >> > + * public ABI.
> >> > + *
> >> > + * No new fields may be added to it without a major version bump.
> >> > + */
> >> > +typedef struct AVChannelCustom {
> >> > +enum AVChannel id;
> >> > +} AVChannelCustom;
> >> >
> >>
> >> Consider adding a 25-byte or so of a description field string here that
> >> would also be copied if the frame/layout is copied?
> >> That way, users can assign a description label to each channel,
> >> for example labeling each channel in a 255 channel custom layout
> >> Opus stream used in production at a venue somewhere.
> >> Also, consider additionally reserving 16-bytes here, just in case.
> >>
> >
> > That would enlarge the layout size by a significant amount. Keep in mind
> > that this is all per frame. And for something that very few people would
> > want to use.
> >
> > These descriptors, if anyone really needs them, can live in side data.
> >
> 
> That's fine with me.
> Can we at least have an opaque uint64_t? I'd accept a uintptr_t,
> or an int too, or even a single-byte uint8_t.

like this?

diff --git a/libavutil/channel_layout.h b/libavutil/channel_layout.h
index 7b77a74b61..7b2ba12532 100644
--- a/libavutil/channel_layout.h
+++ b/libavutil/channel_layout.h
@@ -252,7 +252,8 @@ enum AVMatrixEncoding {
  * No new fields may be added to it without a major version bump.
  */
 typedef struct AVChannelCustom {
-enum AVChannel id;
+enum AVChannel id:16;
+int  reserved:16;
 } AVChannelCustom;
 
 /**

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 5/5] avcodec/mmaldec: Deduplicate AVClasses

[Andreas Rheinhardt]

Possible now that the child_class_next API is gone.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/mmaldec.c | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index 0c7df0a0bc..f8a7319cd6 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -820,16 +820,14 @@ static const AVOption options[]={
 {NULL}
 };
 
-#define FFMMAL_DEC_CLASS(NAME) \
-static const AVClass ffmmal_##NAME##_dec_class = { \
-.class_name = "mmal_" #NAME "_dec", \
-.item_name  = av_default_item_name, \
-.option = options, \
-.version= LIBAVUTIL_VERSION_INT, \
-};
+static const AVClass ffmmal_dec_class = {
+.class_name = "mmal_dec",
+.item_name  = av_default_item_name,
+.option = options,
+.version= LIBAVUTIL_VERSION_INT,
+};
 
 #define FFMMAL_DEC(NAME, ID) \
-FFMMAL_DEC_CLASS(NAME) \
 const AVCodec ff_##NAME##_mmal_decoder = { \
 .name   = #NAME "_mmal", \
 .long_name  = NULL_IF_CONFIG_SMALL(#NAME " (mmal)"), \
@@ -840,7 +838,7 @@ static const AVOption options[]={
 .close  = ffmmal_close_decoder, \
 .receive_frame  = ffmmal_receive_frame, \
 .flush  = ffmmal_flush, \
-.priv_class = &ffmmal_##NAME##_dec_class, \
+.priv_class = &ffmmal_dec_class, \
 .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, \
 .caps_internal  = FF_CODEC_CAP_SETS_PKT_DTS, \
 .pix_fmts   = (const enum AVPixelFormat[]) { AV_PIX_FMT_MMAL, \
-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 4/5] avcodec/mmaldec: Avoid creating unnecessary reference, simplify code

[Andreas Rheinhardt]

ffmal_add_packet() basically duplicated the logic in
av_packet_make_refcounted() with the added twist that it always
created a reference even if one is already available.
This commit stops doing this.

Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/mmaldec.c | 23 ++-
 1 file changed, 6 insertions(+), 17 deletions(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index 3e11d487ab..0c7df0a0bc 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -485,29 +485,19 @@ static int ffmmal_add_packet(AVCodecContext *avctx, 
AVPacket *avpkt,
  int is_extradata)
 {
 MMALDecodeContext *ctx = avctx->priv_data;
-AVBufferRef *buf = NULL;
+const AVBufferRef *buf = NULL;
 int size = 0;
 uint8_t *data = (uint8_t *)"";
 uint8_t *start;
 int ret = 0;
 
 if (avpkt->size) {
-if (avpkt->buf) {
-buf = av_buffer_ref(avpkt->buf);
-size = avpkt->size;
-data = avpkt->data;
-} else {
-buf = av_buffer_alloc(avpkt->size);
-if (buf) {
-memcpy(buf->data, avpkt->data, avpkt->size);
-size = buf->size;
-data = buf->data;
-}
-}
-if (!buf) {
-ret = AVERROR(ENOMEM);
+ret = av_packet_make_refcounted(avpkt);
+if (ret < 0)
 goto done;
-}
+buf  = avpkt->buf;
+data = avpkt->data;
+size = avpkt->size;
 if (!is_extradata)
 ctx->packets_sent++;
 } else {
@@ -573,7 +563,6 @@ static int ffmmal_add_packet(AVCodecContext *avctx, 
AVPacket *avpkt,
 } while (size);
 
 done:
-av_buffer_unref(&buf);
 av_packet_unref(avpkt);
 return ret;
 }
-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 3/5] avcodec/mmaldec: fix pointer type warning

[Andreas Rheinhardt]

From: Ho Ming Shun 

Signed-off-by: Ho Ming Shun 
---
 libavcodec/mmaldec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index 68ecc68f17..3e11d487ab 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -656,7 +656,7 @@ static int ffmal_copy_frame(AVCodecContext *avctx,  AVFrame 
*frame,
 av_image_fill_arrays(src, linesize,
  buffer->data + buffer->type->video.offset[0],
  avctx->pix_fmt, w, h, 1);
-av_image_copy(frame->data, frame->linesize, src, linesize,
+av_image_copy(frame->data, frame->linesize, (const uint8_t **)src, 
linesize,
   avctx->pix_fmt, avctx->width, avctx->height);
 }
 
-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 2/5] avcodec/mmaldec: re-use AVPacket for extra_data

[Andreas Rheinhardt]

From: Ho Ming Shun 

extra_data and normal packets (from ff_decode_get_packet) processing do
not overlap, thus we can re-use the spare AVPacket to send to
ffmmal_add_packet.

Furthermore, this removes allocation of AVPacket on the stack and stops
using deprecated av_init_packet.

Signed-off-by: Ho Ming Shun 
---
 libavcodec/mmaldec.c | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index d336f10350..68ecc68f17 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -782,12 +782,10 @@ static int ffmmal_receive_frame(AVCodecContext *avctx, 
AVFrame *frame)
 int got_frame = 0;
 
 if (avctx->extradata_size && !ctx->extradata_sent) {
-AVPacket pkt = {0};
-av_init_packet(&pkt);
-pkt.data = avctx->extradata;
-pkt.size = avctx->extradata_size;
+avpkt->data = avctx->extradata;
+avpkt->size = avctx->extradata_size;
 ctx->extradata_sent = 1;
-if ((ret = ffmmal_add_packet(avctx, &pkt, 1)) < 0)
+if ((ret = ffmmal_add_packet(avctx, avpkt, 1)) < 0)
 return ret;
 }
 
-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 1/5] avcodec/mmaldec: use decoupled dataflow

[Andreas Rheinhardt]

From: Ho Ming Shun 

MMAL is an fundamentally an asynchronous decoder, which was a bad fit
for the legacy dataflow API. Often multiple packets are enqueued before
a flood of frames are returned from MMAL.

The previous lockstep dataflow meant that any delay in returning packets
from the VPU would cause ctx->queue_decoded_frames to grow with no way
of draining the queue.

Testing this with mpv streaming from a live RTSP source visibly reduced
latency introduced by frames waiting in queue_decoded_frames from
roughly 2s to 0.

Signed-off-by: Ho Ming Shun 
Signed-off-by: Andreas Rheinhardt 
---
 libavcodec/mmaldec.c | 32 +---
 1 file changed, 25 insertions(+), 7 deletions(-)

diff --git a/libavcodec/mmaldec.c b/libavcodec/mmaldec.c
index 8c7d749742..d336f10350 100644
--- a/libavcodec/mmaldec.c
+++ b/libavcodec/mmaldec.c
@@ -83,6 +83,8 @@ typedef struct MMALDecodeContext {
 // libavcodec API can't return new frames, and we have a logical deadlock.
 // This is avoided by queuing such buffers here.
 FFBufferEntry *waiting_buffers, *waiting_buffers_tail;
+/* Packet used to hold received packets temporarily; not owned by us. */
+AVPacket *pkt;
 
 int64_t packets_sent;
 atomic_int packets_buffered;
@@ -355,6 +357,8 @@ static av_cold int ffmmal_init_decoder(AVCodecContext 
*avctx)
 MMAL_COMPONENT_T *decoder;
 int ret = 0;
 
+ctx->pkt = avctx->internal->in_pkt;
+
 bcm_host_init();
 
 if (mmal_vc_init()) {
@@ -570,6 +574,7 @@ static int ffmmal_add_packet(AVCodecContext *avctx, 
AVPacket *avpkt,
 
 done:
 av_buffer_unref(&buf);
+av_packet_unref(avpkt);
 return ret;
 }
 
@@ -655,6 +660,12 @@ static int ffmal_copy_frame(AVCodecContext *avctx,  
AVFrame *frame,
   avctx->pix_fmt, avctx->width, avctx->height);
 }
 
+frame->sample_aspect_ratio = avctx->sample_aspect_ratio;
+frame->width = avctx->width;
+frame->width = avctx->width;
+frame->height = avctx->height;
+frame->format = avctx->pix_fmt;
+
 frame->pts = buffer->pts == MMAL_TIME_UNKNOWN ? AV_NOPTS_VALUE : 
buffer->pts;
 frame->pkt_dts = AV_NOPTS_VALUE;
 
@@ -763,12 +774,12 @@ done:
 return ret;
 }
 
-static int ffmmal_decode(AVCodecContext *avctx, void *data, int *got_frame,
- AVPacket *avpkt)
+static int ffmmal_receive_frame(AVCodecContext *avctx, AVFrame *frame)
 {
 MMALDecodeContext *ctx = avctx->priv_data;
-AVFrame *frame = data;
+AVPacket *const avpkt = ctx->pkt;
 int ret = 0;
+int got_frame = 0;
 
 if (avctx->extradata_size && !ctx->extradata_sent) {
 AVPacket pkt = {0};
@@ -780,7 +791,11 @@ static int ffmmal_decode(AVCodecContext *avctx, void 
*data, int *got_frame,
 return ret;
 }
 
-if ((ret = ffmmal_add_packet(avctx, avpkt, 0)) < 0)
+ret = ff_decode_get_packet(avctx, avpkt);
+if(ret == 0) {
+if ((ret = ffmmal_add_packet(avctx, avpkt, 0)) < 0)
+return ret;
+} else if(ret < 0 && !(ret == AVERROR(EAGAIN)))
 return ret;
 
 if ((ret = ffmmal_fill_input_port(avctx)) < 0)
@@ -789,7 +804,7 @@ static int ffmmal_decode(AVCodecContext *avctx, void *data, 
int *got_frame,
 if ((ret = ffmmal_fill_output_port(avctx)) < 0)
 return ret;
 
-if ((ret = ffmmal_read_frame(avctx, frame, got_frame)) < 0)
+if ((ret = ffmmal_read_frame(avctx, frame, &got_frame)) < 0)
 return ret;
 
 // ffmmal_read_frame() can block for a while. Since the decoder is
@@ -801,7 +816,10 @@ static int ffmmal_decode(AVCodecContext *avctx, void 
*data, int *got_frame,
 if ((ret = ffmmal_fill_input_port(avctx)) < 0)
 return ret;
 
-return ret;
+if(!got_frame && ret == 0)
+return AVERROR(EAGAIN);
+else
+return ret;
 }
 
 static const AVCodecHWConfigInternal *const mmal_hw_configs[] = {
@@ -833,7 +851,7 @@ static const AVOption options[]={
 .priv_data_size = sizeof(MMALDecodeContext), \
 .init   = ffmmal_init_decoder, \
 .close  = ffmmal_close_decoder, \
-.decode = ffmmal_decode, \
+.receive_frame  = ffmmal_receive_frame, \
 .flush  = ffmmal_flush, \
 .priv_class = &ffmmal_##NAME##_dec_class, \
 .capabilities   = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_HARDWARE, \
-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v3 0/5] Switch mmaldec to receive_frame API

[Andreas Rheinhardt]

This is an updated and extended version of [1] by Ho Ming Shun
together with some additions by me. Notice that I am unable
to test these patches (not even compilation), so I hope that
others (Ho Ming Shun) do so.

[1]: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285992.html

Andreas Rheinhardt (2):
  avcodec/mmaldec: Avoid creating unnecessary reference, simplify code
  avcodec/mmaldec: Deduplicate AVClasses

Ho Ming Shun (3):
  avcodec/mmaldec: use decoupled dataflow
  avcodec/mmaldec: re-use AVPacket for extra_data
  avcodec/mmaldec: fix pointer type warning

 libavcodec/mmaldec.c | 81 +++-
 1 file changed, 42 insertions(+), 39 deletions(-)

-- 
2.32.0

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 5/7] avformat/mxfdec: Check for duplicate mxf_read_index_entry_array()

[Michael Niedermayer]

On Wed, Dec 08, 2021 at 12:41:43AM +0100, Tomas Härdin wrote:
> sön 2021-12-05 klockan 22:19 +0100 skrev Michael Niedermayer:
> > Fixes: memleak
> > Fixes: 41596/clusterfuzz-testcase-minimized-ffmpeg_dem_MXF_fuzzer-
> > 6439060204290048
> > 
> > Found-by: continuous fuzzing process
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavformat/mxfdec.c | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/libavformat/mxfdec.c b/libavformat/mxfdec.c
> > index c231c944c01..1d501982793 100644
> > --- a/libavformat/mxfdec.c
> > +++ b/libavformat/mxfdec.c
> > @@ -,6 +,9 @@ static int
> > mxf_read_index_entry_array(AVIOContext *pb, MXFIndexTableSegment *seg
> >  {
> >  int i, length;
> >  
> > +    if (segment->temporal_offset_entries)
> > +    return AVERROR_INVALIDDATA;
> > +
> >  segment->nb_index_entries = avio_rb32(pb);
> >  
> >  length = avio_rb32(pb);
> 
> Should be OK. Not sure if the spec allows multiple IndexEntryArrays per
> index table, but this at least shouldn't break anything since it
> wouldn't have been working correctly before either way.

will apply

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you drop bombs on a foreign country and kill a hundred thousand
innocent people, expect your government to call the consequence
"unprovoked inhuman terrorist attacks" and use it to justify dropping
more bombs and killing more people. The technology changed, the idea is old.


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v22 22/23] avutil/ass_split: Add parsing of hard-space tags (\h)

[Soft Works]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of Soft Works
> Sent: Thursday, December 9, 2021 1:13 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH v22 22/23] avutil/ass_split: Add parsing of
> hard-space tags (\h)
> 
> The \h tag in ASS/SSA is indicating a non-breaking space. See
> https://github.com/Aegisub/aegisite/blob/master/source/docs/3.2/ASS_Tags.html
> .md
> 
> The ass_split implementation is used by almost all text subtitle
> encoders and it didn't handle this tag. Interestingly, several tests
> are testing for \h parsing and had incorrect reference data for those tests.
> 
> The \h tag is specific to ASS and doesn't have any meaning outside of ASS.
> Still, the reference data for ttmlenc, textenc and webvttenc were full of
> \h tags even though this tag doesn't have a meaning there.
> 
> Signed-off-by: softworkz 
> ---
>  libavutil/ass_split.c|  7 +++
>  libavutil/ass_split_internal.h   |  1 +
>  tests/ref/fate/mov-mp4-ttml-dfxp |  8 
>  tests/ref/fate/mov-mp4-ttml-stpp |  8 
>  tests/ref/fate/sub-textenc   | 10 +-
>  tests/ref/fate/sub-ttmlenc   |  8 
>  tests/ref/fate/sub-webvttenc | 10 +-
>  7 files changed, 30 insertions(+), 22 deletions(-)
> 
> diff --git a/libavutil/ass_split.c b/libavutil/ass_split.c
> index c5963351fc..30512dfc74 100644
> --- a/libavutil/ass_split.c
> +++ b/libavutil/ass_split.c
> @@ -484,6 +484,7 @@ int avpriv_ass_split_override_codes(const
> ASSCodesCallbacks *callbacks, void *pr
>  while (buf && *buf) {
>  if (text && callbacks->text &&
>  (sscanf(buf, "\\%1[nN]", new_line) == 1 ||
> + sscanf(buf, "\\%1[hH]", new_line) == 1 ||
>   !strncmp(buf, "{\\", 2))) {
>  callbacks->text(priv, text, text_len);
>  text = NULL;
> @@ -492,6 +493,12 @@ int avpriv_ass_split_override_codes(const
> ASSCodesCallbacks *callbacks, void *pr
>  if (callbacks->new_line)
>  callbacks->new_line(priv, new_line[0] == 'N');
>  buf += 2;
> +} else if (sscanf(buf, "\\%1[hH]", new_line) == 1) {
> +if (callbacks->hard_space)
> +callbacks->hard_space(priv);
> +else if (callbacks->text)
> +callbacks->text(priv, " ", 1);
> +buf += 2;
>  } else if (!strncmp(buf, "{\\", 2)) {
>  buf++;
>  while (*buf == '\\') {
> diff --git a/libavutil/ass_split_internal.h b/libavutil/ass_split_internal.h
> index 8e8e51115c..d6eaade4a4 100644
> --- a/libavutil/ass_split_internal.h
> +++ b/libavutil/ass_split_internal.h
> @@ -141,6 +141,7 @@ typedef struct {
>   * @{
>   */
>  void (*text)(void *priv, const char *text, int len);
> +void (*hard_space)(void *priv);
>  void (*new_line)(void *priv, int forced);
>  void (*style)(void *priv, char style, int close);
>  void (*color)(void *priv, unsigned int /* color */, unsigned int
> color_id);
> diff --git a/tests/ref/fate/mov-mp4-ttml-dfxp b/tests/ref/fate/mov-mp4-ttml-
> dfxp
> index e24b5d618b..e565ffa1f6 100644
> --- a/tests/ref/fate/mov-mp4-ttml-dfxp
> +++ b/tests/ref/fate/mov-mp4-ttml-dfxp
> @@ -1,9 +1,9 @@
> -2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/mov-mp4-ttml-dfxp.mp4
> -8519 tests/data/fate/mov-mp4-ttml-dfxp.mp4
> +658884e1b789e75c454b25bdf71283c9 *tests/data/fate/mov-mp4-ttml-dfxp.mp4
> +8486 tests/data/fate/mov-mp4-ttml-dfxp.mp4
>  #tb 0: 1/1000
>  #media_type 0: data
>  #codec_id 0: none
> -0,  0,  0,68500, 7866, 0x456c36b7
> +0,  0,  0,68500, 7833, 0x31b22193
>  {
>  "packets": [
>  {
> @@ -15,7 +15,7 @@
>  "dts_time": "0.00",
>  "duration": 68500,
>  "duration_time": "68.50",
> -"size": "7866",
> +"size": "7833",
>  "pos": "44",
>  "flags": "K_"
>  }
> diff --git a/tests/ref/fate/mov-mp4-ttml-stpp b/tests/ref/fate/mov-mp4-ttml-
> stpp
> index 77bd23b7bf..f25b5b2d28 100644
> --- a/tests/ref/fate/mov-mp4-ttml-stpp
> +++ b/tests/ref/fate/mov-mp4-ttml-stpp
> @@ -1,9 +1,9 @@
> -cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-ttml-stpp.mp4
> -8547 tests/data/fate/mov-mp4-ttml-stpp.mp4
> +c9570de0ccebc858b0c662a7e449582c *tests/data/fate/mov-mp4-ttml-stpp.mp4
> +8514 tests/data/fate/mov-mp4-ttml-stpp.mp4
>  #tb 0: 1/1000
>  #media_type 0: data
>  #codec_id 0: none
> -0,  0,  0,68500, 7866, 0x456c36b7
> +0,  0,  0,68500, 7833, 0x31b22193
>  {
>  "packets": [
>  {
> @@ -15,7 +15,7 @@ cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-
> ttml-stpp.mp4
>  "dts_time": "0.00",
>  "duration": 68500,
>  "duration_time": "68.50",
> -"size": "7866",
> +"size": "7833",
>  "pos": "44",
>  "flags": "K_"
>

Re: [FFmpeg-devel] [PATCH 2/4] avcodec/apedec: Use 64bit in 4/3 calculation in do_apply_filter()

[Michael Niedermayer]

On Mon, Dec 06, 2021 at 03:50:34PM +0100, Anton Khirnov wrote:
> Quoting Michael Niedermayer (2021-12-06 12:01:09)
> > On Mon, Dec 06, 2021 at 11:23:12AM +0100, Anton Khirnov wrote:
> > > Quoting Michael Niedermayer (2021-12-03 18:19:54)
> > > > Fixes: Integer overflow
> > > > Fixes: 
> > > > 40973/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6739312704618496
> > > > 
> > > > Found-by: continuous fuzzing process 
> > > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > > Signed-off-by: Michael Niedermayer 
> > > > ---
> > > >  libavcodec/apedec.c | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > > > index 9c723f29977..5c3261cf5cb 100644
> > > > --- a/libavcodec/apedec.c
> > > > +++ b/libavcodec/apedec.c
> > > > @@ -1337,7 +1337,7 @@ static void do_apply_filter(APEContext *ctx, int 
> > > > version, APEFilter *f,
> > > >  absres = FFABSU(res);
> > > >  if (absres)
> > > >  *f->adaptcoeffs = APESIGN(res) *
> > > > -  (8 << ((absres > f->avg * 3LL) + 
> > > > (absres > (f->avg + f->avg / 3;
> > > > +  (8 << ((absres > f->avg * 3LL) + 
> > > > (absres > ((int64_t)f->avg + f->avg / 3;
> > > >  /* equivalent to the following code
> > > >  if (absres <= f->avg * 4 / 3)
> > > >  *f->adaptcoeffs = APESIGN(res) * 8;
> > > > -- 
> > > > 2.17.1
> > > 
> > > Does this not assume a 32bit int?
> > 
> > hmm
> > you mean avg could overflow 64bit ?
> > iam not sure that can happen but
> > we could make avg int32 or uint32 ?
> 
> Seems it cannot, but it's not completely obvious. So changing it to
> uint32 might make sense, since it cannot be negative.

ok will change it to uint32

thx

[...]

-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

When the tyrant has disposed of foreign enemies by conquest or treaty, and
there is nothing more to fear from them, then he is always stirring up
some war or other, in order that the people may require a leader. -- Plato


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of async_thread_routine

[lance . lmwang]

On Thu, Dec 09, 2021 at 11:55:14AM +, Guo, Yejun wrote:
> 
> 
> -Original Message-
> From: ffmpeg-devel  On Behalf Of 
> lance.lmw...@gmail.com
> Sent: 2021年12月9日 9:20
> To: ffmpeg-devel@ffmpeg.org
> Cc: Limin Wang 
> Subject: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of 
> async_thread_routine
> 
> From: Limin Wang 
> 
> Signed-off-by: Limin Wang 
> ---
>  libavfilter/dnn/dnn_backend_common.c | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/libavfilter/dnn/dnn_backend_common.c 
> b/libavfilter/dnn/dnn_backend_common.c
> index 6a9c4cc..8c020e5 100644
> --- a/libavfilter/dnn/dnn_backend_common.c
> +++ b/libavfilter/dnn/dnn_backend_common.c
> @@ -83,10 +83,13 @@ static void *async_thread_routine(void *args)
>  void *request = async_module->args;
>  
>  if (async_module->start_inference(request) != DNN_SUCCESS) {
> -return DNN_ASYNC_FAIL;
> +pthread_exit((void*)DNN_ASYNC_FAIL);
> +return NULL;
> 
> Could you share the reason for this change?
> From man pthread_exit:
> Performing a return from the start function of any thread other than the main 
>  thread  results  in  an  implicit  call  to
> pthread_exit(), using the function's return value as the thread's exit status.
it's void * for the thread function, so I think it's right to return NULL 
instead of error code. Most of usage is 
pthread_exit(NULL). I want to keep the old code return, so use (void*) for 
that, I'm not sure it's good idea or not.

> 
>  }
>  async_module->callback(request);
> -return DNN_ASYNC_SUCCESS;
> +
> +pthread_exit((void*)DNN_ASYNC_SUCCESS);
> +return NULL;
>  }
>  
>  DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
> -- 
> 1.8.3.1
> 
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 4/4] avcodec/targa: Do not return images when there is no image in the tga

[Michael Niedermayer]

On Mon, Dec 06, 2021 at 10:58:31AM +0100, Anton Khirnov wrote:
> Quoting Michael Niedermayer (2021-12-03 15:04:47)
> > On Sat, Sep 18, 2021 at 06:27:15PM +0200, Michael Niedermayer wrote:
> > > On Fri, Sep 17, 2021 at 08:06:48PM +0200, Paul B Mahol wrote:
> > > > On Fri, Sep 17, 2021 at 8:00 PM Michael Niedermayer 
> > > > 
> > > > wrote:
> > > > 
> > > > > On Fri, Sep 17, 2021 at 07:35:32PM +0200, Paul B Mahol wrote:
> > > > > > Please do not apply, This actually changes output to nothing.
> > > > >
> > > > > Do you have such a TGA file without an image in it?
> > > > >
> > > > > Why would that be relevant, comply with TGA specifications please.
> > > > 
> > > > thx
> > > 
> > > It is relevant as it would allow to check how such a file is handled
> > > by various implementations
> > > The specification does not specify that. It just says 
> > > "No image Data included"
> > > and our decoder returns a black image, the specification doesnt say that
> > > means "black image" or i missed it when reading 
> > > after the patch the decoder returns nothing instead of a black image
> > > The other type this might reach is type 8, the specification i have doesnt
> > > say anything about type 8 except that its reserved
> > > 
> > > Truevision TGAa
> > > FILE FORMAT SPECIFICATION
> > > Version 2.0
> > > ...
> > > Image Type - Field 3 (1 byte):
> > > The TGA File Format can be used to store Pseudo-Color, True-Color and 
> > > Direct-Color images of various
> > > pixel depths. Truevision has currently defined seven image types:
> > > image   Description
> > > Type0   No image Data included
> > 
> > ping
> > what do you prefer for this issue ?
> > Id like to do something here not just ignore it even if the conclusion is
> > that this is "not a bug but intended behavior"
> 
> A brief look at other opensource decoders suggest they don't support
> type==0 at all. image-rs does, but it's not clear to be what its output
> is. I'd say not returning a frame is reasonable.

ok, will apply

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v22 02/23] avutil/frame: Prepare AVFrame for subtitle handling

[Andreas Rheinhardt]

Soft Works:
> Root commit for adding subtitle filtering capabilities.
> In detail:
> 
> - Add type (AVMediaType) field to AVFrame
>   Replaces previous way of distinction which was based on checking
>   width and height to determine whether a frame is audio or video
> - Add subtitle fields to AVFrame
> - Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
>   allocation logic. Cannot and must not be used interchangeably, hence
>   the new struct
> 
> Signed-off-by: softworkz 
> ---
>  libavutil/Makefile |   1 +
>  libavutil/frame.c  | 211 -
>  libavutil/frame.h  |  78 -
>  libavutil/subfmt.c |  50 +++
>  libavutil/subfmt.h |  48 +++
>  5 files changed, 364 insertions(+), 24 deletions(-)
>  create mode 100644 libavutil/subfmt.c
> 
> diff --git a/libavutil/subfmt.c b/libavutil/subfmt.c
> new file mode 100644
> index 00..3998a6d486
> --- /dev/null
> +++ b/libavutil/subfmt.c
> @@ -0,0 +1,50 @@
> +/*
> + * Copyright (c) 2021 softworkz
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +#include 
> +#include "common.h"
> +#include "subfmt.h"
> +
> +typedef struct SubtitleFmtInfo {
> +enum AVSubtitleType fmt;
> +const char* name;
> +} SubtitleFmtInfo;

This struct is unused now.

> +
> +static const char sub_fmt_info[AV_SUBTITLE_FMT_NB][24] = {
> +[AV_SUBTITLE_FMT_UNKNOWN] = "Unknown subtitle format",
> +[AV_SUBTITLE_FMT_BITMAP]  = "Graphical subtitles",
> +[AV_SUBTITLE_FMT_TEXT]= "Text subtitles (plain)",
> +[AV_SUBTITLE_FMT_ASS] = "Text subtitles (ass)",
> +};
> +
> +const char *av_get_subtitle_fmt_name(enum AVSubtitleType sub_fmt)
> +{
> +if (sub_fmt < 0 || sub_fmt >= AV_SUBTITLE_FMT_NB)
> +return NULL;
> +return sub_fmt_info[sub_fmt];
> +}
> +
> +enum AVSubtitleType av_get_subtitle_fmt(const char *name)
> +{
> +for (int i = 0; i < AV_SUBTITLE_FMT_NB; i++)
> +if (!strcmp(sub_fmt_info[i], name))
> +return i;
> +return AV_SUBTITLE_FMT_NONE;
> +}
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v22 01/23] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values

[Soft Works]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of Andreas
> Rheinhardt
> Sent: Thursday, December 9, 2021 1:17 PM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v22 01/23] avcodec, avutil: Move enum
> AVSubtitleType to avutil, add new and deprecate old values
> 
> Soft Works:
> > Signed-off-by: softworkz 
> > ---
> >  libavcodec/avcodec.h | 19 +
> >  libavutil/Makefile   |  1 +
> >  libavutil/subfmt.h   | 66 
> >  libavutil/version.h  |  1 +
> >  4 files changed, 69 insertions(+), 18 deletions(-)
> >  create mode 100644 libavutil/subfmt.h
> >
> > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> > index 7ee8bc2b7c..b05c12e47e 100644
> > --- a/libavcodec/avcodec.h
> > +++ b/libavcodec/avcodec.h
> > @@ -35,6 +35,7 @@
> >  #include "libavutil/frame.h"
> >  #include "libavutil/log.h"
> >  #include "libavutil/pixfmt.h"
> > +#include "libavutil/subfmt.h"
> >  #include "libavutil/rational.h"
> >
> >  #include "codec.h"
> > @@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
> >   * @}
> >   */
> >
> > -enum AVSubtitleType {
> > -SUBTITLE_NONE,
> > -
> > -SUBTITLE_BITMAP,///< A bitmap, pict will be set
> > -
> > -/**
> > - * Plain text, the text field must be set by the decoder and is
> > - * authoritative. ass and pict fields may contain approximations.
> > - */
> > -SUBTITLE_TEXT,
> > -
> > -/**
> > - * Formatted text, the ass field must be set by the decoder and is
> > - * authoritative. pict and text fields may contain approximations.
> > - */
> > -SUBTITLE_ASS,
> > -};
> > -
> >  #define AV_SUBTITLE_FLAG_FORCED 0x0001
> >
> >  typedef struct AVSubtitleRect {
> > diff --git a/libavutil/Makefile b/libavutil/Makefile
> > index 529046dbc8..c7843db1e4 100644
> > --- a/libavutil/Makefile
> > +++ b/libavutil/Makefile
> > @@ -74,6 +74,7 @@ HEADERS = adler32.h
> \
> >sha512.h  \
> >spherical.h   \
> >stereo3d.h\
> > +  subfmt.h  \
> >threadmessage.h   \
> >time.h\
> >timecode.h\
> > diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
> > new file mode 100644
> > index 00..88078a0d14
> > --- /dev/null
> > +++ b/libavutil/subfmt.h
> > @@ -0,0 +1,66 @@
> > +/*
> > + * Copyright (c) 2021 softworkz
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-
> 1301 USA
> > + */
> > +
> > +#ifndef AVUTIL_SUBFMT_H
> > +#define AVUTIL_SUBFMT_H
> > +
> > +enum AVSubtitleType {
> > +
> > +/**
> > + * Subtitle format unknown.
> > + */
> > +AV_SUBTITLE_FMT_NONE = -1,
> > +
> > +/**
> > + * Subtitle format unknown.
> > + */
> > +AV_SUBTITLE_FMT_UNKNOWN = 0,
> > +#ifdef FF_API_OLD_SUBTITLES
> 
> You should include version.h and check via #if, not #ifdef.
> The way you are doing it here only works because avcodec.h already
> happened to indirectly include libavutil/version.h before this header.

I forgot this one. Anton already mentioned it.
Thanks!
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/4] avformat/mxfdec: Check component_depth in mxf_get_color_range()

[Michael Niedermayer]

On Wed, Dec 08, 2021 at 12:38:56AM +0100, Tomas Härdin wrote:
> lör 2021-12-04 klockan 22:32 +0100 skrev Michael Niedermayer:
> > Fixes: shift exponent 4294967163 is too large for 32-bit type 'int'
> > Fixes: 41449/clusterfuzz-testcase-minimized-ffmpeg_IO_DEMUXER_fuzzer-
> > 6183636217495552
> > 
> > Found-by: continuous fuzzing process
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavformat/mxfdec.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavformat/mxfdec.c b/libavformat/mxfdec.c
> > index af9d33f7969..c231c944c01 100644
> > --- a/libavformat/mxfdec.c
> > +++ b/libavformat/mxfdec.c
> > @@ -2274,12 +2274,12 @@ static enum AVColorRange
> > mxf_get_color_range(MXFContext *mxf, MXFDescriptor *des
> >  /* CDCI range metadata */
> >  if (!descriptor->component_depth)
> >  return AVCOL_RANGE_UNSPECIFIED;
> > -    if (descriptor->black_ref_level == 0 &&
> > +    if (descriptor->black_ref_level == 0 && descriptor-
> > >component_depth < 31 &&
> >  descriptor->white_ref_level == ((1< > >component_depth) - 1) &&
> >  (descriptor->color_range    == (1< > >component_depth) ||
> >   descriptor->color_range    == ((1< > >component_depth) - 1)))
> >  return AVCOL_RANGE_JPEG;
> > -    if (descriptor->component_depth >= 8 &&
> > +    if (descriptor->component_depth >= 8 && descriptor-
> > >component_depth < 31 &&
> >  descriptor->black_ref_level == (1  <<(descriptor-
> > >component_depth - 4)) &&
> >  descriptor->white_ref_level == (235<<(descriptor-
> > >component_depth - 8)) &&
> >  descriptor->color_range == ((14<<(descriptor-
> > >component_depth - 4)) + 1))
> 
> Looks OK

will apply

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No human being will ever know the Truth, for even if they happen to say it
by chance, they would not even known they had done so. -- Xenophanes


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v22 01/23] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values

[Andreas Rheinhardt]

Soft Works:
> Signed-off-by: softworkz 
> ---
>  libavcodec/avcodec.h | 19 +
>  libavutil/Makefile   |  1 +
>  libavutil/subfmt.h   | 66 
>  libavutil/version.h  |  1 +
>  4 files changed, 69 insertions(+), 18 deletions(-)
>  create mode 100644 libavutil/subfmt.h
> 
> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> index 7ee8bc2b7c..b05c12e47e 100644
> --- a/libavcodec/avcodec.h
> +++ b/libavcodec/avcodec.h
> @@ -35,6 +35,7 @@
>  #include "libavutil/frame.h"
>  #include "libavutil/log.h"
>  #include "libavutil/pixfmt.h"
> +#include "libavutil/subfmt.h"
>  #include "libavutil/rational.h"
>  
>  #include "codec.h"
> @@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
>   * @}
>   */
>  
> -enum AVSubtitleType {
> -SUBTITLE_NONE,
> -
> -SUBTITLE_BITMAP,///< A bitmap, pict will be set
> -
> -/**
> - * Plain text, the text field must be set by the decoder and is
> - * authoritative. ass and pict fields may contain approximations.
> - */
> -SUBTITLE_TEXT,
> -
> -/**
> - * Formatted text, the ass field must be set by the decoder and is
> - * authoritative. pict and text fields may contain approximations.
> - */
> -SUBTITLE_ASS,
> -};
> -
>  #define AV_SUBTITLE_FLAG_FORCED 0x0001
>  
>  typedef struct AVSubtitleRect {
> diff --git a/libavutil/Makefile b/libavutil/Makefile
> index 529046dbc8..c7843db1e4 100644
> --- a/libavutil/Makefile
> +++ b/libavutil/Makefile
> @@ -74,6 +74,7 @@ HEADERS = adler32.h 
> \
>sha512.h  \
>spherical.h   \
>stereo3d.h\
> +  subfmt.h  \
>threadmessage.h   \
>time.h\
>timecode.h\
> diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
> new file mode 100644
> index 00..88078a0d14
> --- /dev/null
> +++ b/libavutil/subfmt.h
> @@ -0,0 +1,66 @@
> +/*
> + * Copyright (c) 2021 softworkz
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +#ifndef AVUTIL_SUBFMT_H
> +#define AVUTIL_SUBFMT_H
> +
> +enum AVSubtitleType {
> +
> +/**
> + * Subtitle format unknown.
> + */
> +AV_SUBTITLE_FMT_NONE = -1,
> +
> +/**
> + * Subtitle format unknown.
> + */
> +AV_SUBTITLE_FMT_UNKNOWN = 0,
> +#ifdef FF_API_OLD_SUBTITLES

You should include version.h and check via #if, not #ifdef.
The way you are doing it here only works because avcodec.h already
happened to indirectly include libavutil/version.h before this header.

> +SUBTITLE_NONE = 0,  ///< Deprecated, use AV_SUBTITLE_FMT_NONE 
> instead.
> +#endif
> +
> +/**
> + * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
> + */
> +AV_SUBTITLE_FMT_BITMAP = 1,
> +#ifdef FF_API_OLD_SUBTITLES
> +SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP 
> instead.
> +#endif
> +
> +/**
> + * Plain text in AVSubtitleRect.text.
> + */
> +AV_SUBTITLE_FMT_TEXT = 2,
> +#ifdef FF_API_OLD_SUBTITLES
> +SUBTITLE_TEXT = 2,  ///< Deprecated, use AV_SUBTITLE_FMT_TEXT 
> instead.
> +#endif
> +
> +/**
> + * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
> + */
> +AV_SUBTITLE_FMT_ASS = 3,
> +#ifdef FF_API_OLD_SUBTITLES
> +SUBTITLE_ASS = 3,   ///< Deprecated, use AV_SUBTITLE_FMT_ASS 
> instead.
> +#endif
> +
> +AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE 
> THIS if you want to link with shared libav* because the number of formats 
> might differ between versions.
> +};
> +
> +#endif /* AVUTIL_SUBFMT_H */
> diff --git a/libavutil/version.h b/libavutil/version.h
> index 017fc277a6..c6cd0b0d79 100644
> --- a/libavutil/version.h
> +++ b/libavutil/version.h
> @@ -109,6 +109,7 @@
>  #define FF_API_DECLARE_ALIGNED

[FFmpeg-devel] [PATCH v22 23/23] avcodec/webvttenc: convert hard-space tags to

[Soft Works]

Signed-off-by: softworkz 
---
 libavcodec/webvttenc.c   |  6 ++
 tests/ref/fate/sub-webvttenc | 10 +-
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/libavcodec/webvttenc.c b/libavcodec/webvttenc.c
index 43dace6575..a4997969ee 100644
--- a/libavcodec/webvttenc.c
+++ b/libavcodec/webvttenc.c
@@ -123,6 +123,11 @@ static void webvtt_new_line_cb(void *priv, int forced)
 webvtt_print(priv, "\n");
 }
 
+static void webvtt_hard_space_cb(void *priv)
+{
+webvtt_print(priv, " ");
+}
+
 static void webvtt_style_cb(void *priv, char style, int close)
 {
 if (style == 's') // strikethrough unsupported
@@ -147,6 +152,7 @@ static void webvtt_end_cb(void *priv)
 static const ASSCodesCallbacks webvtt_callbacks = {
 .text = webvtt_text_cb,
 .new_line = webvtt_new_line_cb,
+.hard_space   = webvtt_hard_space_cb,
 .style= webvtt_style_cb,
 .color= NULL,
 .font_name= NULL,
diff --git a/tests/ref/fate/sub-webvttenc b/tests/ref/fate/sub-webvttenc
index f4172dcc84..ee9de2859e 100644
--- a/tests/ref/fate/sub-webvttenc
+++ b/tests/ref/fate/sub-webvttenc
@@ -132,26 +132,26 @@ but show this: {normal text}
 \ N is a forced line break
 \ h is a hard space
 Normal spaces at the start and at the end of the line are trimmed while hard 
spaces are not trimmed.
-The line will never break automatically right before or after a hard space. :-D
+The line will never break automatically right before or after a hard space. :-D
 
 00:54.501 --> 00:56.500
 
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
 A (Normal  spaces followed by a letter)
 A (No hard spaces followed by a letter)
 
 00:56.501 --> 00:58.500
- A (05 hard spaces followed by a letter)
+ A (05 hard spaces followed by a letter)
 A (Normal  spaces followed by a letter)
 A (No hard spaces followed by a letter)
 Show this: \TEST and this: \-)
 
 00:58.501 --> 01:00.500
 
-A letter followed by 05 hard spaces: A 
+A letter followed by 05 hard spaces: A 
 A letter followed by normal  spaces: A
 A letter followed by no hard spaces: A
-05 hard  spaces between letters: A A
+05 hard  spaces between letters: A A
 5 normal spaces between letters: A A
 
 ^--Forced line break
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v22 22/23] avutil/ass_split: Add parsing of hard-space tags (\h)

[Soft Works]

The \h tag in ASS/SSA is indicating a non-breaking space. See
https://github.com/Aegisub/aegisite/blob/master/source/docs/3.2/ASS_Tags.html.md

The ass_split implementation is used by almost all text subtitle
encoders and it didn't handle this tag. Interestingly, several tests
are testing for \h parsing and had incorrect reference data for those tests.

The \h tag is specific to ASS and doesn't have any meaning outside of ASS.
Still, the reference data for ttmlenc, textenc and webvttenc were full of
\h tags even though this tag doesn't have a meaning there.

Signed-off-by: softworkz 
---
 libavutil/ass_split.c|  7 +++
 libavutil/ass_split_internal.h   |  1 +
 tests/ref/fate/mov-mp4-ttml-dfxp |  8 
 tests/ref/fate/mov-mp4-ttml-stpp |  8 
 tests/ref/fate/sub-textenc   | 10 +-
 tests/ref/fate/sub-ttmlenc   |  8 
 tests/ref/fate/sub-webvttenc | 10 +-
 7 files changed, 30 insertions(+), 22 deletions(-)

diff --git a/libavutil/ass_split.c b/libavutil/ass_split.c
index c5963351fc..30512dfc74 100644
--- a/libavutil/ass_split.c
+++ b/libavutil/ass_split.c
@@ -484,6 +484,7 @@ int avpriv_ass_split_override_codes(const ASSCodesCallbacks 
*callbacks, void *pr
 while (buf && *buf) {
 if (text && callbacks->text &&
 (sscanf(buf, "\\%1[nN]", new_line) == 1 ||
+ sscanf(buf, "\\%1[hH]", new_line) == 1 ||
  !strncmp(buf, "{\\", 2))) {
 callbacks->text(priv, text, text_len);
 text = NULL;
@@ -492,6 +493,12 @@ int avpriv_ass_split_override_codes(const 
ASSCodesCallbacks *callbacks, void *pr
 if (callbacks->new_line)
 callbacks->new_line(priv, new_line[0] == 'N');
 buf += 2;
+} else if (sscanf(buf, "\\%1[hH]", new_line) == 1) {
+if (callbacks->hard_space)
+callbacks->hard_space(priv);
+else if (callbacks->text)
+callbacks->text(priv, " ", 1);
+buf += 2;
 } else if (!strncmp(buf, "{\\", 2)) {
 buf++;
 while (*buf == '\\') {
diff --git a/libavutil/ass_split_internal.h b/libavutil/ass_split_internal.h
index 8e8e51115c..d6eaade4a4 100644
--- a/libavutil/ass_split_internal.h
+++ b/libavutil/ass_split_internal.h
@@ -141,6 +141,7 @@ typedef struct {
  * @{
  */
 void (*text)(void *priv, const char *text, int len);
+void (*hard_space)(void *priv);
 void (*new_line)(void *priv, int forced);
 void (*style)(void *priv, char style, int close);
 void (*color)(void *priv, unsigned int /* color */, unsigned int color_id);
diff --git a/tests/ref/fate/mov-mp4-ttml-dfxp b/tests/ref/fate/mov-mp4-ttml-dfxp
index e24b5d618b..e565ffa1f6 100644
--- a/tests/ref/fate/mov-mp4-ttml-dfxp
+++ b/tests/ref/fate/mov-mp4-ttml-dfxp
@@ -1,9 +1,9 @@
-2e7e01c821c111466e7a2844826b7f6d *tests/data/fate/mov-mp4-ttml-dfxp.mp4
-8519 tests/data/fate/mov-mp4-ttml-dfxp.mp4
+658884e1b789e75c454b25bdf71283c9 *tests/data/fate/mov-mp4-ttml-dfxp.mp4
+8486 tests/data/fate/mov-mp4-ttml-dfxp.mp4
 #tb 0: 1/1000
 #media_type 0: data
 #codec_id 0: none
-0,  0,  0,68500, 7866, 0x456c36b7
+0,  0,  0,68500, 7833, 0x31b22193
 {
 "packets": [
 {
@@ -15,7 +15,7 @@
 "dts_time": "0.00",
 "duration": 68500,
 "duration_time": "68.50",
-"size": "7866",
+"size": "7833",
 "pos": "44",
 "flags": "K_"
 }
diff --git a/tests/ref/fate/mov-mp4-ttml-stpp b/tests/ref/fate/mov-mp4-ttml-stpp
index 77bd23b7bf..f25b5b2d28 100644
--- a/tests/ref/fate/mov-mp4-ttml-stpp
+++ b/tests/ref/fate/mov-mp4-ttml-stpp
@@ -1,9 +1,9 @@
-cbd2c7ff864a663b0d893deac5a0caec *tests/data/fate/mov-mp4-ttml-stpp.mp4
-8547 tests/data/fate/mov-mp4-ttml-stpp.mp4
+c9570de0ccebc858b0c662a7e449582c *tests/data/fate/mov-mp4-ttml-stpp.mp4
+8514 tests/data/fate/mov-mp4-ttml-stpp.mp4
 #tb 0: 1/1000
 #media_type 0: data
 #codec_id 0: none
-0,  0,  0,68500, 7866, 0x456c36b7
+0,  0,  0,68500, 7833, 0x31b22193
 {
 "packets": [
 {
@@ -15,7 +15,7 @@ cbd2c7ff864a663b0d893deac5a0caec 
*tests/data/fate/mov-mp4-ttml-stpp.mp4
 "dts_time": "0.00",
 "duration": 68500,
 "duration_time": "68.50",
-"size": "7866",
+"size": "7833",
 "pos": "44",
 "flags": "K_"
 }
diff --git a/tests/ref/fate/sub-textenc b/tests/ref/fate/sub-textenc
index 3ea56b38f0..910ca3d6e3 100644
--- a/tests/ref/fate/sub-textenc
+++ b/tests/ref/fate/sub-textenc
@@ -160,18 +160,18 @@ but show this: {normal text}
 \ N is a forced line break
 \ h is a hard space
 Normal spaces at the start and at the end of the line are trimmed while hard 
spaces are not trimmed.
-The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D
+The line w

[FFmpeg-devel] [PATCH v22 21/23] doc/APIchanges: update for subtitle filtering changes

[Soft Works]

Signed-off-by: softworkz 
---
 doc/APIchanges   | 23 +++
 libavcodec/version.h |  2 +-
 libavutil/version.h  |  4 ++--
 3 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 2914ad6734..bb47faf13d 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,29 @@ libavutil: 2021-04-27
 
 API changes, most recent first:
 
+2021-12-05 - xx - lavc 59.15.100 - avcodec.h
+  Deprecate avcodec_encode_subtitle(), use regular encode api now
+
+2021-12-05 - xx - lavc 59.15.100 - codec_desc.h
+  Add avcodec_descriptor_get_subtitle_format()
+
+2021-12-05 - xx - lavc 59.15.100 - avcodec.h
+  Deprecate avsubtitle_free()
+  Deprecate avcodec_decode_subtitle2(), use regular decode api now
+
+2021-12-05 - xx - lavu 57.11.100 - frame.h
+  Add AVMediaType field to AVFrame
+  Add Fields for carrying subtitle data to AVFrame
+  (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.)
+  Add av_frame_get_buffer2() and deprecate av_frame_get_buffer()
+
+2021-12-05 - xx - lavu 57.11.100 - subfmt.h
+  Add struct AVSubtitleArea (replaces AVSubtitle)
+  Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt()
+
+2021-12-05 - xx - lavu 57.11.100 - subfmt.h
+  Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing
+
 2021-11-xx - xx - lavfi 8.19.100 - avfilter.h
   Add AVFILTER_FLAG_METADATA_ONLY.
 
diff --git a/libavcodec/version.h b/libavcodec/version.h
index 8a0b94f5aa..db71fb1130 100644
--- a/libavcodec/version.h
+++ b/libavcodec/version.h
@@ -28,7 +28,7 @@
 #include "libavutil/version.h"
 
 #define LIBAVCODEC_VERSION_MAJOR  59
-#define LIBAVCODEC_VERSION_MINOR  14
+#define LIBAVCODEC_VERSION_MINOR  15
 #define LIBAVCODEC_VERSION_MICRO 100
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
diff --git a/libavutil/version.h b/libavutil/version.h
index c6cd0b0d79..cc8caadf50 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,8 +79,8 @@
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  57
-#define LIBAVUTIL_VERSION_MINOR  10
-#define LIBAVUTIL_VERSION_MICRO 101
+#define LIBAVUTIL_VERSION_MINOR  11
+#define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
LIBAVUTIL_VERSION_MINOR, \
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v22 20/23] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles

[Soft Works]

Signed-off-by: softworkz 
---
 configure |   1 +
 doc/filters.texi  | 164 +++
 libavfilter/Makefile  |   1 +
 libavfilter/allfilters.c  |   1 +
 libavfilter/sf_subscale.c | 883 ++
 5 files changed, 1050 insertions(+)
 create mode 100644 libavfilter/sf_subscale.c

diff --git a/configure b/configure
index 360e91d762..4be4303380 100755
--- a/configure
+++ b/configure
@@ -3706,6 +3706,7 @@ sr_filter_deps="avformat swscale"
 sr_filter_select="dnn"
 stereo3d_filter_deps="gpl"
 splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
 subtitles_filter_deps="avformat avcodec libass"
 super2xsai_filter_deps="gpl"
 pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 26bf6014d0..8c49f79f86 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26226,6 +26226,170 @@ Set the rendering margin in pixels.
 For rendering, alway use the latest event only, which is covering the given 
point in time.
 @end table
 
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical 
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make sure that subtitle 

[FFmpeg-devel] [PATCH v22 19/23] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)

[Soft Works]

Signed-off-by: softworkz 
---
 configure|   1 +
 doc/filters.texi |  55 +
 libavfilter/Makefile |   2 +
 libavfilter/allfilters.c |   1 +
 libavfilter/sf_graphicsub2text.c | 354 +++
 5 files changed, 413 insertions(+)
 create mode 100644 libavfilter/sf_graphicsub2text.c

diff --git a/configure b/configure
index 73a7267cf1..360e91d762 100755
--- a/configure
+++ b/configure
@@ -3640,6 +3640,7 @@ frei0r_filter_deps="frei0r"
 frei0r_src_filter_deps="frei0r"
 fspp_filter_deps="gpl"
 gblur_vulkan_filter_deps="vulkan spirv_compiler"
+graphicsub2text_filter_deps="libtesseract"
 hflip_vulkan_filter_deps="vulkan spirv_compiler"
 histeq_filter_deps="gpl"
 hqdn3d_filter_deps="gpl"
diff --git a/doc/filters.texi b/doc/filters.texi
index 743c36c432..26bf6014d0 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25820,6 +25820,61 @@ ffmpeg -i 
"https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple
 @end example
 @end itemize
 
+@section graphicsub2text
+
+Converts graphic subtitles to text subtitles by performing OCR.
+
+For this filter to be available, ffmpeg needs to be compiled with libtesseract 
(see https://github.com/tesseract-ocr/tesseract).
+Language models need to be downloaded from 
https://github.com/tesseract-ocr/tessdata and put into as subfolder named 
'tessdata' or into a folder specified via the environment variable 
'TESSDATA_PREFIX'.
+The path can also be specified via filter option (see below).
+
+Note: These models are including the data for both OCR modes.
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [text]
+
+It accepts the following parameters:
+
+@table @option
+@item ocr_mode
+The character recognition mode to use.
+
+Supported OCR modes are:
+
+@table @var
+@item 0, tesseract
+This is the classic libtesseract operation mode. It is fast but less accurate 
than LSTM.
+@item 1, lstm
+Newer OCR implementation based on ML models. Provides usually better results, 
requires more processing resources.
+@item 2, both
+Use a combination of both modes.
+@end table
+
+@item tessdata_path
+The path to a folder containing the language models to be used.
+
+@item language
+The recognition language. It needs to match the first three characters of a  
language model file in the tessdata path.
+
+@end table
+
+
+@subsection Examples
+
+@itemize
+@item
+Convert DVB graphic subtitles to ASS (text) subtitles
+
+Note: For this to work, you need to have the data file 'eng.traineddata' in a 
'tessdata' subfolder (see above).
+@example
+ffmpeg ffmpeg -loglevel verbose -i 
"https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; 
-filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv
+@end example
+@end itemize
+
+
 @section graphicsub2video
 
 Renders graphic subtitles as video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 2224e5fe5f..3b972e134b 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -296,6 +296,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER)   += 
vf_gblur_vulkan.o vulkan.o vulka
 OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
 OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
 OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER)   += vf_overlaygraphicsubs.o 
framesync.o
+OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER)   += vf_overlaygraphicsubs.o 
framesync.o
 OBJS-$(CONFIG_GRAPHMONITOR_FILTER)   += f_graphmonitor.o
 OBJS-$(CONFIG_GRAYWORLD_FILTER)  += vf_grayworld.o
 OBJS-$(CONFIG_GREYEDGE_FILTER)   += vf_colorconstancy.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 6adde2b9f6..f70f08dc5a 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -545,6 +545,7 @@ extern const AVFilter ff_avf_showwaves;
 extern const AVFilter ff_avf_showwavespic;
 extern const AVFilter ff_vaf_spectrumsynth;
 extern const AVFilter ff_sf_censor;
+extern const AVFilter ff_sf_graphicsub2text;
 extern const AVFilter ff_sf_showspeaker;
 extern const AVFilter ff_sf_splitcc;
 extern const AVFilter ff_sf_stripstyles;
diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c
new file mode 100644
index 00..ef10d60efd
--- /dev/null
+++ b/libavfilter/sf_graphicsub2text.c
@@ -0,0 +1,354 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for mor

[FFmpeg-devel] [PATCH v22 18/23] avfilter/splitcc: Add splitcc filter for closed caption handling

[Soft Works]

- splitcc {V -> VS)
  Extract closed-caption (A53) data from video
  frames as subtitle Frames

ffmpeg -y -loglevel verbose -i 
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts"; 
-filter_complex 
"[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs" 
output.mkv

Signed-off-by: softworkz 
---
 configure|   1 +
 doc/filters.texi |  63 +++
 libavfilter/Makefile |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/sf_splitcc.c | 378 +++
 5 files changed, 444 insertions(+)
 create mode 100644 libavfilter/sf_splitcc.c

diff --git a/configure b/configure
index 16ab6b46c3..73a7267cf1 100755
--- a/configure
+++ b/configure
@@ -3704,6 +3704,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
 sr_filter_deps="avformat swscale"
 sr_filter_select="dnn"
 stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
 subtitles_filter_deps="avformat avcodec libass"
 super2xsai_filter_deps="gpl"
 pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 241c5d469b..743c36c432 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26054,6 +26054,69 @@ ffmpeg -i INPUT -filter_complex 
"showspeaker=format=colon:style='@{\\c&HDD&\
 @end example
 @end itemize
 
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just 
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame 
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle 
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video 
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts"; 
-filter_complex  
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs" 
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the 
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely 
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts"; 
-filter_complex  
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
 output.mkv
+@end example
+@end itemize
+
+
 @section textsub2video
 
 Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7d7da0c59a..2224e5fe5f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER)   += 
vsink_nullsink.o
 OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
 OBJS-$(CONFIG_SHOW_SPEAKER_FILTER)   += sf_textmod.o
 OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
 OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
 
 # multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 27680a2f00..6adde2b9f6 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
 extern const AVFilter ff_vaf_spectrumsynth;
 extern const AVFilter ff_sf_censor;
 extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
 extern const AVFilter ff_sf_stripstyles;
 extern const AVFilter ff_sf_textmod;
 extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..3556d084d9
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,378 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABIL

[FFmpeg-devel] [PATCH v22 17/23] avfilter/stripstyles: Add stripstyles filter

[Soft Works]

- stripstyles {S -> S)
  Remove all inline styles from subtitle events

Signed-off-by: softworkz 
---
 doc/filters.texi |  37 +++
 libavfilter/Makefile |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/sf_stripstyles.c | 196 +++
 4 files changed, 235 insertions(+)
 create mode 100644 libavfilter/sf_stripstyles.c

diff --git a/doc/filters.texi b/doc/filters.texi
index e87d3993df..241c5d469b 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25704,6 +25704,43 @@ ffmpeg -i 
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
 @end example
 @end itemize
 
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle 
lines for creating effects, so in most cases it is safe to remove the animation 
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter 
out certain effects where an ASS author duplicates the text onto multiple 
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass 
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv"; 
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
 
 @section textmod
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index d2995fbec6..7d7da0c59a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER)   += 
vsink_nullsink.o
 OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
 OBJS-$(CONFIG_SHOW_SPEAKER_FILTER)   += sf_textmod.o
 OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
 
 # multimedia filters
 OBJS-$(CONFIG_ABITSCOPE_FILTER)  += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 673f9fb839..27680a2f00 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
 extern const AVFilter ff_vaf_spectrumsynth;
 extern const AVFilter ff_sf_censor;
 extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
 extern const AVFilter ff_sf_textmod;
 extern const AVFilter ff_svf_graphicsub2video;
 extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..82cb9c7647
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,196 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void di

[FFmpeg-devel] [PATCH v22 16/23] avfilter/textmod: Add textmod, censor and show_speaker filters

[Soft Works]

- textmod {S -> S)
  Modify subtitle text in a number of ways

- censor {S -> S)
  Censor subtitles using a word list

- show_speaker {S -> S)
  Prepend speaker names from ASS subtitles to the visible text lines

Signed-off-by: softworkz 
---
 doc/filters.texi | 206 
 libavfilter/Makefile |   5 +
 libavfilter/allfilters.c |   3 +
 libavfilter/sf_textmod.c | 697 +++
 4 files changed, 911 insertions(+)
 create mode 100644 libavfilter/sf_textmod.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 7041f9031e..e87d3993df 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25644,6 +25644,145 @@ existing filters using @code{--disable-filters}.
 
 Below is a description of the currently available subtitle filters.
 
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last 
character of a word.
+For words with less than 4 characters, the last character will be replaced as 
well.
+For words with less than 3 characters, the first character will be replaced as 
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a 
word.
+For words with less than 3 characters, the first character will be replaced as 
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must 
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i 
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv"; 
-filter_complex 
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y 
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as 
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be 
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all 
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be 
specified. Multiple words must be separated by the delimiter char specified vie 
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all 
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple 
words must be separated by the delimiter char specified vie the separator 
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must 
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; 
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smooth

[FFmpeg-devel] [PATCH v22 15/23] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters

[Soft Works]

- overlaytextsubs {VS -> V)
  Overlay text subtitles onto a video stream.

- textsubs2video {S -> V)
  Converts text subtitles to video frames

Signed-off-by: softworkz 
---
 configure|   2 +
 doc/filters.texi | 113 ++
 libavfilter/Makefile |   2 +
 libavfilter/allfilters.c |   4 +-
 libavfilter/vf_overlaytextsubs.c | 646 +++
 5 files changed, 766 insertions(+), 1 deletion(-)
 create mode 100644 libavfilter/vf_overlaytextsubs.c

diff --git a/configure b/configure
index 7b203d2bf2..16ab6b46c3 100755
--- a/configure
+++ b/configure
@@ -3666,6 +3666,7 @@ overlay_opencl_filter_deps="opencl"
 overlay_qsv_filter_deps="libmfx"
 overlay_qsv_filter_select="qsvvpp"
 overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
 owdenoise_filter_deps="gpl"
 pad_opencl_filter_deps="opencl"
 pan_filter_deps="swresample"
@@ -3710,6 +3711,7 @@ superequalizer_filter_deps="avcodec"
 superequalizer_filter_select="rdft"
 surround_filter_deps="avcodec"
 surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
 tinterlace_filter_deps="gpl"
 tinterlace_merge_test_deps="tinterlace_filter"
 tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index db4552ba10..7041f9031e 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25752,6 +25752,119 @@ Overlay PGS subtitles
 ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
 @end example
 @end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it 
does no longer require to open and access the source stream separately, which 
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, 
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given 
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i 
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv"; 
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the 
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when 
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to 
display are changing.
+In cases where subtitles include advanced features (like animation), this 
parameter determines the frequency by which the overlay frames should be 
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given 
point in time.
+@end table
+
 @c man end SUBTITLE FILTERS
 
 @chapter Multimedia Filters
diff --git a/libavf

[FFmpeg-devel] [PATCH v22 13/23] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API

[Soft Works]

This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.

The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).

The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.

Justification for changed test refs:

- sub2video
  The new results are identical excepting the last frame which
  is due to the implementation changes

- sub2video_basic
  The previous results had some incorrect output because multiple
  frames had the same dts
  The non-empty content frames are visually identical, the different
  CRC is due to the different blending algorithm that is being used.

- sub2video_time_limited
  The third frame in the previous ref was a repetition, which doesn't
  happen anymore with the new subtitle filtering.

- sub-dvb
  Running ffprobe -show_frames on the source file shows that there
  are 7 subtitle frames with 0 rects in the source at the start
  and 2 at the end. This translates to the 14 and 4 additional
  entries in the new test results.

- filter-overlay-dvdsub-2397
  Overlay results have slightly different CRCs due to different
  blending implementation

Signed-off-by: softworkz 
---
 fftools/ffmpeg.c  | 584 +++---
 fftools/ffmpeg.h  |  15 +-
 fftools/ffmpeg_filter.c   | 217 +---
 fftools/ffmpeg_hw.c   |   2 +-
 fftools/ffmpeg_opt.c  |   3 +-
 tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ---
 tests/ref/fate/sub-dvb| 162 +++---
 tests/ref/fate/sub2video  | 116 ++---
 tests/ref/fate/sub2video_basic| 135 ++---
 tests/ref/fate/sub2video_time_limited |   4 +-
 10 files changed, 725 insertions(+), 694 deletions(-)

diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 0f1834cdbe..0eee4163ac 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -143,8 +143,6 @@ static int want_sdp = 1;
 static BenchmarkTimeStamps current_time;
 AVIOContext *progress_avio = NULL;
 
-static uint8_t *subtitle_out;
-
 InputStream **input_streams = NULL;
 intnb_input_streams = 0;
 InputFile   **input_files   = NULL;
@@ -169,163 +167,6 @@ static int restore_tty;
 static void free_input_threads(void);
 #endif
 
-/* sub2video hack:
-   Convert subtitles to video with alpha to insert them in filter graphs.
-   This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width  = ist->dec_ctx->width  ? ist->dec_ctx->width  
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height 
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d) 
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
-   AV_BUFFERSRC_FLAG_KEEP_REF |
-   AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer 
source(%s).\n",
-   av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
- 

[FFmpeg-devel] [PATCH v22 14/23] avfilter/avfilter: Fix hardcoded input index

[Soft Works]

This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.

Signed-off-by: softworkz 
---
 libavfilter/avfilter.c | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int 
status, AVRational lin
 return AV_NOPTS_VALUE;
 }
 
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
 {
 int ret = -1;
 
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
 link->frame_blocked_in = 1;
 if (link->srcpad->request_frame)
 ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
 if (ret < 0) {
 if (ret != AVERROR(EAGAIN) && ret != link->status_in)
 ff_avfilter_link_set_in_status(link, ret, 
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext 
*filter, AVFilterLink *in)
 {
 unsigned out = 0, progress = 0;
 int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
 
 av_assert0(!in->status_out);
 if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter, 
AVFilterLink *in)
 while (!in->status_out) {
 if (!filter->outputs[out]->status_in) {
 progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out], 
input_index);
 if (ret < 0)
 return ret;
 }
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext 
*filter)
 for (i = 0; i < filter->nb_outputs; i++) {
 if (filter->outputs[i]->frame_wanted_out &&
 !filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
 }
 }
 return FFERROR_NOT_READY;
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v22 12/23] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters

[Soft Works]

- overlaygraphicsubs (VS -> V)
  Overlay graphic subtitles onto a video stream

- graphicsub2video {S -> V)
  Converts graphic subtitles to video frames (with alpha)
  Gets auto-inserted for retaining compatibility with
  sub2video command lines

Signed-off-by: softworkz 
---
 doc/filters.texi| 118 +
 libavfilter/Makefile|   2 +
 libavfilter/allfilters.c|   2 +
 libavfilter/vf_overlaygraphicsubs.c | 737 
 4 files changed, 859 insertions(+)
 create mode 100644 libavfilter/vf_overlaygraphicsubs.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 8eff460cd9..db4552ba10 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25636,6 +25636,124 @@ tools.
 
 @c man end VIDEO SINKS
 
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion 
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is 
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the 
'overlay_graphicsubs' filter which is more efficient and takes less processing 
resources.
+
+This filter is still useful in cases where the overlay is done with hardware 
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the 
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy 
command lines.
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e. 
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles, 
which also means that no processing is done at times where no subtitles are to 
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24, 
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i 
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
 -filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
 @chapter Multimedia Filters
 @c man begin MULTIMEDIA FILTERS
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 664566a18a..525b3a6e3c 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -295,6 +295,7 @@ OBJS-$(CONFIG_GBLUR_FILTER)  += vf_gblur.o
 OBJS-$(CONFIG_GBLUR_VULKAN_FILTER)   += vf_gblur_vulkan.o vulkan.o 
vulkan_filter.o
 OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
 OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER)   += vf_overlaygraphicsubs.o 
framesync.o
 OBJS-$(CONFIG_GRAPHMONITOR_FILTER)   += f_graphmonitor.o
 OBJS-$(CONFIG_GRAYWORLD_FILTER)  

[FFmpeg-devel] [PATCH v22 11/23] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters

[Soft Works]

Signed-off-by: softworkz 
---
 configure|  2 +-
 libavfilter/allfilters.c |  2 ++
 libavfilter/buffersink.c | 54 ++
 libavfilter/buffersink.h |  7 
 libavfilter/buffersrc.c  | 72 
 libavfilter/buffersrc.h  |  1 +
 6 files changed, 137 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 04f319b197..7b203d2bf2 100755
--- a/configure
+++ b/configure
@@ -7808,7 +7808,7 @@ print_enabled_components(){
 fi
 done
 if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer 
vsink_buffer ssink_sbuffer; do
 printf "&ff_%s,\n" $c >> $TMPH
 done
 fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1af2cbcc8..e134ac8059 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -553,8 +553,10 @@ extern const AVFilter ff_avsrc_movie;
  * being the same while having different 'types'). */
 extern  const AVFilter ff_asrc_abuffer;
 extern  const AVFilter ff_vsrc_buffer;
+extern  const AVFilter ff_ssrc_sbuffer;
 extern  const AVFilter ff_asink_abuffer;
 extern  const AVFilter ff_vsink_buffer;
+extern  const AVFilter ff_ssink_sbuffer;
 extern const AVFilter ff_af_afifo;
 extern const AVFilter ff_vf_fifo;
 
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
 #include "libavutil/internal.h"
 #include "libavutil/opt.h"
 
+#include "libavcodec/avcodec.h"
+
 #define FF_INTERNAL_FIELDS 1
 #include "framequeue.h"
 
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
 int *sample_rates;  ///< list of accepted sample rates
 int sample_rates_size;
 
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle 
types, must be terminated with -1
+int subtitle_types_size;
+
 AVFrame *peeked_frame;
 } BufferSinkContext;
 
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
 return 0;
 }
 
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i])) 
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
 #define OFFSET(x) offsetof(BufferSinkContext, x)
 #define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
 static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
 { NULL },
 };
 #undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats", 
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
 
 AVFILTER_DEFINE_CLASS(buffersink);
 AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
 
 static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
 {
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
 .outputs   = NULL,
 FILTER_QUERY_FUNC(asink_query_formats),
 };
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name  = "sbuffersink",
+.description   = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make 
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init  = common_init,
+.activate  = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs   = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
  */
 attribute_deprecated
 AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
 #endif
 
 /**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/libavfilter/buffersrc.

[FFmpeg-devel] [PATCH v22 10/23] avfilter/avfilter: Handle subtitle frames

[Soft Works]

Signed-off-by: softworkz 
---
 libavfilter/avfilter.c  |  8 +---
 libavfilter/avfilter.h  | 11 +++
 libavfilter/avfiltergraph.c |  5 +
 libavfilter/formats.c   | 22 ++
 libavfilter/formats.h   |  3 +++
 libavfilter/internal.h  | 18 +++---
 6 files changed, 61 insertions(+), 6 deletions(-)

diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
 ref->linesize[0], ref->linesize[1], ref->linesize[2], 
ref->linesize[3],
 ref->pts, ref->pkt_pos);
 
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
 ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
 ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
 ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
 ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
 ref->key_frame,
 av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
 ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
 ref->channel_layout,
 ref->nb_samples,
 ref->sample_rate);
+break;
 }
 
 ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
 #include "libavutil/log.h"
 #include "libavutil/samplefmt.h"
 #include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
 #include "libavutil/rational.h"
 
 #include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
  * and outputs use the same sample rate and channel count/layout.
  */
 const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
 /**
  * Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
  */
@@ -351,6 +358,10 @@ typedef struct AVFilter {
  * Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
  */
 enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
 } formats;
 
 int priv_size;  ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log, 
AVFilterLink *link, AVFilterForm
 return ret;
 break;
 
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
 default:
 av_assert0(!"reached");
 }
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void 
*log_ctx)
 if (!link)
 continue;
 
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
 neg = ff_filter_get_negotiation(link);
 av_assert0(neg);
 for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..5c972bb183 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include "libavutil/subfmt.h"
 #include "libavutil/avassert.h"
 #include "libavutil/channel_layout.h"
 #include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l, 
uint64_t channel_layout)
 return 0;
 }
 
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
 AVFilterFormats *ff_make_formats_list_singleton(int fmt)
 {
 int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
 return NULL;
 fmt++;
 }
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
 }
 
 return ret;
@@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx)
 type= AVMEDIA_TYPE_AUDIO

[FFmpeg-devel] [PATCH v22 09/23] avfilter/subtitles: Add subtitles.c for subtitle frame allocation

[Soft Works]

Analog to avfilter/video.c and avfilter/audio.c

Signed-off-by: softworkz 
---
 libavfilter/Makefile|  1 +
 libavfilter/avfilter.c  |  4 +++
 libavfilter/internal.h  |  1 +
 libavfilter/subtitles.c | 63 +
 libavfilter/subtitles.h | 44 
 5 files changed, 113 insertions(+)
 create mode 100644 libavfilter/subtitles.c
 create mode 100644 libavfilter/subtitles.h

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c8082c4a2f..664566a18a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o   
  \
framequeue.o \
graphdump.o  \
graphparser.o\
+   subtitles.o  \
video.o  \
 
 OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
 #include "formats.h"
 #include "framepool.h"
 #include "internal.h"
+#include "subtitles.h"
 
 #include "libavutil/ffversion.h"
 const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link, 
AVFrame **rframe)
 case AVMEDIA_TYPE_AUDIO:
 out = ff_get_audio_buffer(link, frame->nb_samples);
 break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
 default:
 return AVERROR(EINVAL);
 }
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
 union {
 AVFrame *(*video)(AVFilterLink *link, int w, int h);
 AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
 } get_buffer;
 
 /**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei

[FFmpeg-devel] [PATCH v22 08/23] fftools/play, probe: Adjust for subtitle changes

[Soft Works]

Signed-off-by: softworkz 
---
 fftools/ffplay.c  | 102 +-
 fftools/ffprobe.c |  48 ++
 2 files changed, 78 insertions(+), 72 deletions(-)

diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..0af32888da 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
 /* Common struct for handling all types of decoded data and allocated render 
buffers. */
 typedef struct Frame {
 AVFrame *frame;
-AVSubtitle sub;
 int serial;
 double pts;   /* presentation timestamp for the frame */
 double duration;  /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx, 
PacketQueue *queue, S
 return 0;
 }
 
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
 int ret = AVERROR(EAGAIN);
 
 for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame, 
AVSubtitle *sub) {
 }
 }
 break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
 }
 if (ret == AVERROR_EOF) {
 d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame 
*frame, AVSubtitle *sub) {
 av_packet_unref(d->pkt);
 } while (1);
 
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) : 
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both 
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
 } else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet 
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
 }
 }
 }
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
 static void frame_queue_unref_item(Frame *vp)
 {
 av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
 }
 
 static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size, 
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
 if (frame_queue_nb_remaining(&is->subpq) > 0) {
 sp = frame_queue_peek(&is->subpq);
 
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time / 
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time / 
1000)) {
 if (!sp->uploaded) {
 uint8_t* pixels[4];
 int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
 if (realloc_texture(&is->sub_texture, 
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
 return;
 
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w = 
area->w, .h = area->h };
 
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width  - 
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height - 
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width  - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
 
 is->sub_convert_ctx = 
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec

[FFmpeg-devel] [PATCH v22 07/23] avcodec/subtitles: Replace deprecated enum values

[Soft Works]

Signed-off-by: softworkz 
---
 libavcodec/ass.h   | 2 +-
 libavcodec/assdec.c| 2 +-
 libavcodec/dvbsubdec.c | 2 +-
 libavcodec/dvdsubdec.c | 2 +-
 libavcodec/dvdsubenc.c | 2 +-
 libavcodec/pgssubdec.c | 2 +-
 libavcodec/xsubdec.c   | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 9c9cb01c8a..8e885b3ae2 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -82,7 +82,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const 
char *dialog,
 rects[sub->num_rects]   = av_mallocz(sizeof(*rects[0]));
 if (!rects[sub->num_rects])
 return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
 ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
 if (!ass_str)
 return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void 
*data, int *got_sub_ptr,
 if (!sub->rects[0])
 return AVERROR(ENOMEM);
 sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
 sub->rects[0]->ass  = av_strdup(avpkt->data);
 if (!sub->rects[0]->ass)
 return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 81ccaf4c57..b13244c803 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx, 
AVSubtitle *sub, int *got_ou
 rect->w = region->width;
 rect->h = region->height;
 rect->nb_colors = (1 << region->depth);
-rect->type  = SUBTITLE_BITMAP;
+rect->type  = AV_SUBTITLE_FMT_BITMAP;
 rect->linesize[0] = region->width;
 
 clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx, 
AVSubtitle *sub_header,
 sub_header->rects[0]->y = y1;
 sub_header->rects[0]->w = w;
 sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
 sub_header->rects[0]->linesize[0] = w;
 sub_header->rects[0]->flags = is_menu ? 
AV_SUBTITLE_FLAG_FORCED : 0;
 }
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index 4916410fc8..fbab88a992 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -273,7 +273,7 @@ static int encode_dvd_subtitles(AVCodecContext* avctx, 
AVPacket* avpkt,
 return AVERROR(EINVAL);
 
 for (i = 0; i < rects; i++)
-if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) {
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) {
 av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
 return AVERROR(EINVAL);
 }
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index 388639a110..4829babb7c 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -539,7 +539,7 @@ static int display_end_segment(AVCodecContext *avctx, void 
*data,
 return AVERROR(ENOMEM);
 }
 sub->num_rects++;
-sub->rects[i]->type = SUBTITLE_BITMAP;
+sub->rects[i]->type = AV_SUBTITLE_FMT_BITMAP;
 
 /* Process bitmap */
 object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 979399bae6..5c9e65f737 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data, 
int *got_sub_ptr,
 }
 sub->rects[0]->x = x; sub->rects[0]->y = y;
 sub->rects[0]->w = w; sub->rects[0]->h = h;
-sub->rects[0]->type = SUBTITLE_BITMAP;
+sub->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
 sub->rects[0]->linesize[0] = w;
 sub->rects[0]->data[0] = av_malloc(w * h);
 sub->rects[0]->nb_colors = 4;
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v22 06/23] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api

[Soft Works]

Signed-off-by: softworkz 
---
 libavcodec/assenc.c|  90 +
 libavcodec/avcodec.h   |   5 +-
 libavcodec/dvbsubenc.c |  96 +--
 libavcodec/dvdsubenc.c | 100 +++--
 libavcodec/encode.c|  63 -
 libavcodec/movtextenc.c| 112 +++--
 libavcodec/srtenc.c| 104 ++
 libavcodec/tests/avcodec.c |   2 -
 libavcodec/ttmlenc.c   |  98 
 libavcodec/webvttenc.c |  82 +++
 libavcodec/xsubenc.c   |  87 +---
 11 files changed, 584 insertions(+), 255 deletions(-)

diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..94601bba68 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,48 +22,92 @@
 #include 
 
 #include "avcodec.h"
+#include "encode.h"
 #include "libavutil/ass_internal.h"
 #include "internal.h"
 #include "libavutil/avstring.h"
 #include "libavutil/internal.h"
 #include "libavutil/mem.h"
 
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header = 
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
 static av_cold int ass_encode_init(AVCodecContext *avctx)
 {
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header, 
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header, 
avctx->subtitle_header_size);
+avctx->extradata_size   = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
 return 0;
 }
 
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
 {
-int i, len, total_len = 0;
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
 
-for (i=0; inum_rects; i++) {
-const char *ass = sub->rects[i]->ass;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
 
-if (sub->rects[i]->type != SUBTITLE_ASS) {
-av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n");
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type 
supported.\n");
 return AVERROR(EINVAL);
 }
 
-len = av_strlcpy(buf+total_len, ass, bufsize-total_len);
+if (ass)
+req_len += strlen(ass);
+}
 
-if (len > bufsize-total_len-1) {
-av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n");
-return AVERROR_BUFFER_TOO_SMALL;
-}
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
 
-total_len += len;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (ass) {
+size_t len = av_strlcpy((char *)avpkt->data + total_len, ass, 
avpkt->size - total_len);
+total_len += len;
+}
 }
 
-return total_len;
+avpkt->size = total_len;
+*got_packet = total_len > 0;
+
+return 0;
 }
 
 #if CONFIG_SSA_ENCODER
@@ -73,7 +117,7 @@ const AVCodec ff_ssa_encoder = {
 .type = AVMEDIA_TYPE_SUBTITLE,
 .id  

[FFmpeg-devel] [PATCH v22 05/23] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing

[Soft Works]

Signed-off-by: softworkz 
---
 libavcodec/Makefile   |  56 +++
 libavcodec/ass.h  | 147 ++
 libavcodec/assdec.c   |   2 +-
 libavcodec/assenc.c   |   2 +-
 libavcodec/ccaption_dec.c |  19 +--
 libavcodec/jacosubdec.c   |   2 +-
 libavcodec/libaribb24.c   |   2 +-
 libavcodec/libzvbi-teletextdec.c  |  14 +-
 libavcodec/microdvddec.c  |   7 +-
 libavcodec/movtextdec.c   |   3 +-
 libavcodec/movtextenc.c   |  20 +--
 libavcodec/mpl2dec.c  |   2 +-
 libavcodec/realtextdec.c  |   2 +-
 libavcodec/samidec.c  |   2 +-
 libavcodec/srtdec.c   |   2 +-
 libavcodec/srtenc.c   |  16 +-
 libavcodec/subviewerdec.c |   2 +-
 libavcodec/textdec.c  |   4 +-
 libavcodec/ttmlenc.c  |  15 +-
 libavcodec/webvttdec.c|   2 +-
 libavcodec/webvttenc.c|  16 +-
 libavutil/Makefile|   2 +
 {libavcodec => libavutil}/ass.c   |  87 ---
 libavutil/ass_internal.h  | 133 
 {libavcodec => libavutil}/ass_split.c |  30 ++--
 .../ass_split_internal.h  |  24 +--
 26 files changed, 339 insertions(+), 274 deletions(-)
 rename {libavcodec => libavutil}/ass.c (65%)
 create mode 100644 libavutil/ass_internal.h
 rename {libavcodec => libavutil}/ass_split.c (94%)
 rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (89%)

diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 4122a9b144..b12638da5e 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o 
pngdsp.o
 OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
 OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
 OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
 OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
 OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
 OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -253,7 +253,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
 OBJS-$(CONFIG_C93_DECODER) += c93.o
 OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
   cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
 OBJS-$(CONFIG_CDGRAPHICS_DECODER)  += cdgraphics.o
 OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
 OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -425,7 +425,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER)   += interplayacm.o
 OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER)  += dpcm.o
 OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
 OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
 OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
   jpeg2000dwt.o
 OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o 
jpeg2000dsp.o \
@@ -447,7 +447,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
 OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
 OBJS-$(CONFIG_METASOUND_DECODER)   += metasound.o metasound_data.o \
   twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
 OBJS-$(CONFIG_MIMIC_DECODER)   += mimic.o
 OBJS-$(CONFIG_MJPEG_DECODER)   += mjpegdec.o mjpegdec_common.o
 OBJS-$(CONFIG_MJPEG_QSV_DECODER)   += qsvdec.o
@@ -462,8 +462,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
 OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
 OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
 OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += movtextenc.o ass_split.o
+OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o
+OBJS-$(CONFIG_MOVTE

[FFmpeg-devel] [PATCH v22 04/23] avfilter/subtitles: Update vf_subtitles to use new decoding api

[Soft Works]

Signed-off-by: softworkz 
---
 libavfilter/vf_subtitles.c | 54 +-
 1 file changed, 42 insertions(+), 12 deletions(-)

diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 3fc4eeb63d..66b2082894 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
 # include "libavformat/avformat.h"
 #endif
 #include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
 #include "libavutil/opt.h"
 #include "libavutil/parseutils.h"
 #include "drawutils.h"
 #include "avfilter.h"
 #include "internal.h"
 #include "formats.h"
-#include "video.h"
 
 typedef struct AssContext {
 const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
 return 0;
 }
 
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame, 
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
 AVFILTER_DEFINE_CLASS(subtitles);
 
 static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
 AVStream *st;
 AVPacket pkt;
 AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
 
 /* Init libass */
 ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
 ret = AVERROR_DECODER_NOT_FOUND;
 goto end;
 }
+
 dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
 av_log(ctx, AV_LOG_ERROR,
-   "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+   "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
 goto end;
 }
+
 if (ass->charenc)
 av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
 
@@ -448,18 +474,22 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
   dec_ctx->subtitle_header_size);
 while (av_read_frame(fmt, &pkt) >= 0) {
 int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
 
 if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
 if (ret < 0) {
 av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
 } else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts, 
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration   = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time = av_rescale_q(sub->subtitle_pts, 
AV_TIME_BASE_Q, av_make_q(1, 1000));
+const int64_t duration   = sub->subtitle_end_time;
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
 if (!ass_line)
 break;
 ass_process_chunk(ass->track, ass_line, strlen(ass_line),
@@ -468,7 +498,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
 }
 }
 av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
 }
 
 end:
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v22 03/23] avcodec/subtitles: Introduce new frame-based subtitle decoding API

[Soft Works]

- Add avcodec_decode_subtitle3 which takes subtitle frames,
  serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API

Signed-off-by: softworkz 
---
 libavcodec/avcodec.h|   8 +-
 libavcodec/codec_desc.c |  11 +++
 libavcodec/codec_desc.h |   8 ++
 libavcodec/decode.c |  54 ++--
 libavcodec/internal.h   |  16 
 libavcodec/utils.c  | 182 
 6 files changed, 270 insertions(+), 9 deletions(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index b05c12e47e..3e734d3003 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
 
 /**
  * Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
  * [Script Info] and [V4+ Styles] section, plus the [Events] line and
  * the Format line following. It shouldn't include any Dialogue line.
  * - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2417,7 +2417,10 @@ int avcodec_close(AVCodecContext *avctx);
  * Free all allocated data in the given subtitle struct.
  *
  * @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
  */
+attribute_deprecated
 void avsubtitle_free(AVSubtitle *sub);
 
 /**
@@ -2510,7 +2513,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int 
xpos, int ypos);
  * must be freed with avsubtitle_free if *got_sub_ptr is set.
  * @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed, 
otherwise, it is nonzero.
  * @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet, 
avcodec_receive_frame) instead.
  */
+attribute_deprecated
 int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
 int *got_sub_ptr,
 AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID 
codec_id)
 const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
 return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
 }
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const 
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const 
AVCodecDescriptor *prev);
  */
 const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
 
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const 
AVCodecDescriptor *codec_descriptor);
+
 /**
  * @}
  */
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 52bf5dcd33..ac267f0df6 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,37 @@ static int decode_receive_frame_internal(AVCodecContext 
*avctx, AVFrame *frame)
 return ret;
 }
 
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame, 
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = av_frame_get_buffer2(frame, 0);
+
+if (ret >= 0)
+ret = ff_frame_put_subtitle(frame, &subtitle);
+
+frame->pkt_dts = avpkt->dts;
+}
+
+avsubtitle_free(&subtitle);
+
+return ret;
+}
+
 int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const 
AVPacket *avpkt)
 {
 AVCodecInternal *avci = avctx->internal;
@@ -590,6 +621,9 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext 
*avctx, const AVPacke
 if (avpkt && !avpkt->size && avpkt->data)
 return AVERROR(EINVAL);
 
+if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE)
+return decode_subtitle_shim(a

[FFmpeg-devel] [PATCH v22 02/23] avutil/frame: Prepare AVFrame for subtitle handling

[Soft Works]

Root commit for adding subtitle filtering capabilities.
In detail:

- Add type (AVMediaType) field to AVFrame
  Replaces previous way of distinction which was based on checking
  width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
  allocation logic. Cannot and must not be used interchangeably, hence
  the new struct

Signed-off-by: softworkz 
---
 libavutil/Makefile |   1 +
 libavutil/frame.c  | 211 -
 libavutil/frame.h  |  78 -
 libavutil/subfmt.c |  50 +++
 libavutil/subfmt.h |  48 +++
 5 files changed, 364 insertions(+), 24 deletions(-)
 create mode 100644 libavutil/subfmt.c

diff --git a/libavutil/Makefile b/libavutil/Makefile
index c7843db1e4..7e79936876 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o  \
stereo3d.o   \
+   subfmt.o \
threadmessage.o  \
time.o   \
timecode.o   \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 0912ad9131..62d29ae7d6 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
 #include "imgutils.h"
 #include "mem.h"
 #include "samplefmt.h"
+#include "subfmt.h"
 #include "hwcontext.h"
 
 #define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
 return name[val];
 }
 #endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int 
copy_data);
+
 static void get_frame_defaults(AVFrame *frame)
 {
 memset(frame, 0, sizeof(*frame));
@@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame)
 frame->colorspace  = AVCOL_SPC_UNSPECIFIED;
 frame->color_range = AVCOL_RANGE_UNSPECIFIED;
 frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags   = 0;
+frame->subtitle_start_time = 0;
+frame->subtitle_end_time   = 0;
+frame->num_subtitle_areas  = 0;
+frame->subtitle_areas  = NULL;
+frame->subtitle_pts= 0;
+frame->subtitle_header = NULL;
 }
 
 static void free_side_data(AVFrameSideData **ptr_sd)
@@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
 
 }
 
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
 int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout || 
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
 {
 if (frame->format < 0)
 return AVERROR(EINVAL);
 
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
 return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout || 
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
 return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+return get_subtitle_buffer(frame);
+default:
+return AVERROR(EINVAL);
+}
 }
 
 static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy)
 {
 int ret, i;
 
+dst->type   = src->type;
 dst->key_frame  = src->key_frame;
 dst->pict_type  = src->pict_type;
 dst->sample_aspect_ratio= src->sample_aspect_ratio;
@@ -288,6 +329,12 @@ static int frame_copy_props(AVFrame *dst, const AVFrame 
*src, int force_copy)
 dst->colorspace = src->colorspace;
 dst->color_range= src->color_range;
 dst->chroma_location= src->chroma_location;
+dst->subtitle_start_time= src->subtitle_start_time;
+dst->subtitle_end_time  = src->subtitle_end_time;
+dst->subtitle_pts   =

[FFmpeg-devel] [PATCH v22 01/23] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values

[Soft Works]

Signed-off-by: softworkz 
---
 libavcodec/avcodec.h | 19 +
 libavutil/Makefile   |  1 +
 libavutil/subfmt.h   | 66 
 libavutil/version.h  |  1 +
 4 files changed, 69 insertions(+), 18 deletions(-)
 create mode 100644 libavutil/subfmt.h

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..b05c12e47e 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
 #include "libavutil/frame.h"
 #include "libavutil/log.h"
 #include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
 #include "libavutil/rational.h"
 
 #include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
  * @}
  */
 
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
 #define AV_SUBTITLE_FLAG_FORCED 0x0001
 
 typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index 529046dbc8..c7843db1e4 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h   
  \
   sha512.h  \
   spherical.h   \
   stereo3d.h\
+  subfmt.h  \
   threadmessage.h   \
   time.h\
   timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..88078a0d14
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,66 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0,  ///< Deprecated, use AV_SUBTITLE_FMT_NONE 
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP 
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2,  ///< Deprecated, use AV_SUBTITLE_FMT_TEXT 
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#ifdef FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3,   ///< Deprecated, use AV_SUBTITLE_FMT_ASS 
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE 
THIS if you want to link with shared libav* because the number of formats might 
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 017fc277a6..c6cd0b0d79 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -109,6 +109,7 @@
 #define FF_API_DECLARE_ALIGNED  (LIBAVUTIL_VERSION_MAJOR < 58)
 #define FF_API_COLORSPACE_NAME  (LIBAVUTIL_VERSION_MAJOR < 58)
 #define FF_API_AV_MALLOCZ_ARRAY (LIBAVUTIL_VERSION_MAJOR < 58)
+#define FF_API_OLD_SUBTITLES(LIBAVUTIL_VERSION_MAJOR < 58)
 
 /**
  * @}
-- 
2.30.2.windows.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmp

[FFmpeg-devel] [PATCH v22 00/23] Subtitle Filtering

[Soft Works]

New in V22

- Two new commits, implementing parsing of hard-space ASS tags (\h}
  The ccaption_dec decoder emits a lot of \h tags via splitcc 
  which the encoders (all using ass_split) didn't handle properly
- Combined APIchanges and version bumps into a single commit at the end
- Moved new fields to end of AVFrame
- Changed version bumps as suggested
  => no more major bumps now!
  
New in V21

- Rebased. Patchset was broken again.
- Don't declare AVFrame.subtitle_pts as deprecated and clarify doc text
- Remove AVFilter.subs_list and AVFilter.sub_fmt from APIchanges
- Change include in formats.c

New in V20

- Rebased. V19 didn't apply cleanly anymore

New in V19

- Document API changes in all relevant commits
- Move enum AVSubtitleType: Improved commit message
- Put deprecated enum values under removal guards
- Document AV_SUBTITLE_FMT_NB
- Fixed all trailing whitespace
- splitcc: Add scatter_realtime_output option
- splitcc: Emit empty subtitle frames instead of repeating
- New commit: Replace deprecated enum values

New in V18

- rebased to latest head (to fix merge conflict in allfilters.c)
- textmod: Removed two trailing blanks from sf_textmod.c
- xsubdec: fix xsubdec regression found by Michael; packet size estimation 
  as flawed
- splitcc: Fix uninitialzed locals
- splitcc: Do not propagate hwcontext on secondary output
- testsub2video: fix output timeing, re-add mutex locking and implement
  render_latest_only parameter

New in V17

- Applied almost all suggestions that were made (thanks everybody!)
- Split the initial commit, no longer moving the legacy structs (AVSubtitle..)
- Use the regular encoding API for subtitles
- Updated subtitle encoders to handle packet allocation
- Removed compatibility layer for accessing legacy encoders
- Updated the compatibility implementation for the legacy subtitle encode api
- Reordered commits and merged ffmpeg.c changes into a single commit

New in V16

- Added missing reference to subfmt.h iun libzvbi-teletextdec.c
- Fixed Fate error in patch 15/16
- Removed all trsiling whitespace in tilers.texi

New in V15

- Rebased to upstream changes 
- avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide 
  compatibility shim for legacy api
- fftools/ffmpeg: Use new frame-based subtitle encoding API
- AVSubtitleArea: copy flags field, make params const
- graphicsubs2text: Don't emit duplicate frames
- graphicsubs2text: Combined OCR output into a single AVSubtitleArea
  (I have a prototype for detecting text colors and positions, but it's not 
  ready at this point)
- splitcc: cleanup local subtitle_header ref
- stripstyles: add parameter for ass layer selection
- avcodec/subtitles: deferred loading of ass header for text subtitle encoders
- verified all example command lines in the docs are working, added somre more

Kind regards,
softworkz

softworkz (23):
  avcodec,avutil: Move enum AVSubtitleType to avutil, add new and
deprecate old values
  avutil/frame: Prepare AVFrame for subtitle handling
  avcodec/subtitles: Introduce new frame-based subtitle decoding API
  avfilter/subtitles: Update vf_subtitles to use new decoding api
  avcodec,avutil: Move ass helper functions to avutil as avpriv_ and
extend ass dialog parsing
  avcodec/subtitles: Migrate subtitle encoders to frame-based API and
provide a compatibility shim for the legacy api
  avcodec/subtitles: Replace deprecated enum values
  fftools/play,probe: Adjust for subtitle changes
  avfilter/subtitles: Add subtitles.c for subtitle frame allocation
  avfilter/avfilter: Handle subtitle frames
  avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
  avfilter/overlaygraphicsubs: Add overlaygraphicsubs and
graphicsub2video filters
  fftools/ffmpeg: Replace sub2video with subtitle frame filtering and
use new frame-based subtitle encoding API
  avfilter/avfilter: Fix hardcoded input index
  avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video
filters
  avfilter/textmod: Add textmod, censor and show_speaker filters
  avfilter/stripstyles: Add stripstyles filter
  avfilter/splitcc: Add splitcc filter for closed caption handling
  avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
  avfilter/subscale: Add filter for scaling and/or re-arranging
graphical subtitles
  doc/APIchanges: update for subtitle filtering changes
  avutil/ass_split: Add parsing of hard-space tags (\h)
  avcodec/webvttenc: convert hard-space tags to  

 configure |   7 +-
 doc/APIchanges|  23 +
 doc/filters.texi  | 756 +++
 fftools/ffmpeg.c  | 584 ++--
 fftools/ffmpeg.h  |  15 +-
 fftools/ffmpeg_filter.c   | 217 +++--
 fftools/ffmpeg_hw.c   |   2 +-
 fftools/ffmpeg_opt.c  |   3 +-
 fftools/ffplay.c  | 102 +-
 

Re: [FFmpeg-devel] [PATCH v2] libswresample/swresamplec: Err num(negative-size) was used as a function parameter

[Michael Niedermayer]

On Wed, Dec 08, 2021 at 11:17:13AM +0800, Yu Yang wrote:
> If cannot allocate memory, ERROR(ENOMEM) '-12' as a parameter will be 
> constantly being returned.
> When run resample() firstly, negative size param would cause buffer-overflow 
> and SEGV in swri_rematrix(). 
> When run swri_rematrix() firstly, resample() would not cause error but Err 
> num as a wrong parameter passing.
> Err num should be returned immediately. And remove assert to ensure the 
> return of the error code.
> 
> coredump info:
> #0 0x499517 in posix_memalign (/home/r1/ffmpeg/ffmpeg_4.4.1+0x499517)
> #1 0x6c1f0b4 in av_malloc 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:86:9
> #2 0x6c208fe in av_mallocz 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:239:17
> #3 0x6c207ad in av_mallocz_array 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavutil/mem.c:195:12
> #4 0x654b2e5 in swri_realloc_audio 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:418:14
> #5 0x654f9a1 in swr_convert_internal 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:601:17
> #6 0x654d2c0 in swr_convert 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libswresample/swresample.c:766:19
> #7 0x186cf56 in flush_frame 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/af_aresample.c:251:13
> #8 0x186a454 in request_frame 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/af_aresample.c:288:20
> #9 0x787d9c in ff_request_frame_to_filter 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:459:15
> #10 0x7877f1 in forward_status_change 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1257:19
> #11 0x77ed7e in ff_filter_activate_default 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1288:20
> #12 0x77e4e1 in ff_filter_activate 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfilter.c:1441:11
> #13 0x793b3f in ff_filter_graph_run_once 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfiltergraph.c:1403:12
> #14 0x7a7bee in get_frame_internal 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/buffersink.c:131:19
> #15 0x7a7287 in av_buffersink_get_frame_flags 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/buffersink.c:142:12
> #16 0x792888 in avfilter_graph_request_oldest 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/libavfilter/avfiltergraph.c:1356:17
> #17 0x5d07df in transcode_from_filter 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4639:11
> #18 0x59e557 in transcode_step 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4729:20
> #19 0x593970 in transcode 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:4805:15
> #20 0x58f7a4 in main 
> /home/r1/ffmpeg/ffmpeg-4.4.1/build/src/fftools/ffmpeg.c:5010:9
> #21 0x7f6fd2dee0b2 in __libc_start_main 
> /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
> 
> SUMMARY: AddressSanitizer: negative-size-param 
> (/home/r1/ffmpeg/ffmpeg_4.4.1+0x497e67) in __asan_memcpy
> 
> Reported-by: TOTE Robot 
> Signed-off-by: Yu Yang 
> ---
>  libswresample/swresample.c | 8 +---
>  1 file changed, 5 insertions(+), 3 deletions(-)

will apply

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is a danger to trust the dream we wish for rather than
the science we have, -- Dr. Kenneth Brown


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of async_thread_routine

[Guo, Yejun]

-Original Message-
From: ffmpeg-devel  On Behalf Of 
lance.lmw...@gmail.com
Sent: 2021年12月9日 9:20
To: ffmpeg-devel@ffmpeg.org
Cc: Limin Wang 
Subject: [FFmpeg-devel] [PATCH 3/5] avfilter/dnn: fix the return value of 
async_thread_routine

From: Limin Wang 

Signed-off-by: Limin Wang 
---
 libavfilter/dnn/dnn_backend_common.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/libavfilter/dnn/dnn_backend_common.c 
b/libavfilter/dnn/dnn_backend_common.c
index 6a9c4cc..8c020e5 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -83,10 +83,13 @@ static void *async_thread_routine(void *args)
 void *request = async_module->args;
 
 if (async_module->start_inference(request) != DNN_SUCCESS) {
-return DNN_ASYNC_FAIL;
+pthread_exit((void*)DNN_ASYNC_FAIL);
+return NULL;

Could you share the reason for this change?
From man pthread_exit:
Performing a return from the start function of any thread other than the main  
thread  results  in  an  implicit  call  to
pthread_exit(), using the function's return value as the thread's exit status.

 }
 async_module->callback(request);
-return DNN_ASYNC_SUCCESS;
+
+pthread_exit((void*)DNN_ASYNC_SUCCESS);
+return NULL;
 }
 
 DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
-- 
1.8.3.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 3/4] avfilter/gblur_vulkan: fix incorrect semantics

[Wu Jianhua]

Lynne:
Sent: 2021年12月9日 19:26
To: FFmpeg development discussions and patches
Subject: Re: [FFmpeg-devel] [PATCH v2 3/4] avfilter/gblur_vulkan: fix incorrect 
semantics

9 Dec 2021, 10:36 by jianhua...@intel.com:

>> The input and output are arrays of images, so it's better to use the plural
>> to align them to the variable name of VkDescriptorImageInfo.
>>
>> Signed-off-by: Wu Jianhua 
>> ---
>>  libavfilter/vf_gblur_vulkan.c | 60 +--
>>  1 file changed, 30 insertions(+), 30 deletions(-)
>>
>> diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c
>> index a2e33d1c90..2dbbbd0965 100644
>> --- a/libavfilter/vf_gblur_vulkan.c
>> +++ b/libavfilter/vf_gblur_vulkan.c
>> @@ -50,31 +50,31 @@ typedef struct GBlurVulkanContext {
>>  } GBlurVulkanContext;
>>
>
> Not going to apply either patch, image[] looks
> perfectly fine to me, and you didn't need to reindent the entire
> shader kernel either.
>

Alright.  I’m really like using the plural for arrays and indentation to make 
them more
clear and concise, but it’s okay to me to drop them if you really don’t like 
them.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[James Almer]

On 12/9/2021 8:21 AM, Lynne wrote:

9 Dec 2021, 11:14 by an...@khirnov.net:


Quoting Lynne (2021-12-08 13:53:52)


That's not a goal, it's anti-goal, and a cause for hysterical raisin
picking in the future.
Having an instantly debuggable structure rather than one that



Instantly debuggable structure?

Again: very very little code needs to store actual AVChannels. In the
submitted tree this is just af_join. So the gain from adding an offset
would simplify very little code, while complicating all the callers
dealing with masks, of which there are quite many.



It would make debugging much less hazardous,
and any code that deals with masks uses the existing
#defines. And even if it dealt directly, it's an offset that
can be #defined so users could directly map wavetableext
flags with a shift.


You're asking to make code uglier and harder to use, and to stray away 
from decades old standard mask conventions across software, for some 
supposed easier time debugging.

I'm with Anton and Hendrik, and i don't agree with this approach at all.


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/4] avfilter: add a transpose_vulkan filter

[Wu Jianhua]

Lynne:
Sent: 2021年12月9日 19:17
To: FFmpeg development discussions and patches
Subject: Re: [FFmpeg-devel] [PATCH v2 1/4] avfilter: add a transpose_vulkan 
filter

9 Dec 2021, 10:36 by jianhua...@intel.com:

>> The following command is on how to apply transpose_vulkan filter:
>> ffmpeg -init_hw_device vulkan -i input.264 -vf \
>> hwupload=extra_hw_frames=16,transpose_vulkan,hwdownload,format=yuv420p 
>> output.264
>>
>> Signed-off-by: Wu Jianhua 
>> ---
>>  configure |   1 +
>>  libavfilter/Makefile  |   1 +
>>  libavfilter/allfilters.c  |   1 +
>>  libavfilter/vf_transpose_vulkan.c | 316 ++
>>  4 files changed, 319 insertions(+)
>>  create mode 100644 libavfilter/vf_transpose_vulkan.c
>>
>
> Could you make it match what the software transpose filter does
> by default, including the options?
>
Sure I do. The commit has a same effect as what software did. And the options 
will be
introduced when a different effects implemented in the next few days or weeks. 
I’m not
sure.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 3/4] avfilter/gblur_vulkan: fix incorrect semantics

[Lynne]

9 Dec 2021, 10:36 by jianhua...@intel.com:

> The input and output are arrays of images, so it's better to use the plural
> to align them to the variable name of VkDescriptorImageInfo.
>
> Signed-off-by: Wu Jianhua 
> ---
>  libavfilter/vf_gblur_vulkan.c | 60 +--
>  1 file changed, 30 insertions(+), 30 deletions(-)
>
> diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c
> index a2e33d1c90..2dbbbd0965 100644
> --- a/libavfilter/vf_gblur_vulkan.c
> +++ b/libavfilter/vf_gblur_vulkan.c
> @@ -50,31 +50,31 @@ typedef struct GBlurVulkanContext {
>  } GBlurVulkanContext;
>  
>  static const char gblur_horizontal[] = {
> -C(0, void gblur(const ivec2 pos, const int index)
>   )
> -C(0, {   
>   )
> -C(1, vec4 sum = texture(input_image[index], pos) * kernel[0];
>   )
> -C(0, 
>   )
> -C(1, for(int i = 1; i < kernel.length(); i++) {  
>   )
> -C(2, sum += texture(input_image[index], pos + vec2(i, 0.0)) * 
> kernel[i];   )
> -C(2, sum += texture(input_image[index], pos - vec2(i, 0.0)) * 
> kernel[i];   )
> -C(1, }   
>   )
> -C(0, 
>   )
> -C(1, imageStore(output_image[index], pos, sum);  
>   )
> -C(0, }   
>   )
> +C(0, void gblur(const ivec2 pos, const int index)
>)
> +C(0, {   
>)
> +C(1, vec4 sum = texture(input_images[index], pos) * kernel[0];   
>)
> +C(0, 
>)
> +C(1, for (int i = 1; i < kernel.length(); i++) { 
>)
> +C(2, sum += texture(input_images[index], pos + vec2(i, 0.0)) * 
> kernel[i];   )
> +C(2, sum += texture(input_images[index], pos - vec2(i, 0.0)) * 
> kernel[i];   )
> +C(1, }   
>)
> +C(0, 
>)
> +C(1, imageStore(output_images[index], pos, sum); 
>)
> +C(0, }   
>)
>  };
>  
>  static const char gblur_vertical[] = {
> -C(0, void gblur(const ivec2 pos, const int index)
>   )
> -C(0, {   
>   )
> -C(1, vec4 sum = texture(input_image[index], pos) * kernel[0];
>   )
> -C(0, 
>   )
> -C(1, for(int i = 1; i < kernel.length(); i++) {  
>   )
> -C(2, sum += texture(input_image[index], pos + vec2(0.0, i)) * 
> kernel[i];   )
> -C(2, sum += texture(input_image[index], pos - vec2(0.0, i)) * 
> kernel[i];   )
> -C(1, }   
>   )
> -C(0, 
>   )
> -C(1, imageStore(output_image[index], pos, sum);  
>   )
> -C(0, }   
>   )
> +C(0, void gblur(const ivec2 pos, const int index)
>)
> +C(0, {   
>)
> +C(1, vec4 sum = texture(input_images[index], pos) * kernel[0];   
>)
> +C(0, 
>)
> +C(1, for (int i = 1; i < kernel.length(); i++) { 
>)
> +C(2, sum += texture(input_images[index], pos + vec2(0.0, i)) * 
> kernel[i];   )
> +C(2, sum += texture(input_images[index], pos - vec2(0.0, i)) * 
> kernel[i];   )
> +C(1, }   
>)
> +C(0, 
>)
> +C(1, imageStore(output_images[index], pos, sum); 
>)
> +C(0, }   
>)
>  };
>  
>  static inline float gaussian(float sigma, float x)
> @@ -13

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Lynne]

9 Dec 2021, 11:14 by an...@khirnov.net:

> Quoting Lynne (2021-12-08 13:53:52)
>
>> That's not a goal, it's anti-goal, and a cause for hysterical raisin
>> picking in the future.
>> Having an instantly debuggable structure rather than one that
>>
>
> Instantly debuggable structure?
>
> Again: very very little code needs to store actual AVChannels. In the
> submitted tree this is just af_join. So the gain from adding an offset
> would simplify very little code, while complicating all the callers
> dealing with masks, of which there are quite many.
>

It would make debugging much less hazardous,
and any code that deals with masks uses the existing
#defines. And even if it dealt directly, it's an offset that
can be #defined so users could directly map wavetableext
flags with a shift.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v2 1/4] avfilter: add a transpose_vulkan filter

[Lynne]

9 Dec 2021, 10:36 by jianhua...@intel.com:

> The following command is on how to apply transpose_vulkan filter:
> ffmpeg -init_hw_device vulkan -i input.264 -vf \
> hwupload=extra_hw_frames=16,transpose_vulkan,hwdownload,format=yuv420p 
> output.264
>
> Signed-off-by: Wu Jianhua 
> ---
>  configure |   1 +
>  libavfilter/Makefile  |   1 +
>  libavfilter/allfilters.c  |   1 +
>  libavfilter/vf_transpose_vulkan.c | 316 ++
>  4 files changed, 319 insertions(+)
>  create mode 100644 libavfilter/vf_transpose_vulkan.c
>
> diff --git a/configure b/configure
> index a98a18abaa..12cb49e877 100755
> --- a/configure
> +++ b/configure
> @@ -3718,6 +3718,7 @@ tonemap_vaapi_filter_deps="vaapi 
> VAProcFilterParameterBufferHDRToneMapping"
>  tonemap_opencl_filter_deps="opencl const_nan"
>  transpose_opencl_filter_deps="opencl"
>  transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
> +transpose_vulkan_filter_deps="vulkan spirv_compiler"
>  unsharp_opencl_filter_deps="opencl"
>  uspp_filter_deps="gpl avcodec"
>  vaguedenoiser_filter_deps="gpl"
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index c8082c4a2f..8744cc3c63 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -483,6 +483,7 @@ OBJS-$(CONFIG_TRANSPOSE_FILTER)  += 
> vf_transpose.o
>  OBJS-$(CONFIG_TRANSPOSE_NPP_FILTER)  += vf_transpose_npp.o
>  OBJS-$(CONFIG_TRANSPOSE_OPENCL_FILTER)   += vf_transpose_opencl.o 
> opencl.o opencl/transpose.o
>  OBJS-$(CONFIG_TRANSPOSE_VAAPI_FILTER)+= vf_transpose_vaapi.o 
> vaapi_vpp.o
> +OBJS-$(CONFIG_TRANSPOSE_VULKAN_FILTER)   += vf_transpose_vulkan.o 
> vulkan.o vulkan_filter.o
>  OBJS-$(CONFIG_TRIM_FILTER)   += trim.o
>  OBJS-$(CONFIG_UNPREMULTIPLY_FILTER)  += vf_premultiply.o framesync.o
>  OBJS-$(CONFIG_UNSHARP_FILTER)+= vf_unsharp.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index b1af2cbcc8..9e16b4e71e 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -462,6 +462,7 @@ extern const AVFilter ff_vf_transpose;
>  extern const AVFilter ff_vf_transpose_npp;
>  extern const AVFilter ff_vf_transpose_opencl;
>  extern const AVFilter ff_vf_transpose_vaapi;
> +extern const AVFilter ff_vf_transpose_vulkan;
>  extern const AVFilter ff_vf_trim;
>  extern const AVFilter ff_vf_unpremultiply;
>  extern const AVFilter ff_vf_unsharp;
> diff --git a/libavfilter/vf_transpose_vulkan.c 
> b/libavfilter/vf_transpose_vulkan.c
> new file mode 100644
> index 00..c9bae413c3
> --- /dev/null
> +++ b/libavfilter/vf_transpose_vulkan.c
> @@ -0,0 +1,316 @@
> +/*
> + * copyright (c) 2021 Wu Jianhua 
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 
> USA
> + */
> +
> +#include "libavutil/random_seed.h"
> +#include "libavutil/opt.h"
> +#include "vulkan_filter.h"
> +#include "internal.h"
> +
> +#define CGS 32
> +
> +typedef struct TransposeVulkanContext {
> +FFVulkanContext vkctx;
> +FFVkQueueFamilyCtx qf;
> +FFVkExecContext *exec;
> +FFVulkanPipeline *pl;
> +
> +VkDescriptorImageInfo input_images[3];
> +VkDescriptorImageInfo output_images[3];
> +
> +int initialized;
> +} TransposeVulkanContext;
> +
> +static av_cold int init_filter(AVFilterContext *ctx, AVFrame *in)
> +{
> +int err = 0;
> +FFVkSPIRVShader *shd;
> +TransposeVulkanContext *s = ctx->priv;
> +FFVulkanContext *vkctx = &s->vkctx;
> +const int planes = av_pix_fmt_count_planes(s->vkctx.output_format);
> +
> +FFVulkanDescriptorSetBinding image_descs[] = {
> +{
> +.name   = "input_images",
> +.type   = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,
> +.dimensions = 2,
> +.elems  = planes,
> +.stages = VK_SHADER_STAGE_COMPUTE_BIT,
> +.updater= s->input_images,
> +},
> +{
> +.name   = "output_images",
> +.type   = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,
> +.mem_layout = ff_vk_shader_rep_fmt(s->vkctx.output_format),
> +.mem_quali  = "writeonly",
> +.dimensions = 2,
> +.elems  = pla

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Nicolas George]

Anton Khirnov (12021-12-09):
> It is not possible for them to behave "like that", because our current
> channel layout API does not support duplicated channels at all.

They behave like in the sense they output channels from several sources
in a single stream of AVFrame. They could not properly label it with a
layout. The new API should allow to enhance that, not break it utterly.

> I disagree. Technical limitations that were overcome 10 years ago should
> not guide new API design.

In the case of amerge, it was not a technical limitation, merging
several streams into one so that they can be handled by single-stream
filters is 100% part of the design. I suspect devices that capture
several independent channels are designed that way intentionally too,
possibly to reduce the risk of desynchronization.

> If you insist then we can have a TC vote about this.

Please, the TC is for when discussion has failed. Let us see what
arguments other developers bring to the discussion first.

I observe that about the ability to attach an arbitrary string label to
any channel, Lynne was with me for independent reasons.

And of course, if it comes to the TC, since you are a member yourself, I
expect you to recuse yourself from the proceedings on this question, as
you would be judge and party.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 001/279] Add a new channel layout API

[Anton Khirnov]

Quoting Lynne (2021-12-08 13:53:52)
> That's not a goal, it's anti-goal, and a cause for hysterical raisin
> picking in the future.
> Having an instantly debuggable structure rather than one that

Instantly debuggable structure?

Again: very very little code needs to store actual AVChannels. In the
submitted tree this is just af_join. So the gain from adding an offset
would simplify very little code, while complicating all the callers
dealing with masks, of which there are quite many.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API

[Anton Khirnov]

Quoting Nicolas George (2021-12-08 16:08:54)
> Anton Khirnov (12021-12-08):
> We get to decide what is a valid use case and what is not. And in this
> case, since the devices and filter I quoted already behave like that,

It is not possible for them to behave "like that", because our current
channel layout API does not support duplicated channels at all.

> I posit that it IS a valid use case. Therefore, this new API must be
> capable of handling them.

I disagree. Technical limitations that were overcome 10 years ago should
not guide new API design.

If you insist then we can have a TC vote about this.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v2 4/4] avfilter/flip_vulkan: fix incorrect semantics

[Wu Jianhua]

The input and output are arrays of images, so it's better to use the plural
to align them to the variable name of VkDescriptorImageInfo.

Signed-off-by: Wu Jianhua 
---
 libavfilter/vf_flip_vulkan.c | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/libavfilter/vf_flip_vulkan.c b/libavfilter/vf_flip_vulkan.c
index 0223786ef1..6a6709e79b 100644
--- a/libavfilter/vf_flip_vulkan.c
+++ b/libavfilter/vf_flip_vulkan.c
@@ -52,7 +52,7 @@ static av_cold int init_filter(AVFilterContext *ctx, AVFrame 
*in, enum FlipType
 
 FFVulkanDescriptorSetBinding image_descs[] = {
 {
-.name   = "input_image",
+.name   = "input_images",
 .type   = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,
 .dimensions = 2,
 .elems  = planes,
@@ -60,7 +60,7 @@ static av_cold int init_filter(AVFilterContext *ctx, AVFrame 
*in, enum FlipType
 .updater= s->input_images,
 },
 {
-.name   = "output_image",
+.name   = "output_images",
 .type   = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,
 .mem_layout = ff_vk_shader_rep_fmt(s->vkctx.output_format),
 .mem_quali  = "writeonly",
@@ -89,33 +89,33 @@ static av_cold int init_filter(AVFilterContext *ctx, 
AVFrame *in, enum FlipType
 ff_vk_set_compute_shader_sizes(shd, (int [3]){ CGS, 1, 1 });
 RET(ff_vk_add_descriptor_set(vkctx, s->pl, shd, image_descs, 
FF_ARRAY_ELEMS(image_descs), 0));
 
-GLSLC(0, void main()   
 );
-GLSLC(0, { 
 );
-GLSLC(1, ivec2 size;   
 );
-GLSLC(1, const ivec2 pos = ivec2(gl_GlobalInvocationID.xy);
 );
+GLSLC(0, void main()   
  );
+GLSLC(0, { 
  );
+GLSLC(1, ivec2 size;   
  );
+GLSLC(1, const ivec2 pos = ivec2(gl_GlobalInvocationID.xy);
  );
 for (int i = 0; i < planes; i++) {
-GLSLC(0,   
 );
-GLSLF(1, size = imageSize(output_image[%i]);   
   ,i);
-GLSLC(1, if (IS_WITHIN(pos, size)) {   
 );
+GLSLC(0,   
  );
+GLSLF(1, size = imageSize(output_images[%i]);  
,i);
+GLSLC(1, if (IS_WITHIN(pos, size)) {   
  );
 switch (type)
 {
 case FLIP_HORIZONTAL:
-GLSLF(2, vec4 res = texture(input_image[%i], ivec2(size.x - 
pos.x, pos.y));   ,i);
+GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.x - 
pos.x, pos.y));   ,i);
 break;
 case FLIP_VERTICAL:
-GLSLF(2, vec4 res = texture(input_image[%i], ivec2(pos.x, 
size.y - pos.y));   ,i);
+GLSLF(2, vec4 res = texture(input_images[%i], ivec2(pos.x, 
size.y - pos.y));   ,i);
 break;
 case FLIP_BOTH:
-GLSLF(2, vec4 res = texture(input_image[%i], ivec2(size.xy - 
pos.xy));, i);
+GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.xy - 
pos.xy));,i);
 break;
 default:
-GLSLF(2, vec4 res = texture(input_image[%i], pos); 
   ,i);
+GLSLF(2, vec4 res = texture(input_images[%i], pos);
,i);
 break;
 }
-GLSLF(2, imageStore(output_image[%i], pos, res);   
   ,i);
-GLSLC(1, } 
 );
+GLSLF(2, imageStore(output_images[%i], pos, res);  
,i);
+GLSLC(1, } 
  );
 }
-GLSLC(0, } 
 );
+GLSLC(0, } 
  );
 
 RET(ff_vk_compile_shader(vkctx, shd, "main"));
 RET(ff_vk_init_pipeline_layout(vkctx, s->pl));
-- 
2.25.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [PATCH v2 3/4] avfilter/gblur_vulkan: fix incorrect semantics

[Wu Jianhua]

The input and output are arrays of images, so it's better to use the plural
to align them to the variable name of VkDescriptorImageInfo.

Signed-off-by: Wu Jianhua 
---
 libavfilter/vf_gblur_vulkan.c | 60 +--
 1 file changed, 30 insertions(+), 30 deletions(-)

diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c
index a2e33d1c90..2dbbbd0965 100644
--- a/libavfilter/vf_gblur_vulkan.c
+++ b/libavfilter/vf_gblur_vulkan.c
@@ -50,31 +50,31 @@ typedef struct GBlurVulkanContext {
 } GBlurVulkanContext;
 
 static const char gblur_horizontal[] = {
-C(0, void gblur(const ivec2 pos, const int index)  
)
-C(0, { 
)
-C(1, vec4 sum = texture(input_image[index], pos) * kernel[0];  
)
-C(0,   
)
-C(1, for(int i = 1; i < kernel.length(); i++) {
)
-C(2, sum += texture(input_image[index], pos + vec2(i, 0.0)) * 
kernel[i];   )
-C(2, sum += texture(input_image[index], pos - vec2(i, 0.0)) * 
kernel[i];   )
-C(1, } 
)
-C(0,   
)
-C(1, imageStore(output_image[index], pos, sum);
)
-C(0, } 
)
+C(0, void gblur(const ivec2 pos, const int index)  
 )
+C(0, { 
 )
+C(1, vec4 sum = texture(input_images[index], pos) * kernel[0]; 
 )
+C(0,   
 )
+C(1, for (int i = 1; i < kernel.length(); i++) {   
 )
+C(2, sum += texture(input_images[index], pos + vec2(i, 0.0)) * 
kernel[i];   )
+C(2, sum += texture(input_images[index], pos - vec2(i, 0.0)) * 
kernel[i];   )
+C(1, } 
 )
+C(0,   
 )
+C(1, imageStore(output_images[index], pos, sum);   
 )
+C(0, } 
 )
 };
 
 static const char gblur_vertical[] = {
-C(0, void gblur(const ivec2 pos, const int index)  
)
-C(0, { 
)
-C(1, vec4 sum = texture(input_image[index], pos) * kernel[0];  
)
-C(0,   
)
-C(1, for(int i = 1; i < kernel.length(); i++) {
)
-C(2, sum += texture(input_image[index], pos + vec2(0.0, i)) * 
kernel[i];   )
-C(2, sum += texture(input_image[index], pos - vec2(0.0, i)) * 
kernel[i];   )
-C(1, } 
)
-C(0,   
)
-C(1, imageStore(output_image[index], pos, sum);
)
-C(0, } 
)
+C(0, void gblur(const ivec2 pos, const int index)  
 )
+C(0, { 
 )
+C(1, vec4 sum = texture(input_images[index], pos) * kernel[0]; 
 )
+C(0,   
 )
+C(1, for (int i = 1; i < kernel.length(); i++) {   
 )
+C(2, sum += texture(input_images[index], pos + vec2(0.0, i)) * 
kernel[i];   )
+C(2, sum += texture(input_images[index], pos - vec2(0.0, i)) * 
kernel[i];   )
+C(1, } 
 )
+C(0,   
 )
+C(1, imageStore(output_images[index], pos, sum);   
 )
+C(0, } 
 )
 };
 
 static inline float gaussian(float sigma, float x)
@@ -133,14 +133,14 @@ static av_cold int init_filter(AVFilterContext *ctx, 
AVFrame *in)
 
 FFVulkanDescriptorSetBinding image_descs[] = {
 {
-.name   = "input_image",
+.name   = "input_images",
 .type   = VK_D

[FFmpeg-devel] [PATCH v2 1/4] avfilter: add a transpose_vulkan filter

[Wu Jianhua]

The following command is on how to apply transpose_vulkan filter:
ffmpeg -init_hw_device vulkan -i input.264 -vf \
hwupload=extra_hw_frames=16,transpose_vulkan,hwdownload,format=yuv420p 
output.264

Signed-off-by: Wu Jianhua 
---
 configure |   1 +
 libavfilter/Makefile  |   1 +
 libavfilter/allfilters.c  |   1 +
 libavfilter/vf_transpose_vulkan.c | 316 ++
 4 files changed, 319 insertions(+)
 create mode 100644 libavfilter/vf_transpose_vulkan.c

diff --git a/configure b/configure
index a98a18abaa..12cb49e877 100755
--- a/configure
+++ b/configure
@@ -3718,6 +3718,7 @@ tonemap_vaapi_filter_deps="vaapi 
VAProcFilterParameterBufferHDRToneMapping"
 tonemap_opencl_filter_deps="opencl const_nan"
 transpose_opencl_filter_deps="opencl"
 transpose_vaapi_filter_deps="vaapi VAProcPipelineCaps_rotation_flags"
+transpose_vulkan_filter_deps="vulkan spirv_compiler"
 unsharp_opencl_filter_deps="opencl"
 uspp_filter_deps="gpl avcodec"
 vaguedenoiser_filter_deps="gpl"
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c8082c4a2f..8744cc3c63 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -483,6 +483,7 @@ OBJS-$(CONFIG_TRANSPOSE_FILTER)  += 
vf_transpose.o
 OBJS-$(CONFIG_TRANSPOSE_NPP_FILTER)  += vf_transpose_npp.o
 OBJS-$(CONFIG_TRANSPOSE_OPENCL_FILTER)   += vf_transpose_opencl.o opencl.o 
opencl/transpose.o
 OBJS-$(CONFIG_TRANSPOSE_VAAPI_FILTER)+= vf_transpose_vaapi.o 
vaapi_vpp.o
+OBJS-$(CONFIG_TRANSPOSE_VULKAN_FILTER)   += vf_transpose_vulkan.o vulkan.o 
vulkan_filter.o
 OBJS-$(CONFIG_TRIM_FILTER)   += trim.o
 OBJS-$(CONFIG_UNPREMULTIPLY_FILTER)  += vf_premultiply.o framesync.o
 OBJS-$(CONFIG_UNSHARP_FILTER)+= vf_unsharp.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1af2cbcc8..9e16b4e71e 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -462,6 +462,7 @@ extern const AVFilter ff_vf_transpose;
 extern const AVFilter ff_vf_transpose_npp;
 extern const AVFilter ff_vf_transpose_opencl;
 extern const AVFilter ff_vf_transpose_vaapi;
+extern const AVFilter ff_vf_transpose_vulkan;
 extern const AVFilter ff_vf_trim;
 extern const AVFilter ff_vf_unpremultiply;
 extern const AVFilter ff_vf_unsharp;
diff --git a/libavfilter/vf_transpose_vulkan.c 
b/libavfilter/vf_transpose_vulkan.c
new file mode 100644
index 00..c9bae413c3
--- /dev/null
+++ b/libavfilter/vf_transpose_vulkan.c
@@ -0,0 +1,316 @@
+/*
+ * copyright (c) 2021 Wu Jianhua 
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/random_seed.h"
+#include "libavutil/opt.h"
+#include "vulkan_filter.h"
+#include "internal.h"
+
+#define CGS 32
+
+typedef struct TransposeVulkanContext {
+FFVulkanContext vkctx;
+FFVkQueueFamilyCtx qf;
+FFVkExecContext *exec;
+FFVulkanPipeline *pl;
+
+VkDescriptorImageInfo input_images[3];
+VkDescriptorImageInfo output_images[3];
+
+int initialized;
+} TransposeVulkanContext;
+
+static av_cold int init_filter(AVFilterContext *ctx, AVFrame *in)
+{
+int err = 0;
+FFVkSPIRVShader *shd;
+TransposeVulkanContext *s = ctx->priv;
+FFVulkanContext *vkctx = &s->vkctx;
+const int planes = av_pix_fmt_count_planes(s->vkctx.output_format);
+
+FFVulkanDescriptorSetBinding image_descs[] = {
+{
+.name   = "input_images",
+.type   = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,
+.dimensions = 2,
+.elems  = planes,
+.stages = VK_SHADER_STAGE_COMPUTE_BIT,
+.updater= s->input_images,
+},
+{
+.name   = "output_images",
+.type   = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,
+.mem_layout = ff_vk_shader_rep_fmt(s->vkctx.output_format),
+.mem_quali  = "writeonly",
+.dimensions = 2,
+.elems  = planes,
+.stages = VK_SHADER_STAGE_COMPUTE_BIT,
+.updater= s->output_images,
+},
+};
+
+image_descs[0].sampler = ff_vk_init_sampler(vkctx, 1, VK_FILTER_LINEAR);
+if (!image_descs[0].sampler)
+return AVERROR_EXTERNAL;
+
+ff_vk_qf

[FFmpeg-devel] [PATCH v2 2/4] avfilter/vf_transpose: fix un-checked potential memory allocation failure

[Wu Jianhua]

Signed-off-by: Wu Jianhua 
---
 libavfilter/vf_transpose.c | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_transpose.c b/libavfilter/vf_transpose.c
index f9f0d70cd5..b964daeee3 100644
--- a/libavfilter/vf_transpose.c
+++ b/libavfilter/vf_transpose.c
@@ -328,6 +328,7 @@ static int filter_slice(AVFilterContext *ctx, void *arg, 
int jobnr,
 
 static int filter_frame(AVFilterLink *inlink, AVFrame *in)
 {
+int err = 0;
 AVFilterContext *ctx = inlink->dst;
 TransContext *s = ctx->priv;
 AVFilterLink *outlink = ctx->outputs[0];
@@ -339,10 +340,13 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
 
 out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
 if (!out) {
-av_frame_free(&in);
-return AVERROR(ENOMEM);
+err = AVERROR(ENOMEM);
+goto fail;
 }
-av_frame_copy_props(out, in);
+
+err = av_frame_copy_props(out, in);
+if (err < 0)
+goto fail;
 
 if (in->sample_aspect_ratio.num == 0) {
 out->sample_aspect_ratio = in->sample_aspect_ratio;
@@ -356,6 +360,11 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
   FFMIN(outlink->h, ff_filter_get_nb_threads(ctx)));
 av_frame_free(&in);
 return ff_filter_frame(outlink, out);
+
+fail:
+av_frame_free(&in);
+av_frame_free(&out);
+return err;
 }
 
 #define OFFSET(x) offsetof(TransContext, x)
-- 
2.25.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v3 2/2] avcodec/vpp_qsv: Copy side data from input to output frame

[Soft Works]

> -Original Message-
> From: ffmpeg-devel  On Behalf Of Xiang,
> Haihao
> Sent: Thursday, December 9, 2021 9:35 AM
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] avcodec/vpp_qsv: Copy side data
> from input to output frame
> 
> On Tue, 2021-12-07 at 12:39 +, Soft Works wrote:
> > > -Original Message-
> > > From: ffmpeg-devel  On Behalf Of Anton
> > > Khirnov
> > > Sent: Tuesday, December 7, 2021 12:51 PM
> > > To: FFmpeg development discussions and patches 
> > > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] avcodec/vpp_qsv: Copy side
> data
> > > from input to output frame
> > >
> > > Quoting Soft Works (2021-12-07 09:55:37)
> > > >
> > > >
> > > > > -Original Message-
> > > > > From: ffmpeg-devel  On Behalf Of
> Anton
> > > > > Khirnov
> > > > > Sent: Tuesday, December 7, 2021 9:04 AM
> > > > > To: ffmpeg-devel@ffmpeg.org
> > > > > Subject: Re: [FFmpeg-devel] [PATCH v3 2/2] avcodec/vpp_qsv: Copy side
> > >
> > > data
> > > > > from input to output frame
> > > > >
> > > > > Quoting Soft Works (2021-12-03 08:58:31)
> > > > > > Signed-off-by: softworkz 
> > > > > > ---
> > > > > >  libavfilter/qsvvpp.c |  5 +
> > > > > >  libavfilter/vf_overlay_qsv.c | 19 +++
> > > > > >  2 files changed, 20 insertions(+), 4 deletions(-)
> > > > > >
> > > > > > diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c
> > > > > > index d1218355c7..b291216292 100644
> > > > > > --- a/libavfilter/qsvvpp.c
> > > > > > +++ b/libavfilter/qsvvpp.c
> > > > > > @@ -849,6 +849,11 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s,
> > > > >
> > > > > AVFilterLink *inlink, AVFrame *picr
> > > > > >  return AVERROR(EAGAIN);
> > > > > >  break;
> > > > > >  }
> > > > > > +
> > > > > > +ret = av_frame_copy_side_data(out_frame->frame, in_frame-
> > > >
> > > > frame,
> > > > > 0);
> > > > > > +if (ret < 0)
> > > > > > +return ret;
> > > > > > +
> > > > > >  out_frame->frame->pts = av_rescale_q(out_frame-
> > > > > > surface.Data.TimeStamp,
> > > > > >   default_tb, outlink-
> > > > > > time_base);
> > > > > >
> > > > > > diff --git a/libavfilter/vf_overlay_qsv.c
> > >
> > > b/libavfilter/vf_overlay_qsv.c
> > > > > > index 7e76b39aa9..02518e020c 100644
> > > > > > --- a/libavfilter/vf_overlay_qsv.c
> > > > > > +++ b/libavfilter/vf_overlay_qsv.c
> > > > > > @@ -231,13 +231,24 @@ static int process_frame(FFFrameSync *fs)
> > > > > >  {
> > > > > >  AVFilterContext  *ctx = fs->parent;
> > > > > >  QSVOverlayContext  *s = fs->opaque;
> > > > > > +AVFrame   *frame0 = NULL;
> > > > > >  AVFrame*frame = NULL;
> > > > > > -int   ret = 0, i;
> > > > > > +int   ret = 0;
> > > > > >
> > > > > > -for (i = 0; i < ctx->nb_inputs; i++) {
> > > > > > +for (unsigned i = 0; i < ctx->nb_inputs; i++) {
> > > > > >  ret = ff_framesync_get_frame(fs, i, &frame, 0);
> > > > > > -if (ret == 0)
> > > > > > -ret = ff_qsvvpp_filter_frame(s->qsv, ctx->inputs[i],
> > >
> > > frame);
> > > > > > +
> > > > > > +if (ret == 0) {
> > > > > > +AVFrame *temp;
> > > > > > +
> > > > > > +if (i == 0)
> > > > > > +frame0 = frame;
> > > > > > +else
> > > > > > +ret = av_frame_copy_side_data(frame, frame0, 0);
> > > > > > +
> > > > > > +ret = ret < 0 ? ret : ff_qsvvpp_filter_frame(s->qsv,
> ctx-
> > > > > > inputs[i], frame);
> > > > >
> > > > > I don't quite understand the ownership semantics here. This function
> > > > > does not free frame, so I assume ff_qsvvpp_filter_frame() takes
> > > > > ownership of it. That would mean you're not allowed to keep a pointer
> to
> > > > > it and access it later, because it might have already been freed.
> > > >
> > > > The filter is using framesync, which is taking care of the ownership.
> > > > ff_qsvvpp_filter_frame() clones or copies the frame, depending on case.
> > > > Other than with the normal overlay filter, the frame from input0 is
> > > > not used for output. But the regular overlay filter has established the
> > > > convention that side data from input0 is being kept at the output.
> > >
> > > Okay, if you're sure that framesync guarantees the frame remaining valid
> > > then I have no objections.
> > >
> > > But note that temp is unused and should be removed.
> >
> > OK, thanks. Let's see what Haihao says, he's closest to the subject at the
> > moment.
> 
> Is it possible that the frame from input1 and the frame from input0 have the
> same type of side data ? If yes, which one will take effect in the output
> frame
> ?

You can see in the code above that I'm copying the side data from frame0 
to frame1 before submitting frame1 to qsvvpp. So, it's always the side data from
frame0 that will be used.
This achieves the same behavior as the other overlay filters

Re: [FFmpeg-devel] [PATCH] libavutil/hwcontext_qsv: clean padding when upload qsv frames

[Xiang, Haihao]

On Thu, 2021-12-02 at 15:40 +0800, Wenbin Chen wrote:
> When we upload a frame that is not padded as MSDK requires, we create a
> new AVFrame to copy data. The frame's padding data is uninitialized so
> it brings run to run problem. For example, If we run the following
> command serveral times we will get different outputs.
> 
> ffmpeg -init_hw_device qsv=qsv:hw -qsv_device /dev/dri/renderD128
> -filter_hw_device qsv -f rawvideo -s 192x200 -pix_fmt p010
> -i 192x200_P010.yuv -vf "format=nv12,hwupload=extra_hw_frames=16"
> -c:v hevc_qsv output.265
> 
> According to 
> https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#encoding-procedures
> "Note: It is the application's responsibility to fill pixels outside
> of crop window when it is smaller than frame to be encoded. Especially
> in cases when crops are not aligned to minimum coding block size (16
> for AVC, 8 for HEVC and VP9)"
> 
> I add a function to fill padding area with border pixel to fix this
> run2run problem, and also move the new AVFrame to global structure
> to reduce redundant allocation operation to increase preformance.
> 
> Signed-off-by: Wenbin Chen 
> ---
>  libavutil/hwcontext_qsv.c | 96 +--
>  1 file changed, 83 insertions(+), 13 deletions(-)
> 
> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> index 268be9f8a1..983494666b 100644
> --- a/libavutil/hwcontext_qsv.c
> +++ b/libavutil/hwcontext_qsv.c
> @@ -47,6 +47,7 @@
>  #include "pixfmt.h"
>  #include "pixdesc.h"
>  #include "time.h"
> +#include "imgutils.h"
>  
>  #define QSV_VERSION_ATLEAST(MAJOR, MINOR)   \
>  (MFX_VERSION_MAJOR > (MAJOR) || \
> @@ -90,6 +91,7 @@ typedef struct QSVFramesContext {
>  
>  mfxExtOpaqueSurfaceAlloc opaque_alloc;
>  mfxExtBuffer *ext_buffers[1];
> +AVFrame realigned_tmp_frame;
>  } QSVFramesContext;
>  
>  static const struct {
> @@ -137,6 +139,54 @@ static uint32_t qsv_get_d3d11va_bind_flags(int mem_type)
>  }
>  #endif
>  
> +static int qsv_fill_border(AVFrame *dst, const AVFrame *src)
> +{
> +const AVPixFmtDescriptor *desc;
> +int i, planes_nb = 0;
> +if (dst->format != src->format)
> +return AVERROR(EINVAL);
> +
> +desc = av_pix_fmt_desc_get(dst->format);
> +
> +for (i = 0; i < desc->nb_components; i++)
> +planes_nb = FFMAX(planes_nb, desc->comp[i].plane + 1);
> +
> +for (i = 0; i < planes_nb; i++) {
> +int sheight, dheight, y;
> +ptrdiff_t swidth = av_image_get_linesize(src->format,
> + src->width,
> + i);
> +ptrdiff_t dwidth = av_image_get_linesize(dst->format,
> + dst->width,
> + i);
> +const AVComponentDescriptor comp = desc->comp[i];
> +if (swidth < 0 || dwidth < 0) {
> +av_log(NULL, AV_LOG_ERROR, "av_image_get_linesize failed\n");
> +return AVERROR(EINVAL);
> +}
> +sheight = src->height;
> +dheight = dst->height;
> +if (i) {
> +sheight = AV_CEIL_RSHIFT(src->height, desc->log2_chroma_h);
> +dheight = AV_CEIL_RSHIFT(dst->height, desc->log2_chroma_h);
> +}
> +//fill right padding
> +for (y = 0; y < sheight; y++) {
> +void *line_ptr = dst->data[i] + y*dst->linesize[i] + swidth;
> +av_memcpy_backptr(line_ptr,
> +   comp.depth > 8 ? 2 : 1,
> +   dwidth - swidth);
> +}
> +//fill bottom padding
> +for (y = sheight; y < dheight; y++) {
> +memcpy(dst->data[i]+y*dst->linesize[i],
> +   dst->data[i]+(sheight-1)*dst->linesize[i],
> +   dwidth);
> +}
> +}
> +return 0;
> +}
> +
>  static int qsv_device_init(AVHWDeviceContext *ctx)
>  {
>  AVQSVDeviceContext *hwctx = ctx->hwctx;
> @@ -220,6 +270,7 @@ static void qsv_frames_uninit(AVHWFramesContext *ctx)
>  av_freep(&s->surface_ptrs);
>  av_freep(&s->surfaces_internal);
>  av_freep(&s->handle_pairs_internal);
> +av_frame_unref(&s->realigned_tmp_frame);
>  av_buffer_unref(&s->child_frames_ref);
>  }
>  
> @@ -1014,12 +1065,13 @@ static int qsv_transfer_data_to(AVHWFramesContext
> *ctx, AVFrame *dst,
>  QSVFramesContext   *s = ctx->internal->priv;
>  mfxFrameSurface1   in = {{ 0 }};
>  mfxFrameSurface1 *out = (mfxFrameSurface1*)dst->data[3];
> +mfxFrameInfo tmp_info;
>  
>  mfxSyncPoint sync = NULL;
>  mfxStatus err;
>  int ret = 0;
>  /* make a copy if the input is not padded as libmfx requires */
> -AVFrame tmp_frame;
> +AVFrame *tmp_frame = &s->realigned_tmp_frame;
>  const AVFrame *src_frame;
>  int realigned = 0;
>  
> @@ -1048,24 +1100,40 @@ static int qsv_transfer_data_to(AVHWFramesContext
> *ctx, AVFrame *dst,