PR #25101 opened by Romain Beauxis (toots) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25101 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25101.patch
There is currently no way to edit packet side data during a stream copy. The use that motivated this is skip samples: some demuxers mark the last samples of a stream to be discarded, and when remuxing it can be necessary to keep them, or to set different skip and discard values than the source has. More generally, it is sometimes useful to drop side data such as a display matrix or replay gain without re-encoding. This series adds a `side_data` bitstream filter with two kinds of options: - `remove` drops the side data of the listed types, joined with `+`, e.g. `side_data=remove=display_matrix+replaygain`. - `skip_samples.skip`, `skip_samples.discard`, `skip_samples.skip_reason` and `skip_samples.discard_reason` set the fields of the skip samples side data. `skip_samples.packet` selects which packets they apply to: `all` (the default), `first`, `last`, or the packets with a given `pts` or `dts` (`skip_samples.ts`). For `last`, each packet is held back until the next one or the end of the stream arrives. To name side data types in an option, the first commit gives each `AV_PKT_DATA_*` type a short, stable handle (`display_matrix`, `skip_samples`, …) through an X-macro, `FF_PACKET_SIDE_DATA_HANDLES`, in `packet_internal.h`. The handles are written out rather than derived from `av_packet_side_data_name()`, because that name is meant for people and may be reworded. The bsf builds its `remove` option constants from this macro, so the standard AVOption array parser does the parsing and `ffmpeg -h bsf=side_data` lists the accepted names; the documentation points there instead of keeping its own copy of the list. Tests: - `fate-packet-side-data-handle` checks that every side data type has exactly one handle and that no two types share one. It compares the handles against a reference, since a released handle must not change. It fails when a new `AV_PKT_DATA_*` type is added without a handle. - `fate-ffmpeg-side-data-bsf-*` covers removal, setting each skip samples field, values of 0 on packets with and without the side data, and the `first`, `pts` and `last` packet selections. From 9d4c0152cd44d6cf95289ae7c614e1e34174b646 Mon Sep 17 00:00:00 2001 From: Romain Beauxis <[email protected]> Date: Thu, 15 Oct 2026 15:30:11 +0200 Subject: [PATCH 1/3] avcodec/packet_internal: add a handle for each side data type Add a list that gives each side data type a short handle made of lower case letters, digits and underscores, so that a filter can let its user name a type in an option. Each handle is written out rather than derived from the name that av_packet_side_data_name() returns, as that one is meant for people and may be reworded. --- libavcodec/packet_internal.h | 55 ++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/libavcodec/packet_internal.h b/libavcodec/packet_internal.h index dbaf744ee9..7b9474a4f7 100644 --- a/libavcodec/packet_internal.h +++ b/libavcodec/packet_internal.h @@ -27,4 +27,59 @@ int ff_side_data_set_prft(AVPacket *pkt, int64_t timestamp); +/** + * The handle of each side data type: a short, stable name made of lower case + * letters, digits and underscores, for use where a name has to be typed, such + * as in the options of a filter. HANDLE(type, handle) is called with the type + * without its AV_PKT_DATA_ prefix. + * + * Every type below AV_PKT_DATA_NB has one, which the packet_side_data_handle + * test checks. A handle that has been released must not change. + */ +#define FF_PACKET_SIDE_DATA_HANDLES(HANDLE) \ + HANDLE(PALETTE, "palette") \ + HANDLE(NEW_EXTRADATA, "new_extradata") \ + HANDLE(PARAM_CHANGE, "param_change") \ + HANDLE(H263_MB_INFO, "h263_mb_info") \ + HANDLE(REPLAYGAIN, "replaygain") \ + HANDLE(DISPLAYMATRIX, "display_matrix") \ + HANDLE(STEREO3D, "stereo3d") \ + HANDLE(AUDIO_SERVICE_TYPE, "audio_service_type") \ + HANDLE(QUALITY_STATS, "quality_stats") \ + HANDLE(FALLBACK_TRACK, "fallback_track") \ + HANDLE(CPB_PROPERTIES, "cpb_properties") \ + HANDLE(SKIP_SAMPLES, "skip_samples") \ + HANDLE(JP_DUALMONO, "jp_dualmono") \ + HANDLE(STRINGS_METADATA, "strings_metadata") \ + HANDLE(SUBTITLE_POSITION, "subtitle_position") \ + HANDLE(MATROSKA_BLOCKADDITIONAL, "matroska_blockadditional") \ + HANDLE(WEBVTT_IDENTIFIER, "webvtt_identifier") \ + HANDLE(WEBVTT_SETTINGS, "webvtt_settings") \ + HANDLE(METADATA_UPDATE, "metadata_update") \ + HANDLE(MPEGTS_STREAM_ID, "mpegts_stream_id") \ + HANDLE(MASTERING_DISPLAY_METADATA, "mastering_display") \ + HANDLE(CONTENT_LIGHT_LEVEL, "content_light_level") \ + HANDLE(SPHERICAL, "spherical") \ + HANDLE(A53_CC, "a53_cc") \ + HANDLE(ENCRYPTION_INIT_INFO, "encryption_init_info") \ + HANDLE(ENCRYPTION_INFO, "encryption_info") \ + HANDLE(AFD, "afd") \ + HANDLE(PRFT, "prft") \ + HANDLE(ICC_PROFILE, "icc_profile") \ + HANDLE(DOVI_CONF, "dovi_conf") \ + HANDLE(S12M_TIMECODE, "s12m_timecode") \ + HANDLE(DYNAMIC_HDR10_PLUS, "hdr10_plus") \ + HANDLE(DYNAMIC_HDR_SMPTE_2094_APP5, "hdr_smpte_2094_app5") \ + HANDLE(AMBIENT_VIEWING_ENVIRONMENT, "ambient_viewing_environment") \ + HANDLE(IAMF_MIX_GAIN_PARAM, "iamf_mix_gain_param") \ + HANDLE(IAMF_DEMIXING_INFO_PARAM, "iamf_demixing_info_param") \ + HANDLE(IAMF_RECON_GAIN_INFO_PARAM, "iamf_recon_gain_info_param") \ + HANDLE(FRAME_CROPPING, "frame_cropping") \ + HANDLE(LCEVC, "lcevc") \ + HANDLE(3D_REFERENCE_DISPLAYS, "3d_reference_displays") \ + HANDLE(RTCP_SR, "rtcp_sr") \ + HANDLE(EXIF, "exif") \ + HANDLE(HEVC_CONF, "hevc_conf") \ + HANDLE(GAIN_MAP_PARAMS, "gain_map_params") + #endif // AVCODEC_PACKET_INTERNAL_H -- 2.52.0 From a35e89955daf30e2bf4bd398f439611bd60b9345 Mon Sep 17 00:00:00 2001 From: Romain Beauxis <[email protected]> Date: Thu, 15 Oct 2026 15:31:57 +0200 Subject: [PATCH 2/3] tests/fate: check that every packet side data type has a handle Walk all the side data types: each one has to have a handle, which has to find the type again and not be shared. The handles are compared with the reference, so that a new type fails the test until it gets one and a released handle cannot change unnoticed. --- libavcodec/Makefile | 1 + libavcodec/tests/.gitignore | 1 + libavcodec/tests/packet_side_data_handle.c | 58 ++++++++++++++++++++++ tests/fate/libavcodec.mak | 4 ++ tests/ref/fate/packet-side-data-handle | 44 ++++++++++++++++ 5 files changed, 108 insertions(+) create mode 100644 libavcodec/tests/packet_side_data_handle.c create mode 100644 tests/ref/fate/packet-side-data-handle diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 249b0fe0b8..c73ee46ae2 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1389,6 +1389,7 @@ TESTPROGS = avcodec \ htmlsubtitles \ jpeg2000dwt \ mathops \ + packet_side_data_handle \ TESTPROGS-$(CONFIG_AC3DSP) += ac3 TESTPROGS-$(CONFIG_APV_DECODER) += apv diff --git a/libavcodec/tests/.gitignore b/libavcodec/tests/.gitignore index 8e4f26c2ad..3e986b00dd 100644 --- a/libavcodec/tests/.gitignore +++ b/libavcodec/tests/.gitignore @@ -19,5 +19,6 @@ /mjpegenc_huffman /mpeg12framerate /mpegaudiodecheader +/packet_side_data_handle /rangecoder /snowenc diff --git a/libavcodec/tests/packet_side_data_handle.c b/libavcodec/tests/packet_side_data_handle.c new file mode 100644 index 0000000000..152903ad47 --- /dev/null +++ b/libavcodec/tests/packet_side_data_handle.c @@ -0,0 +1,58 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdio.h> +#include <string.h> + +#include "libavcodec/packet.h" +#include "libavcodec/packet_internal.h" + +#define SET_HANDLE(type, handle) \ + if (handles[AV_PKT_DATA_ ## type]) { \ + printf("%s: second handle %s\n", #type, handle); \ + ret = 1; \ + } \ + handles[AV_PKT_DATA_ ## type] = handle; + +/* + * Every side data type needs exactly one handle, and no two types may share + * one. The handles are printed and compared with the reference, since a + * released handle must not change. + */ +int main(void) +{ + const char *handles[AV_PKT_DATA_NB] = { NULL }; + int ret = 0; + + FF_PACKET_SIDE_DATA_HANDLES(SET_HANDLE) + + for (int type = 0; type < AV_PKT_DATA_NB; type++) { + printf("%2d: %s\n", type, handles[type] ? handles[type] : "(none)"); + if (!handles[type]) + ret = 1; + + for (int other = 0; handles[type] && other < type; other++) { + if (handles[other] && !strcmp(handles[type], handles[other])) { + printf("%2d: handle of type %d\n", type, other); + ret = 1; + } + } + } + + return ret; +} diff --git a/tests/fate/libavcodec.mak b/tests/fate/libavcodec.mak index 3cde003212..713b1feb36 100644 --- a/tests/fate/libavcodec.mak +++ b/tests/fate/libavcodec.mak @@ -17,6 +17,10 @@ fate-avpacket: libavcodec/tests/avpacket$(EXESUF) fate-avpacket: CMD = run libavcodec/tests/avpacket$(EXESUF) fate-avpacket: CMP = null +FATE_LIBAVCODEC-yes += fate-packet-side-data-handle +fate-packet-side-data-handle: libavcodec/tests/packet_side_data_handle$(EXESUF) +fate-packet-side-data-handle: CMD = run libavcodec/tests/packet_side_data_handle$(EXESUF) + FATE_LIBAVCODEC-yes += fate-bitstream-be fate-bitstream-be: libavcodec/tests/bitstream_be$(EXESUF) fate-bitstream-be: CMD = run libavcodec/tests/bitstream_be$(EXESUF) diff --git a/tests/ref/fate/packet-side-data-handle b/tests/ref/fate/packet-side-data-handle new file mode 100644 index 0000000000..cdc2939e18 --- /dev/null +++ b/tests/ref/fate/packet-side-data-handle @@ -0,0 +1,44 @@ + 0: palette + 1: new_extradata + 2: param_change + 3: h263_mb_info + 4: replaygain + 5: display_matrix + 6: stereo3d + 7: audio_service_type + 8: quality_stats + 9: fallback_track +10: cpb_properties +11: skip_samples +12: jp_dualmono +13: strings_metadata +14: subtitle_position +15: matroska_blockadditional +16: webvtt_identifier +17: webvtt_settings +18: metadata_update +19: mpegts_stream_id +20: mastering_display +21: spherical +22: content_light_level +23: a53_cc +24: encryption_init_info +25: encryption_info +26: afd +27: prft +28: icc_profile +29: dovi_conf +30: s12m_timecode +31: hdr10_plus +32: iamf_mix_gain_param +33: iamf_demixing_info_param +34: iamf_recon_gain_info_param +35: ambient_viewing_environment +36: frame_cropping +37: lcevc +38: 3d_reference_displays +39: rtcp_sr +40: exif +41: hdr_smpte_2094_app5 +42: hevc_conf +43: gain_map_params -- 2.52.0 From b8e3104f44d6f825b62045ff6322f98e69b26d38 Mon Sep 17 00:00:00 2001 From: Romain Beauxis <[email protected]> Date: Thu, 15 Oct 2026 16:40:24 +0200 Subject: [PATCH 3/3] avcodec/bsf: add the side_data bitstream filter Add a filter that edits the side data of packets. The remove option drops the side data of some types. The types are named by their handle, as option constants joined with '+'. The skip_samples options set the samples to skip and to discard, and their reasons, in the skip samples side data of audio packets. One use is to keep the samples at the end of a stream that a demuxer asks to discard, as in skip_samples.discard=0:skip_samples.packet=last. The fields are set in every packet by default, or only in the first, the last, or the packets with a given pts or dts. For the last one, each packet is held back until the next one, or the end of the stream, comes. --- Changelog | 1 + doc/bitstream_filters.texi | 51 ++++ libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 1 + libavcodec/bsf/side_data.c | 251 ++++++++++++++++++ tests/fate/ffmpeg.mak | 32 +++ tests/ref/fate/ffmpeg-side-data-bsf-priming | 8 + tests/ref/fate/ffmpeg-side-data-bsf-remove | 8 + .../fate/ffmpeg-side-data-bsf-skip-samples | 8 + .../ffmpeg-side-data-bsf-skip-samples-discard | 8 + .../ffmpeg-side-data-bsf-skip-samples-first | 9 + .../ffmpeg-side-data-bsf-skip-samples-last | 13 + .../ffmpeg-side-data-bsf-skip-samples-pts | 9 + .../ffmpeg-side-data-bsf-skip-samples-zero | 8 + 14 files changed, 408 insertions(+) create mode 100644 libavcodec/bsf/side_data.c create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-priming create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-remove create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-discard create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-first create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-last create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-pts create mode 100644 tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-zero diff --git a/Changelog b/Changelog index 32e7c8bfe7..2e4c0ba105 100644 --- a/Changelog +++ b/Changelog @@ -12,6 +12,7 @@ version <next>: - DVD-Audio LPCM decoder and demuxing support - AVFoundation input device selection by unique ID and USB serial number - NVIDIA optical flow accelerated interpolation filter (vf_fruc_vulkan) +- side_data bitstream filter - H.264 data partitioning support - DSD (dsd_msbf) encoder - Rhetorex ADPCM decoder diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 47bb75892b..a3994030cd 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -1062,6 +1062,57 @@ ffmpeg -i INPUT -c:a copy -bsf:a setts=pts=DTS out.mkv Log basic packet information. Mainly useful for testing, debugging, and development. +@section side_data +Remove or modify the side data of packets. + +It accepts the following parameters: +@table @option +@item remove +Set the side data types to remove, joined with @samp{+}. Each type is given +by its handle; @command{ffmpeg -h bsf=side_data} lists them. + +@item skip_samples.skip +@item skip_samples.discard +Set the number of samples to skip at the start of the packet, or to discard at +its end, in the skip samples side data. A packet without the side data gets it +for a value other than 0. By default the value is left as it is. + +@item skip_samples.skip_reason +@item skip_samples.discard_reason +Set the reason of the skip, or of the discard, in the skip samples side data. +It is a number between 0 and 255. By default the value is left as it is. + +@item skip_samples.packet +Set the packets whose skip samples side data is set. It accepts the following +values: +@table @samp +@item all +Every packet. This is the default. +@item first +The first packet. +@item last +The last packet. Each packet is held back until the next one comes, so that +the last one is known at the end of the stream. +@item pts +@item dts +The packets whose pts, or dts, is @option{skip_samples.ts}. +@end table + +@item skip_samples.ts +Set the timestamp of the packets to set, for @samp{pts} and @samp{dts}, in the +time base of the stream. +@end table + +For example, to drop the display matrix and the replay gain of a stream: +@example +ffmpeg -i INPUT -c copy -bsf:v side_data=remove=display_matrix+replaygain output.mkv +@end example + +To keep the last samples of an audio stream that the demuxer asks to discard: +@example +ffmpeg -i INPUT -c copy -bsf:a side_data=skip_samples.discard=0:skip_samples.packet=last output.mka +@end example + @section smpte436m_to_eia608 Convert from a @code{SMPTE_436M_ANC} data stream to a @code{EIA_608} stream, diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index ed57db5281..4a569b9de4 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -66,6 +66,7 @@ extern const FFBitStreamFilter ff_pgs_frame_merge_bsf; extern const FFBitStreamFilter ff_prores_metadata_bsf; extern const FFBitStreamFilter ff_remove_extradata_bsf; extern const FFBitStreamFilter ff_setts_bsf; +extern const FFBitStreamFilter ff_side_data_bsf; extern const FFBitStreamFilter ff_showinfo_bsf; extern const FFBitStreamFilter ff_smpte436m_to_eia608_bsf; extern const FFBitStreamFilter ff_text2movsub_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index c0badf5ed9..ca110ab723 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -48,6 +48,7 @@ OBJS-$(CONFIG_PRORES_METADATA_BSF) += bsf/prores_metadata.o OBJS-$(CONFIG_REMOVE_EXTRADATA_BSF) += bsf/remove_extradata.o OBJS-$(CONFIG_SETTS_BSF) += bsf/setts.o OBJS-$(CONFIG_SHOWINFO_BSF) += bsf/showinfo.o +OBJS-$(CONFIG_SIDE_DATA_BSF) += bsf/side_data.o OBJS-$(CONFIG_SMPTE436M_TO_EIA608_BSF) += bsf/smpte436m_to_eia608.o OBJS-$(CONFIG_TEXT2MOVSUB_BSF) += bsf/movsub.o OBJS-$(CONFIG_TRACE_HEADERS_BSF) += bsf/trace_headers.o diff --git a/libavcodec/bsf/side_data.c b/libavcodec/bsf/side_data.c new file mode 100644 index 0000000000..4f92fd82a5 --- /dev/null +++ b/libavcodec/bsf/side_data.c @@ -0,0 +1,251 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Side data bitstream filter -- remove or modify the side data of packets. + */ + +#include <assert.h> +#include <stdint.h> +#include <string.h> + +#include "libavcodec/bsf.h" +#include "libavcodec/bsf_internal.h" +#include "libavcodec/packet_internal.h" + +#include "libavutil/intreadwrite.h" +#include "libavutil/macros.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" + +enum SkipSamplesPacket { + SKIP_SAMPLES_ALL, + SKIP_SAMPLES_FIRST, + SKIP_SAMPLES_LAST, + SKIP_SAMPLES_PTS, + SKIP_SAMPLES_DTS, +}; + +typedef struct SideDataContext { + const AVClass *class; + + int *remove; + unsigned nb_remove; + uint64_t remove_mask; ///< bit i set: remove the side data of type i + + int skip_samples[2]; ///< skip, discard; -1 to leave as is + int skip_reasons[2]; ///< skip reason, discard reason; -1 to leave as is + int skip_packet; ///< enum SkipSamplesPacket: the packets to set + int64_t skip_ts; ///< timestamp of the packet to set + + int seen_first; ///< a packet has been output + AVPacket *held; ///< latest packet, output once the next one shows it is not the last +} SideDataContext; + +static int side_data_init(AVBSFContext *ctx) +{ + SideDataContext *s = ctx->priv_data; + + static_assert(AV_PKT_DATA_NB <= 64, "remove_mask has a bit per type"); + + for (unsigned i = 0; i < s->nb_remove; i++) + s->remove_mask |= UINT64_C(1) << s->remove[i]; + + if ((s->skip_packet == SKIP_SAMPLES_PTS || s->skip_packet == SKIP_SAMPLES_DTS) && + s->skip_ts == AV_NOPTS_VALUE) { + av_log(ctx, AV_LOG_ERROR, "skip_samples.packet=pts or dts needs skip_samples.ts\n"); + return AVERROR(EINVAL); + } + + if (s->skip_packet == SKIP_SAMPLES_LAST) { + s->held = av_packet_alloc(); + if (!s->held) + return AVERROR(ENOMEM); + } + + return 0; +} + +static void remove_side_data(AVPacket *pkt, uint64_t remove_mask) +{ + int kept = 0; + + for (int i = 0; i < pkt->side_data_elems; i++) { + AVPacketSideData sd = pkt->side_data[i]; + + if (remove_mask & (UINT64_C(1) << sd.type)) + av_free(sd.data); + else + pkt->side_data[kept++] = sd; + } + pkt->side_data_elems = kept; +} + +/** + * Set a field of the skip samples side data: the two counts are 32 bit little + * endian at 0 and 4, the two reasons one byte at 8 and 9. + * Absent side data is only created for a value that is not 0. Side data + * shorter than 10 bytes is replaced, keeping its fields. + */ +static int set_skip_samples(AVPacket *pkt, int offset, int size, int value) +{ + size_t sd_size; + uint8_t *sd = av_packet_get_side_data(pkt, AV_PKT_DATA_SKIP_SAMPLES, &sd_size); + + if (!sd && !value) + return 0; + + if (!sd || sd_size < 10) { + uint8_t fields[10] = { 0 }; + + if (sd) { + memcpy(fields, sd, sd_size); + av_packet_side_data_remove(pkt->side_data, &pkt->side_data_elems, + AV_PKT_DATA_SKIP_SAMPLES); + } + sd = av_packet_new_side_data(pkt, AV_PKT_DATA_SKIP_SAMPLES, 10); + if (!sd) + return AVERROR(ENOMEM); + memcpy(sd, fields, 10); + } + + if (size == 4) + AV_WL32(sd + offset, value); + else + sd[offset] = value; + + return 0; +} + +/** + * Get the next packet to output. With skip_samples.packet=last, each packet + * is held until the next one, or the end of the stream, comes. + */ +static int get_packet(AVBSFContext *ctx, AVPacket *pkt, int *is_last) +{ + SideDataContext *s = ctx->priv_data; + int ret = ff_bsf_get_packet_ref(ctx, pkt); + + *is_last = 0; + if (!s->held) + return ret; + + if (ret == AVERROR_EOF && !AVPACKET_IS_EMPTY(s->held)) { + av_packet_move_ref(pkt, s->held); + *is_last = 1; + return 0; + } + if (ret < 0) + return ret; + + FFSWAP(AVPacket, *pkt, *s->held); + + return AVPACKET_IS_EMPTY(pkt) ? AVERROR(EAGAIN) : 0; +} + +static int skip_samples_target(SideDataContext *s, const AVPacket *pkt, int is_last) +{ + switch (s->skip_packet) { + case SKIP_SAMPLES_FIRST: return !s->seen_first; + case SKIP_SAMPLES_LAST: return is_last; + case SKIP_SAMPLES_PTS: return pkt->pts == s->skip_ts; + case SKIP_SAMPLES_DTS: return pkt->dts == s->skip_ts; + } + return 1; +} + +static int side_data_filter(AVBSFContext *ctx, AVPacket *pkt) +{ + SideDataContext *s = ctx->priv_data; + int is_last; + int ret = get_packet(ctx, pkt, &is_last); + + if (ret < 0) + return ret; + + if (s->remove_mask) + remove_side_data(pkt, s->remove_mask); + + for (int i = 0; i < 2 && ret >= 0 && skip_samples_target(s, pkt, is_last); i++) { + if (s->skip_samples[i] >= 0) + ret = set_skip_samples(pkt, 4 * i, 4, s->skip_samples[i]); + if (s->skip_reasons[i] >= 0 && ret >= 0) + ret = set_skip_samples(pkt, 8 + i, 1, s->skip_reasons[i]); + } + s->seen_first = 1; + + if (ret < 0) + av_packet_unref(pkt); + + return ret; +} + +static void side_data_flush(AVBSFContext *ctx) +{ + SideDataContext *s = ctx->priv_data; + + av_packet_unref(s->held); +} + +static void side_data_close(AVBSFContext *ctx) +{ + SideDataContext *s = ctx->priv_data; + + av_packet_free(&s->held); +} + +static const AVOptionArrayDef remove_def = { .sep = '+' }; + +#define OFFSET(x) offsetof(SideDataContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_SUBTITLE_PARAM | AV_OPT_FLAG_BSF_PARAM) +#define TYPE_CONST(type, handle) \ + { handle, NULL, 0, AV_OPT_TYPE_CONST, { .i64 = AV_PKT_DATA_ ## type }, .flags = FLAGS, .unit = "type" }, +static const AVOption options[] = { + { "remove", "side data types to remove, joined with '+'", OFFSET(remove), AV_OPT_TYPE_INT | AV_OPT_TYPE_FLAG_ARRAY, { .arr = &remove_def }, 0, AV_PKT_DATA_NB - 1, FLAGS, .unit = "type" }, + FF_PACKET_SIDE_DATA_HANDLES(TYPE_CONST) + { "skip_samples.skip", "set the samples to skip at the start of the packet", OFFSET(skip_samples[0]), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT32_MAX, FLAGS }, + { "skip_samples.discard", "set the samples to discard at the end of the packet", OFFSET(skip_samples[1]), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT32_MAX, FLAGS }, + { "skip_samples.skip_reason", "set the reason of the skip", OFFSET(skip_reasons[0]), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 255, FLAGS }, + { "skip_samples.discard_reason", "set the reason of the discard", OFFSET(skip_reasons[1]), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 255, FLAGS }, + { "skip_samples.packet", "set the packets whose skip samples are set", OFFSET(skip_packet), AV_OPT_TYPE_INT, { .i64 = SKIP_SAMPLES_ALL }, 0, SKIP_SAMPLES_DTS, FLAGS, .unit = "packet" }, + { "all", "every packet", 0, AV_OPT_TYPE_CONST, { .i64 = SKIP_SAMPLES_ALL }, .flags = FLAGS, .unit = "packet" }, + { "first", "the first packet", 0, AV_OPT_TYPE_CONST, { .i64 = SKIP_SAMPLES_FIRST }, .flags = FLAGS, .unit = "packet" }, + { "last", "the last packet", 0, AV_OPT_TYPE_CONST, { .i64 = SKIP_SAMPLES_LAST }, .flags = FLAGS, .unit = "packet" }, + { "pts", "the packets whose pts is the given ts", 0, AV_OPT_TYPE_CONST, { .i64 = SKIP_SAMPLES_PTS }, .flags = FLAGS, .unit = "packet" }, + { "dts", "the packets whose dts is the given ts", 0, AV_OPT_TYPE_CONST, { .i64 = SKIP_SAMPLES_DTS }, .flags = FLAGS, .unit = "packet" }, + { "skip_samples.ts", "set the timestamp of the packet, for packet=pts or dts", OFFSET(skip_ts), AV_OPT_TYPE_INT64, { .i64 = AV_NOPTS_VALUE }, INT64_MIN, INT64_MAX, FLAGS }, + { NULL }, +}; + +static const AVClass side_data_class = { + .class_name = "side_data bsf", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFBitStreamFilter ff_side_data_bsf = { + .p.name = "side_data", + .p.priv_class = &side_data_class, + .priv_data_size = sizeof(SideDataContext), + .init = side_data_init, + .flush = side_data_flush, + .close = side_data_close, + .filter = side_data_filter, +}; diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index 9da5ed9d61..f32456ae8b 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -300,6 +300,38 @@ fate-ffmpeg-trim-bsf-priming-duration: CMD = framecrc -copyts -i $(TARGET_SAMPLE FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, TRIM_BSF PIPE_PROTOCOL) += fate-ffmpeg-trim-bsf-priming-untrimmed fate-ffmpeg-trim-bsf-priming-untrimmed: CMD = framecrc -copyts -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a trim=start=1:start_type=pkt_index:trim_packets=0 -frames:a 3 +# The first packet carries the priming as skip samples side data, which framecrc +# shows as S=, and which the filter takes away. +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-priming +fate-ffmpeg-side-data-bsf-priming: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -frames:a 2 + +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-remove +fate-ffmpeg-side-data-bsf-remove: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=remove=skip_samples -frames:a 2 + +# The counts and the reasons are set on the packet that has the side data, and +# a packet that has none gets it for a value other than 0. +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples +fate-ffmpeg-side-data-bsf-skip-samples: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.skip=100:skip_samples.skip_reason=3:skip_samples.discard_reason=4 -frames:a 2 + +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples-discard +fate-ffmpeg-side-data-bsf-skip-samples-discard: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.discard=5 -frames:a 2 + +# A value of 0 leaves a packet without the side data as it is, and clears it +# in the packet that has it. +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples-zero +fate-ffmpeg-side-data-bsf-skip-samples-zero: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.skip=0:skip_samples.discard=0 -frames:a 2 + +# Only the matching packets are set. The pts one had no side data. +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples-first +fate-ffmpeg-side-data-bsf-skip-samples-first: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.discard=7:skip_samples.packet=first -frames:a 3 + +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples-pts +fate-ffmpeg-side-data-bsf-skip-samples-pts: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.discard=7:skip_samples.packet=pts:skip_samples.ts=-1088 -frames:a 3 + +# The input is cut so that the filter sees the end of the stream. +FATE_SAMPLES_FFMPEG-$(call DEMMUX, MOV, FRAMECRC, SIDE_DATA_BSF PIPE_PROTOCOL) += fate-ffmpeg-side-data-bsf-skip-samples-last +fate-ffmpeg-side-data-bsf-skip-samples-last: CMD = framecrc -t 0.3 -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.m4a -c:a copy -bsf:a side_data=skip_samples.discard=7:skip_samples.packet=last + # The composed skips cover 800 + 300 of the 1024 samples, leaving nothing. FATE_SAMPLES_FFMPEG-$(call DEMMUX, AAC, FRAMECRC, AAC_PARSER TRIM_BSF PIPE_PROTOCOL) += fate-ffmpeg-trim-bsf-skip-samples-empty fate-ffmpeg-trim-bsf-skip-samples-empty: CMD = framecrc -i $(TARGET_SAMPLES)/audiomatch/tones_afconvert_16000_mono_aac_lc.adts -c:a copy -bsf:a trim=start=3217536,trim=end=3083472 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-priming b/tests/ref/fate/ffmpeg-side-data-bsf-priming new file mode 100644 index 0000000000..f0f721c42f --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-priming @@ -0,0 +1,8 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x02c80048 +0, -1088, -1088, 1024, 163, 0xd5f85007 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-remove b/tests/ref/fate/ffmpeg-side-data-bsf-remove new file mode 100644 index 0000000000..1f802bbd45 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-remove @@ -0,0 +1,8 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117 +0, -1088, -1088, 1024, 163, 0xd5f85007 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples new file mode 100644 index 0000000000..4b4e367e88 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples @@ -0,0 +1,8 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x03f2006b +0, -1088, -1088, 1024, 163, 0xd5f85007, S=1, Skip Samples, 10, 0x03f2006b diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-discard b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-discard new file mode 100644 index 0000000000..257b637858 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-discard @@ -0,0 +1,8 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x02e6004d +0, -1088, -1088, 1024, 163, 0xd5f85007, S=1, Skip Samples, 10, 0x001e0005 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-first b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-first new file mode 100644 index 0000000000..d43414a53f --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-first @@ -0,0 +1,9 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x02f2004f +0, -1088, -1088, 1024, 163, 0xd5f85007 +0, -64, -64, 1024, 127, 0x66484065 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-last b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-last new file mode 100644 index 0000000000..08a917c762 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-last @@ -0,0 +1,13 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x02c80048 +0, -1088, -1088, 1024, 163, 0xd5f85007 +0, -64, -64, 1024, 127, 0x66484065 +0, 960, 960, 1024, 94, 0x55222bd6 +0, 1984, 1984, 1024, 314, 0x3c7e923a +0, 3008, 3008, 1024, 207, 0x1efc5d1b +0, 4032, 4032, 1024, 119, 0xb2a13601, S=1, Skip Samples, 10, 0x002a0007 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-pts b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-pts new file mode 100644 index 0000000000..60d4c7d647 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-pts @@ -0,0 +1,9 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x02c80048 +0, -1088, -1088, 1024, 163, 0xd5f85007, S=1, Skip Samples, 10, 0x002a0007 +0, -64, -64, 1024, 127, 0x66484065 diff --git a/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-zero b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-zero new file mode 100644 index 0000000000..6cc714d7a2 --- /dev/null +++ b/tests/ref/fate/ffmpeg-side-data-bsf-skip-samples-zero @@ -0,0 +1,8 @@ +#extradata 0: 2, 0x0030001c +#tb 0: 1/16000 +#media_type 0: audio +#codec_id 0: aac +#sample_rate 0: 16000 +#channel_layout_name 0: mono +0, -2112, -2112, 1024, 4, 0x02f70117, S=1, Skip Samples, 10, 0x00000000 +0, -1088, -1088, 1024, 163, 0xd5f85007 -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
