PR #23374 opened by arch1t3cht URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23374 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23374.patch
A first-party codec mapping for VC-1 was added to Matroska in https://github.com/ietf-wg-cellar/matroska-specification/pull/1080 . This PR pulls most of the mp4 muxer's VC-1 parsing and muxing code out into generic functions and adds support for muxing and demuxing native VC-1 streams in Matroska files. Unfortunately this requires a fair amount of special handling since muxing VC-1 requires more data than is available in just the codecpar (see below). Some comments: - Muxing VC-1 according to SMPTE RP 2025 requires some additional data cannot be obtained from just the stream's codecpar (at least with the VC-1 extradata format FFmpeg currently uses) and has to be collected from the packets while muxing the stream. As discussed on IRC, this PR handles this in the Matroska muxer by writing the (updated) CodecPrivate in write_trailer instead of write_header. A more robust solution to this would be also adding the VC1DecSpecStruc's content to VC1's extradata, but I believe this would constitute an API break. - AFAIK this is the first patch that adds support for demuxing native VC-1 streams to some tool (though I do also have a patch for mpv ready and plan to write a patch for mkvtoolnix). Even if this PR adds the ability to both mux *and* demux native VC-1 streams, mkv VC-1 streams muxed by FFmpeg after this patch will not be able to be read by older FFmpeg versions or other tools. I'm not sure how FFmpeg usually handles these situations. I guess the options are a) to not worry about this, b) to delay merging this patch until equivalent patches to all relevant other tools are ready, c) to only merge the demuxing support now and delay merging the muxing support until later, or d) to lock the native VC-1 muxing behind an off-by-default option. To me, option d) makes the most sense, but I wanted to reach out and ask before adding Yet Another Config Flag. - If it's preferred, I can squash the last two commits together. As it is, the second-to-last commit (adding muxing support) also makes the demuxer *recognize* the stream, but not parse the extradata correctly. >From 999ce35b6903f85dd68e4e86f70589bcdb303714 Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Sun, 10 May 2026 22:15:06 +0200 Subject: [PATCH 1/6] avformat/movenc: Separate vc1_parse_state from vc1_info Prepare to extract the vc1 parsing code into a more generic function. The fields vc1_info are needed for the actual vc1 parsing and muxing while vc1_parse_state is specific to the mov muxer. --- libavformat/movenc.c | 10 +++++----- libavformat/movenc.h | 7 +++++-- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index b466ba7531..d71b09e0e3 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -6496,13 +6496,13 @@ static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) break; } } - if (!trk->entry && trk->vc1_info.first_packet_seen) - trk->vc1_info.first_frag_written = 1; - if (!trk->entry && !trk->vc1_info.first_frag_written) { + if (!trk->entry && trk->vc1_parse_state.first_packet_seen) + trk->vc1_parse_state.first_frag_written = 1; + if (!trk->entry && !trk->vc1_parse_state.first_frag_written) { /* First packet in first fragment */ trk->vc1_info.first_packet_seq = seq; trk->vc1_info.first_packet_entry = entry; - trk->vc1_info.first_packet_seen = 1; + trk->vc1_parse_state.first_packet_seen = 1; } else if ((seq && !trk->vc1_info.packet_seq) || (entry && !trk->vc1_info.packet_entry)) { int i; @@ -6513,7 +6513,7 @@ static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) trk->vc1_info.packet_seq = 1; if (entry) trk->vc1_info.packet_entry = 1; - if (!trk->vc1_info.first_frag_written) { + if (!trk->vc1_parse_state.first_frag_written) { /* First fragment */ if ((!seq || trk->vc1_info.first_packet_seq) && (!entry || trk->vc1_info.first_packet_entry)) { diff --git a/libavformat/movenc.h b/libavformat/movenc.h index f47d1381a9..ea60da2e22 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -167,13 +167,16 @@ typedef struct MOVTrack { struct { int first_packet_seq; int first_packet_entry; - int first_packet_seen; - int first_frag_written; int packet_seq; int packet_entry; int slices; } vc1_info; + struct { + int first_packet_seen; + int first_frag_written; + } vc1_parse_state; + void *eac3_priv; MOVMuxCencContext cenc; -- 2.52.0 >From 2d783172dc500d971c8ee613bb7bff54f411f2a3 Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Sun, 10 May 2026 22:46:46 +0200 Subject: [PATCH 2/6] avformat/movenc: Extract generic vc1 parsing into separate function This is in preparation for moving the parsing logic outside of movenc.c so that it can be used in different muxers. In order to exactly preserve the existing logic of resetting previous sync sample flags when new sequence headers/entry points are found, make the parsing function take an output argument specifying when the muxer should reset previous keyframe flags (where possible). --- libavformat/movenc.c | 99 +++++++++++++++++++++++++++++--------------- libavformat/movenc.h | 9 +--- libavformat/vc1.h | 33 +++++++++++++++ 3 files changed, 100 insertions(+), 41 deletions(-) create mode 100644 libavformat/vc1.h diff --git a/libavformat/movenc.c b/libavformat/movenc.c index d71b09e0e3..475252c757 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -6476,11 +6476,15 @@ static int mov_parse_mpeg2_frame(AVPacket *pkt, uint32_t *flags) return 0; } -static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) -{ +enum VC1MuxAction { + VC1_MUX_ACTION_NONE, ///< No action required + VC1_MUX_ACTION_RESET_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets + VC1_MUX_ACTION_RESET_LATER_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets except the first +}; + +static void ff_vc1_parse_frame(const AVPacket *pkt, VC1MuxInfo *muxinfo, int firstpkt, int *key, enum VC1MuxAction *action) { const uint8_t *start, *next, *end = pkt->data + pkt->size; int seq = 0, entry = 0; - int key = pkt->flags & AV_PKT_FLAG_KEY; start = find_next_marker(pkt->data, end); for (next = start; next < end; start = next) { next = find_next_marker(start + 4, end); @@ -6492,44 +6496,71 @@ static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) entry = 1; break; case VC1_CODE_SLICE: - trk->vc1_info.slices = 1; + muxinfo->slices = 1; break; } } - if (!trk->entry && trk->vc1_parse_state.first_packet_seen) - trk->vc1_parse_state.first_frag_written = 1; - if (!trk->entry && !trk->vc1_parse_state.first_frag_written) { - /* First packet in first fragment */ - trk->vc1_info.first_packet_seq = seq; - trk->vc1_info.first_packet_entry = entry; - trk->vc1_parse_state.first_packet_seen = 1; - } else if ((seq && !trk->vc1_info.packet_seq) || - (entry && !trk->vc1_info.packet_entry)) { - int i; - for (i = 0; i < trk->entry; i++) - trk->cluster[i].flags &= ~MOV_SYNC_SAMPLE; - trk->has_keyframes = 0; + + *action = VC1_MUX_ACTION_NONE; + + if (firstpkt) { + muxinfo->first_packet_seq = seq; + muxinfo->first_packet_entry = entry; + } else if ((seq && !muxinfo->packet_seq) || + (entry && !muxinfo->packet_entry)) { + *action = VC1_MUX_ACTION_RESET_KEYS; + if (seq) - trk->vc1_info.packet_seq = 1; + muxinfo->packet_seq = 1; if (entry) - trk->vc1_info.packet_entry = 1; - if (!trk->vc1_parse_state.first_frag_written) { - /* First fragment */ - if ((!seq || trk->vc1_info.first_packet_seq) && - (!entry || trk->vc1_info.first_packet_entry)) { - /* First packet had the same headers as this one, readd the - * sync sample flag. */ - trk->cluster[0].flags |= MOV_SYNC_SAMPLE; - trk->has_keyframes = 1; - } + muxinfo->packet_entry = 1; + + if ((!seq || muxinfo->first_packet_seq) && + (!entry || muxinfo->first_packet_entry)) { + /* First packet had the same headers as this one */ + *action = VC1_MUX_ACTION_RESET_LATER_KEYS; } } - if (trk->vc1_info.packet_seq && trk->vc1_info.packet_entry) - key = seq && entry; - else if (trk->vc1_info.packet_seq) - key = seq; - else if (trk->vc1_info.packet_entry) - key = entry; + + *key = pkt->flags & AV_PKT_FLAG_KEY; + if (muxinfo->packet_seq && muxinfo->packet_entry) + *key = seq && entry; + else if (muxinfo->packet_seq) + *key = seq; + else if (muxinfo->packet_entry) + *key = entry; +} + +static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) +{ + int firstpacket = 0; + + if (!trk->entry && trk->vc1_parse_state.first_packet_seen) + trk->vc1_parse_state.first_frag_written = 1; + + if (!trk->entry && !trk->vc1_parse_state.first_frag_written) { + /* First packet in first fragment */ + trk->vc1_parse_state.first_packet_seen = 1; + firstpacket = 1; + } + + int key; + enum VC1MuxAction action; + ff_vc1_parse_frame(pkt, &trk->vc1_info, firstpacket, &key, &action); + + if (action == VC1_MUX_ACTION_RESET_KEYS || action == VC1_MUX_ACTION_RESET_LATER_KEYS) { + for (int i = 0; i < trk->entry; i++) + trk->cluster[i].flags &= ~MOV_SYNC_SAMPLE; + trk->has_keyframes = 0; + + if (action == VC1_MUX_ACTION_RESET_LATER_KEYS && !trk->vc1_parse_state.first_frag_written) { + /* First packet had the same headers as this one, readd the + * sync sample flag. */ + trk->cluster[0].flags |= MOV_SYNC_SAMPLE; + trk->has_keyframes = 1; + } + } + if (key) { trk->cluster[trk->entry].flags |= MOV_SYNC_SAMPLE; trk->has_keyframes++; diff --git a/libavformat/movenc.h b/libavformat/movenc.h index ea60da2e22..c5e3be6cf5 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -26,6 +26,7 @@ #include "avformat.h" #include "movenccenc.h" +#include "vc1.h" #include "libavcodec/packet_internal.h" #define MOV_FRAG_INFO_ALLOC_INCREMENT 64 @@ -164,13 +165,7 @@ typedef struct MOVTrack { MOVFragmentInfo *frag_info; unsigned frag_info_capacity; - struct { - int first_packet_seq; - int first_packet_entry; - int packet_seq; - int packet_entry; - int slices; - } vc1_info; + VC1MuxInfo vc1_info; struct { int first_packet_seen; diff --git a/libavformat/vc1.h b/libavformat/vc1.h new file mode 100644 index 0000000000..276b9e624c --- /dev/null +++ b/libavformat/vc1.h @@ -0,0 +1,33 @@ +/* + * VC-1 helper functions for muxers + * Copyright (c) 2026 The FFmpeg Project + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFORMAT_VC1_H +#define AVFORMAT_VC1_H + +typedef struct VC1MuxInfo { + int first_packet_seq; ///< Whether the first packet has a sequence header + int first_packet_entry; ///< Whether the first packet has an entry point + int packet_seq; ///< Whether there exist packets containing a sequence header + int packet_entry; ///< Whether there exist packets containing entry points + int slices; ///< Whether there exist packets containing slices +} VC1MuxInfo; + +#endif /* AVFORMAT_AVC_H */ -- 2.52.0 >From bc0c2592bcc12b03a0716f32512e37a957c68df8 Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Sun, 3 May 2026 19:53:49 +0200 Subject: [PATCH 3/6] avformat/movenc: Make dvc1 writing functions generic This is in preparation for moving these functions outside of movenc.c so that they can be used by other muxers. --- libavformat/movenc.c | 70 ++++++++++++++++++++++++++++---------------- 1 file changed, 45 insertions(+), 25 deletions(-) diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 475252c757..0b838d2253 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1099,31 +1099,27 @@ static int mov_write_wave_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra return update_size(pb, pos); } -static int mov_write_dvc1_structs(MOVTrack *track, uint8_t *buf) -{ +static int mov_write_dvc1_structs(const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len, uint8_t *buf) { uint8_t *unescaped; - const uint8_t *start, *next, *end = track->extradata[track->last_stsd_index] + - track->extradata_size[track->last_stsd_index]; + const uint8_t *start, *next, *end = extradata + extradata_len; + int unescaped_size, seq_found = 0; int level = 0, interlace = 0; - int packet_seq = track->vc1_info.packet_seq; - int packet_entry = track->vc1_info.packet_entry; - int slices = track->vc1_info.slices; + int packet_seq = 1; + int packet_entry = 1; + int slices = 0; PutBitContext pbc; - if (track->start_dts == AV_NOPTS_VALUE) { - /* No packets written yet, vc1_info isn't authoritative yet. */ - /* Assume inline sequence and entry headers. */ - packet_seq = packet_entry = 1; - av_log(NULL, AV_LOG_WARNING, - "moov atom written before any packets, unable to write correct " - "dvc1 atom. Set the delay_moov flag to fix this.\n"); + if (muxinfo) { + packet_seq = muxinfo->packet_seq; + packet_entry = muxinfo->packet_entry; + slices = muxinfo->slices; } - unescaped = av_mallocz(track->extradata_size[track->last_stsd_index] + AV_INPUT_BUFFER_PADDING_SIZE); + unescaped = av_mallocz(extradata_len + AV_INPUT_BUFFER_PADDING_SIZE); if (!unescaped) return AVERROR(ENOMEM); - start = find_next_marker(track->extradata[track->last_stsd_index], end); + start = find_next_marker(extradata, end); for (next = start; next < end; start = next) { GetBitContext gb; int size; @@ -1171,8 +1167,8 @@ static int mov_write_dvc1_structs(MOVTrack *track, uint8_t *buf) put_bits(&pbc, 1, 0); /* reserved */ /* framerate */ - if (track->st->avg_frame_rate.num > 0 && track->st->avg_frame_rate.den > 0) - put_bits32(&pbc, track->st->avg_frame_rate.num / track->st->avg_frame_rate.den); + if (st->avg_frame_rate.num > 0 && st->avg_frame_rate.den > 0) + put_bits32(&pbc, st->avg_frame_rate.num / st->avg_frame_rate.den); else put_bits32(&pbc, 0xffffffff); @@ -1183,23 +1179,47 @@ static int mov_write_dvc1_structs(MOVTrack *track, uint8_t *buf) return 0; } -static int mov_write_dvc1_tag(AVIOContext *pb, MOVTrack *track) -{ +static int mov_write_vc1decspecstruc(AVIOContext *pb, const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len) { uint8_t buf[7] = { 0 }; int ret; - if ((ret = mov_write_dvc1_structs(track, buf)) < 0) + if ((ret = mov_write_dvc1_structs(st, muxinfo, extradata, extradata_len, buf)) < 0) return ret; - avio_wb32(pb, track->extradata_size[track->last_stsd_index] + 8 + sizeof(buf)); - ffio_wfourcc(pb, "dvc1"); avio_write(pb, buf, sizeof(buf)); - avio_write(pb, track->extradata[track->last_stsd_index], - track->extradata_size[track->last_stsd_index]); + avio_write(pb, extradata, extradata_len); return 0; } +static int mov_write_dvc1_tag(AVIOContext *pb, MOVTrack *track) +{ + const VC1MuxInfo *vc1_info = &track->vc1_info; + int ret; + + if (track->start_dts == AV_NOPTS_VALUE) { + /* No packets written yet, vc1_info isn't authoritative yet. */ + /* Assume inline sequence and entry headers. */ + av_log(NULL, AV_LOG_WARNING, + "moov atom written before any packets, unable to write correct " + "dvc1 atom. Set the delay_moov flag to fix this.\n"); + + vc1_info = NULL; + } + + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "dvc1"); + + if ((ret = mov_write_vc1decspecstruc(pb, track->st, vc1_info, + track->extradata[track->last_stsd_index], + track->extradata_size[track->last_stsd_index])) < 0) + return ret; + + return update_size(pb, pos); +} + static int mov_write_glbl_tag(AVIOContext *pb, MOVTrack *track) { avio_wb32(pb, track->extradata_size[track->last_stsd_index] + 8); -- 2.52.0 >From 57d08a8f9b0c89bc3c55b5849d45ebdf228457f7 Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Sun, 3 May 2026 20:12:30 +0200 Subject: [PATCH 4/6] avformat/vc1: Extract vc1 muxing functions from movenc.c --- libavformat/Makefile | 2 +- libavformat/movenc.c | 151 +------------------------------------- libavformat/vc1.c | 170 +++++++++++++++++++++++++++++++++++++++++++ libavformat/vc1.h | 23 ++++++ 4 files changed, 195 insertions(+), 151 deletions(-) create mode 100644 libavformat/vc1.c diff --git a/libavformat/Makefile b/libavformat/Makefile index 0db0c7c2a9..a0a6975120 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -43,7 +43,7 @@ OBJS-$(HAVE_LIBC_MSVCRT) += file_open.o OBJS-$(CONFIG_CBS_APV_LAVF) += cbs_apv.o cbs.o OBJS-$(CONFIG_CBS_AV1_LAVF) += cbs_av1.o cbs.o OBJS-$(CONFIG_ISO_MEDIA) += isom.o -OBJS-$(CONFIG_ISO_WRITER) += avc.o hevc.o vvc.o +OBJS-$(CONFIG_ISO_WRITER) += avc.o hevc.o vc1.o vvc.o OBJS-$(CONFIG_IAMFDEC) += iamf_reader.o iamf_parse.o iamf.o OBJS-$(CONFIG_IAMFENC) += iamf_writer.o iamf.o OBJS-$(CONFIG_NETWORK) += network.o diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 0b838d2253..ae2d1a2a05 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -42,7 +42,6 @@ #include "libavcodec/ac3_parser_internal.h" #include "libavcodec/dnxhddata.h" #include "libavcodec/flac.h" -#include "libavcodec/get_bits.h" #include "libavcodec/internal.h" #include "libavcodec/put_bits.h" @@ -1099,99 +1098,6 @@ static int mov_write_wave_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra return update_size(pb, pos); } -static int mov_write_dvc1_structs(const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len, uint8_t *buf) { - uint8_t *unescaped; - const uint8_t *start, *next, *end = extradata + extradata_len; - - int unescaped_size, seq_found = 0; - int level = 0, interlace = 0; - int packet_seq = 1; - int packet_entry = 1; - int slices = 0; - PutBitContext pbc; - - if (muxinfo) { - packet_seq = muxinfo->packet_seq; - packet_entry = muxinfo->packet_entry; - slices = muxinfo->slices; - } - - unescaped = av_mallocz(extradata_len + AV_INPUT_BUFFER_PADDING_SIZE); - if (!unescaped) - return AVERROR(ENOMEM); - start = find_next_marker(extradata, end); - for (next = start; next < end; start = next) { - GetBitContext gb; - int size; - next = find_next_marker(start + 4, end); - size = next - start - 4; - if (size <= 0) - continue; - unescaped_size = vc1_unescape_buffer(start + 4, size, unescaped); - init_get_bits(&gb, unescaped, 8 * unescaped_size); - if (AV_RB32(start) == VC1_CODE_SEQHDR) { - int profile = get_bits(&gb, 2); - if (profile != PROFILE_ADVANCED) { - av_free(unescaped); - return AVERROR(ENOSYS); - } - seq_found = 1; - level = get_bits(&gb, 3); - /* chromaformat, frmrtq_postproc, bitrtq_postproc, postprocflag, - * width, height */ - skip_bits_long(&gb, 2 + 3 + 5 + 1 + 2*12); - skip_bits(&gb, 1); /* broadcast */ - interlace = get_bits1(&gb); - skip_bits(&gb, 4); /* tfcntrflag, finterpflag, reserved, psf */ - } - } - if (!seq_found) { - av_free(unescaped); - return AVERROR(ENOSYS); - } - - init_put_bits(&pbc, buf, 7); - /* VC1DecSpecStruc */ - put_bits(&pbc, 4, 12); /* profile - advanced */ - put_bits(&pbc, 3, level); - put_bits(&pbc, 1, 0); /* reserved */ - /* VC1AdvDecSpecStruc */ - put_bits(&pbc, 3, level); - put_bits(&pbc, 1, 0); /* cbr */ - put_bits(&pbc, 6, 0); /* reserved */ - put_bits(&pbc, 1, !interlace); /* no interlace */ - put_bits(&pbc, 1, !packet_seq); /* no multiple seq */ - put_bits(&pbc, 1, !packet_entry); /* no multiple entry */ - put_bits(&pbc, 1, !slices); /* no slice code */ - put_bits(&pbc, 1, 0); /* no bframe */ - put_bits(&pbc, 1, 0); /* reserved */ - - /* framerate */ - if (st->avg_frame_rate.num > 0 && st->avg_frame_rate.den > 0) - put_bits32(&pbc, st->avg_frame_rate.num / st->avg_frame_rate.den); - else - put_bits32(&pbc, 0xffffffff); - - flush_put_bits(&pbc); - - av_free(unescaped); - - return 0; -} - -static int mov_write_vc1decspecstruc(AVIOContext *pb, const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len) { - uint8_t buf[7] = { 0 }; - int ret; - - if ((ret = mov_write_dvc1_structs(st, muxinfo, extradata, extradata_len, buf)) < 0) - return ret; - - avio_write(pb, buf, sizeof(buf)); - avio_write(pb, extradata, extradata_len); - - return 0; -} - static int mov_write_dvc1_tag(AVIOContext *pb, MOVTrack *track) { const VC1MuxInfo *vc1_info = &track->vc1_info; @@ -1212,7 +1118,7 @@ static int mov_write_dvc1_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "dvc1"); - if ((ret = mov_write_vc1decspecstruc(pb, track->st, vc1_info, + if ((ret = ff_isom_vc1_write_vc1decspecstruc(pb, track->st, vc1_info, track->extradata[track->last_stsd_index], track->extradata_size[track->last_stsd_index])) < 0) return ret; @@ -6496,61 +6402,6 @@ static int mov_parse_mpeg2_frame(AVPacket *pkt, uint32_t *flags) return 0; } -enum VC1MuxAction { - VC1_MUX_ACTION_NONE, ///< No action required - VC1_MUX_ACTION_RESET_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets - VC1_MUX_ACTION_RESET_LATER_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets except the first -}; - -static void ff_vc1_parse_frame(const AVPacket *pkt, VC1MuxInfo *muxinfo, int firstpkt, int *key, enum VC1MuxAction *action) { - const uint8_t *start, *next, *end = pkt->data + pkt->size; - int seq = 0, entry = 0; - start = find_next_marker(pkt->data, end); - for (next = start; next < end; start = next) { - next = find_next_marker(start + 4, end); - switch (AV_RB32(start)) { - case VC1_CODE_SEQHDR: - seq = 1; - break; - case VC1_CODE_ENTRYPOINT: - entry = 1; - break; - case VC1_CODE_SLICE: - muxinfo->slices = 1; - break; - } - } - - *action = VC1_MUX_ACTION_NONE; - - if (firstpkt) { - muxinfo->first_packet_seq = seq; - muxinfo->first_packet_entry = entry; - } else if ((seq && !muxinfo->packet_seq) || - (entry && !muxinfo->packet_entry)) { - *action = VC1_MUX_ACTION_RESET_KEYS; - - if (seq) - muxinfo->packet_seq = 1; - if (entry) - muxinfo->packet_entry = 1; - - if ((!seq || muxinfo->first_packet_seq) && - (!entry || muxinfo->first_packet_entry)) { - /* First packet had the same headers as this one */ - *action = VC1_MUX_ACTION_RESET_LATER_KEYS; - } - } - - *key = pkt->flags & AV_PKT_FLAG_KEY; - if (muxinfo->packet_seq && muxinfo->packet_entry) - *key = seq && entry; - else if (muxinfo->packet_seq) - *key = seq; - else if (muxinfo->packet_entry) - *key = entry; -} - static void mov_parse_vc1_frame(AVPacket *pkt, MOVTrack *trk) { int firstpacket = 0; diff --git a/libavformat/vc1.c b/libavformat/vc1.c new file mode 100644 index 0000000000..7c06e0a27c --- /dev/null +++ b/libavformat/vc1.c @@ -0,0 +1,170 @@ +/* + * VC-1 helper functions for muxers + * Copyright (c) 2026 The FFmpeg Project + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "vc1.h" + +#include "libavcodec/get_bits.h" +#include "libavcodec/put_bits.h" +#include "libavutil/mem.h" +#include "libavcodec/vc1_common.h" + + +void ff_vc1_parse_frame(const AVPacket *pkt, VC1MuxInfo *muxinfo, int firstpkt, int *key, enum VC1MuxAction *action) { + const uint8_t *start, *next, *end = pkt->data + pkt->size; + int seq = 0, entry = 0; + start = find_next_marker(pkt->data, end); + for (next = start; next < end; start = next) { + next = find_next_marker(start + 4, end); + switch (AV_RB32(start)) { + case VC1_CODE_SEQHDR: + seq = 1; + break; + case VC1_CODE_ENTRYPOINT: + entry = 1; + break; + case VC1_CODE_SLICE: + muxinfo->slices = 1; + break; + } + } + + *action = VC1_MUX_ACTION_NONE; + + if (firstpkt) { + muxinfo->first_packet_seq = seq; + muxinfo->first_packet_entry = entry; + } else if ((seq && !muxinfo->packet_seq) || + (entry && !muxinfo->packet_entry)) { + *action = VC1_MUX_ACTION_RESET_KEYS; + + if (seq) + muxinfo->packet_seq = 1; + if (entry) + muxinfo->packet_entry = 1; + + if ((!seq || muxinfo->first_packet_seq) && + (!entry || muxinfo->first_packet_entry)) { + /* First packet had the same headers as this one */ + *action = VC1_MUX_ACTION_RESET_LATER_KEYS; + } + } + + *key = pkt->flags & AV_PKT_FLAG_KEY; + if (muxinfo->packet_seq && muxinfo->packet_entry) + *key = seq && entry; + else if (muxinfo->packet_seq) + *key = seq; + else if (muxinfo->packet_entry) + *key = entry; +} + +static int ff_isom_vc1_write_vc1decspecstruc_fields(const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len, uint8_t *buf) { + uint8_t *unescaped; + const uint8_t *start, *next, *end = extradata + extradata_len; + + int unescaped_size, seq_found = 0; + int level = 0, interlace = 0; + int packet_seq = 1; + int packet_entry = 1; + int slices = 0; + PutBitContext pbc; + + if (muxinfo) { + packet_seq = muxinfo->packet_seq; + packet_entry = muxinfo->packet_entry; + slices = muxinfo->slices; + } + + unescaped = av_mallocz(extradata_len + AV_INPUT_BUFFER_PADDING_SIZE); + if (!unescaped) + return AVERROR(ENOMEM); + start = find_next_marker(extradata, end); + for (next = start; next < end; start = next) { + GetBitContext gb; + int size; + next = find_next_marker(start + 4, end); + size = next - start - 4; + if (size <= 0) + continue; + unescaped_size = vc1_unescape_buffer(start + 4, size, unescaped); + init_get_bits(&gb, unescaped, 8 * unescaped_size); + if (AV_RB32(start) == VC1_CODE_SEQHDR) { + int profile = get_bits(&gb, 2); + if (profile != PROFILE_ADVANCED) { + av_free(unescaped); + return AVERROR(ENOSYS); + } + seq_found = 1; + level = get_bits(&gb, 3); + /* chromaformat, frmrtq_postproc, bitrtq_postproc, postprocflag, + * width, height */ + skip_bits_long(&gb, 2 + 3 + 5 + 1 + 2*12); + skip_bits(&gb, 1); /* broadcast */ + interlace = get_bits1(&gb); + skip_bits(&gb, 4); /* tfcntrflag, finterpflag, reserved, psf */ + } + } + if (!seq_found) { + av_free(unescaped); + return AVERROR(ENOSYS); + } + + init_put_bits(&pbc, buf, 7); + /* VC1DecSpecStruc */ + put_bits(&pbc, 4, 12); /* profile - advanced */ + put_bits(&pbc, 3, level); + put_bits(&pbc, 1, 0); /* reserved */ + /* VC1AdvDecSpecStruc */ + put_bits(&pbc, 3, level); + put_bits(&pbc, 1, 0); /* cbr */ + put_bits(&pbc, 6, 0); /* reserved */ + put_bits(&pbc, 1, !interlace); /* no interlace */ + put_bits(&pbc, 1, !packet_seq); /* no multiple seq */ + put_bits(&pbc, 1, !packet_entry); /* no multiple entry */ + put_bits(&pbc, 1, !slices); /* no slice code */ + put_bits(&pbc, 1, 0); /* no bframe */ + put_bits(&pbc, 1, 0); /* reserved */ + + /* framerate */ + if (st->avg_frame_rate.num > 0 && st->avg_frame_rate.den > 0) + put_bits32(&pbc, st->avg_frame_rate.num / st->avg_frame_rate.den); + else + put_bits32(&pbc, 0xffffffff); + + flush_put_bits(&pbc); + + av_free(unescaped); + + return 0; +} + +int ff_isom_vc1_write_vc1decspecstruc(AVIOContext *pb, const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len) { + uint8_t buf[7] = { 0 }; + int ret; + + if ((ret = ff_isom_vc1_write_vc1decspecstruc_fields(st, muxinfo, extradata, extradata_len, buf)) < 0) + return ret; + + avio_write(pb, buf, sizeof(buf)); + avio_write(pb, extradata, extradata_len); + + return 0; +} diff --git a/libavformat/vc1.h b/libavformat/vc1.h index 276b9e624c..d725612160 100644 --- a/libavformat/vc1.h +++ b/libavformat/vc1.h @@ -22,6 +22,19 @@ #ifndef AVFORMAT_VC1_H #define AVFORMAT_VC1_H +#include "avformat.h" +#include "avio.h" + +enum VC1MuxAction { + VC1_MUX_ACTION_NONE, ///< No action required + VC1_MUX_ACTION_RESET_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets + VC1_MUX_ACTION_RESET_LATER_KEYS, ///< Found sequence headers or entry points that were not found before; reset the keyframe flag of all previous packets except the first +}; + +/** + * Data collected by calling ff_vc1_parse_frame on every packet of a vc1 stream, + * which can then be passed to ff_isom_vc1_write_vc1decspecstruct. + */ typedef struct VC1MuxInfo { int first_packet_seq; ///< Whether the first packet has a sequence header int first_packet_entry; ///< Whether the first packet has an entry point @@ -30,4 +43,14 @@ typedef struct VC1MuxInfo { int slices; ///< Whether there exist packets containing slices } VC1MuxInfo; +void ff_vc1_parse_frame(const AVPacket *pkt, VC1MuxInfo *muxinfo, int firstpkt, int *key, enum VC1MuxAction *action); + +/** + * muxinfo should be a VC1MuxInfo struct that has been collected by calling ff_vc1_parse_frame on every packet of the stream. + * muxinfo *can* be NULL, in which case fallback values will be written. + * + * It is guaranteed that the number of bytes written by this function is independendent of the value of muxinfo and *muxinfo. + */ +int ff_isom_vc1_write_vc1decspecstruc(AVIOContext *pb, const AVStream *st, const VC1MuxInfo *muxinfo, const uint8_t *extradata, int extradata_len); + #endif /* AVFORMAT_AVC_H */ -- 2.52.0 >From bdf358e24d5aeba09778309a12631852360dfcce Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Fri, 5 Jun 2026 00:14:23 +0200 Subject: [PATCH 5/6] avformat/matroskaenc: Implement native VC-1 muxing This requires plumbing some extra data through a couple of function calls since muxing VC-1 needs some extra information and state. Ignore the VC1MuxAction for now since retroactively removing all previous keyframe flags would require adding a lot of extra bookkeeping. --- libavformat/matroska.c | 1 + libavformat/matroskaenc.c | 54 +++++++++++++++++++++++++++++++-------- 2 files changed, 44 insertions(+), 11 deletions(-) diff --git a/libavformat/matroska.c b/libavformat/matroska.c index 60584e2687..6bd91b219f 100644 --- a/libavformat/matroska.c +++ b/libavformat/matroska.c @@ -101,6 +101,7 @@ const CodecTags ff_mkv_codec_tags[]={ {"V_SNOW" , AV_CODEC_ID_SNOW}, {"V_THEORA" , AV_CODEC_ID_THEORA}, {"V_UNCOMPRESSED" , AV_CODEC_ID_RAWVIDEO}, + {"V_VC1" , AV_CODEC_ID_VC1}, {"V_VP8" , AV_CODEC_ID_VP8}, {"V_VP9" , AV_CODEC_ID_VP9}, diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index c9386db6a0..be07813334 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -38,6 +38,7 @@ #include "matroska.h" #include "mux.h" #include "riff.h" +#include "vc1.h" #include "version.h" #include "vorbiscomment.h" #include "vvc.h" @@ -194,6 +195,7 @@ typedef struct mkv_track { int sample_rate; unsigned offset; int64_t sample_rate_offset; + int first_pkt_seen; int64_t last_timestamp; int64_t duration; int64_t duration_offset; @@ -204,6 +206,7 @@ typedef struct mkv_track { int64_t ts_offset; uint64_t default_duration_low; uint64_t default_duration_high; + VC1MuxInfo vc1_info; /* This callback will be called twice: First with a NULL AVIOContext * to return the size of the (Simple)Block's data via size * and a second time with the AVIOContext set when the data @@ -1148,11 +1151,13 @@ static int get_aac_sample_rates(AVFormatContext *s, MatroskaMuxContext *mkv, #endif static int mkv_assemble_native_codecprivate(AVFormatContext *s, AVIOContext *dyn_cp, - const AVCodecParameters *par, + const AVStream *st, const mkv_track *track, const uint8_t *extradata, int extradata_size, unsigned *size_to_reserve) { + AVCodecParameters *const par = st->codecpar; + switch (par->codec_id) { case AV_CODEC_ID_VORBIS: case AV_CODEC_ID_THEORA: @@ -1178,6 +1183,10 @@ static int mkv_assemble_native_codecprivate(AVFormatContext *s, AVIOContext *dyn case AV_CODEC_ID_VVC: return ff_isom_write_vvcc(dyn_cp, extradata, extradata_size, 0); + case AV_CODEC_ID_VC1: + return ff_isom_vc1_write_vc1decspecstruc(dyn_cp, st, + track->first_pkt_seen ? &track->vc1_info : NULL, + extradata, extradata_size); case AV_CODEC_ID_ALAC: if (extradata_size < 36) { av_log(s, AV_LOG_ERROR, @@ -1228,18 +1237,19 @@ static int mkv_assemble_native_codecprivate(AVFormatContext *s, AVIOContext *dyn } static int mkv_assemble_codecprivate(AVFormatContext *s, AVIOContext *dyn_cp, - AVCodecParameters *par, + AVStream *st, const mkv_track *track, const uint8_t *extradata, int extradata_size, int native_id, int qt_id, uint8_t **codecpriv, int *codecpriv_size, unsigned *max_payload_size) { + AVCodecParameters *const par = st->codecpar; av_unused MatroskaMuxContext *const mkv = s->priv_data; unsigned size_to_reserve = 0; int ret; if (native_id) { - ret = mkv_assemble_native_codecprivate(s, dyn_cp, par, + ret = mkv_assemble_native_codecprivate(s, dyn_cp, st, track, extradata, extradata_size, &size_to_reserve); if (ret < 0) @@ -1333,15 +1343,16 @@ static void mkv_put_codecprivate(AVIOContext *pb, unsigned max_payload_size, static int mkv_update_codecprivate(AVFormatContext *s, MatroskaMuxContext *mkv, uint8_t *side_data, int side_data_size, - AVCodecParameters *par, AVIOContext *pb, + AVStream *st, AVIOContext *pb, mkv_track *track, unsigned alternative_size) { + AVCodecParameters *const par = st->codecpar; AVIOContext *const dyn_bc = mkv->tmp_bc; uint8_t *codecpriv; unsigned max_payload_size; int ret, codecpriv_size; - ret = mkv_assemble_codecprivate(s, dyn_bc, par, + ret = mkv_assemble_codecprivate(s, dyn_bc, st, track, side_data, side_data_size, 1, 0, &codecpriv, &codecpriv_size, &max_payload_size); if (ret < 0) @@ -2173,7 +2184,7 @@ static int mkv_write_track(AVFormatContext *s, MatroskaMuxContext *mkv, uint8_t *codecpriv; int codecpriv_size, max_payload_size; track->codecpriv_offset = avio_tell(pb); - ret = mkv_assemble_codecprivate(s, mkv->tmp_bc, par, + ret = mkv_assemble_codecprivate(s, mkv->tmp_bc, st, track, par->extradata, par->extradata_size, native_id, qt_id, &codecpriv, &codecpriv_size, &max_payload_size); @@ -3029,7 +3040,8 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) { MatroskaMuxContext *mkv = s->priv_data; mkv_track *track = &mkv->tracks[pkt->stream_index]; - AVCodecParameters *par = s->streams[pkt->stream_index]->codecpar; + AVStream *const st = s->streams[pkt->stream_index]; + AVCodecParameters *const par = st->codecpar; uint8_t *side_data; size_t side_data_size; int ret; @@ -3049,7 +3061,7 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) if (!output_sample_rate) output_sample_rate = track->sample_rate; // Space is already reserved, so it's this or a void element. ret = mkv_update_codecprivate(s, mkv, side_data, side_data_size, - par, mkv->track.bc, track, 0); + st, mkv->track.bc, track, 0); if (ret < 0) return ret; avio_seek(mkv->track.bc, track->sample_rate_offset, SEEK_SET); @@ -3069,7 +3081,7 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) return AVERROR(EINVAL); } ret = mkv_update_codecprivate(s, mkv, side_data, side_data_size, - par, mkv->track.bc, track, 0); + st, mkv->track.bc, track, 0); if (ret < 0) return ret; } @@ -3082,7 +3094,7 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) // If the reserved space doesn't suffice, only write // the first four bytes of the av1C. ret = mkv_update_codecprivate(s, mkv, side_data, side_data_size, - par, mkv->track.bc, track, 4); + st, mkv->track.bc, track, 4); if (ret < 0) return ret; } else if (!par->extradata_size) @@ -3176,6 +3188,7 @@ static int mkv_write_packet_internal(AVFormatContext *s, const AVPacket *pkt) static int mkv_write_packet(AVFormatContext *s, const AVPacket *pkt) { MatroskaMuxContext *mkv = s->priv_data; + mkv_track *const track = &mkv->tracks[pkt->stream_index]; int codec_type = s->streams[pkt->stream_index]->codecpar->codec_type; int keyframe = !!(pkt->flags & AV_PKT_FLAG_KEY); int cluster_size; @@ -3183,12 +3196,20 @@ static int mkv_write_packet(AVFormatContext *s, const AVPacket *pkt) int ret; int start_new_cluster; + int first_pkt = !track->first_pkt_seen; + track->first_pkt_seen = 1; + ret = mkv_check_new_extra_data(s, pkt); if (ret < 0) return ret; + if (s->streams[pkt->stream_index]->codecpar->codec_id == AV_CODEC_ID_VC1) { + enum VC1MuxAction action; + ff_vc1_parse_frame(pkt, &track->vc1_info, first_pkt, &keyframe, &action); + } + if (mkv->cluster_pos != -1) { - if (mkv->tracks[pkt->stream_index].write_dts) + if (track->write_dts) cluster_time = pkt->dts - mkv->cluster_pts; else cluster_time = pkt->pts - mkv->cluster_pts; @@ -3289,6 +3310,17 @@ static int mkv_write_trailer(AVFormatContext *s) return ret; } + for (unsigned i = 0; i < s->nb_streams; i++) { + AVStream *const st = s->streams[i]; + + // VC-1's CodecPrivate needs information that must be collected from the stream's packets, + // so rewrite the CodecPrivate now that all packets were seen. + if (st->codecpar->codec_id == AV_CODEC_ID_VC1) { + mkv_update_codecprivate(s, mkv, st->codecpar->extradata, st->codecpar->extradata_size, st, + mkv->track.bc, &mkv->tracks[i], 0); + } + } + ret = mkv_write_chapters(s); if (ret < 0) return ret; -- 2.52.0 >From acdde27f60cc185e4bd09e37ad28ff5175f83021 Mon Sep 17 00:00:00 2001 From: arch1t3cht <[email protected]> Date: Fri, 5 Jun 2026 13:54:39 +0200 Subject: [PATCH 6/6] avformat/matroskadec: Support demuxing native VC-1 streams Do not extract mov_read_dvc1 into a separate generic function this time since it's fairly simple and works very differently from matroskadec's mkv_parse_video_codec (with one using AVIO and copying the extradata directly and the other using an existing buffer and only returning the extradata offset), but add comments so that the two implementations can be kept in sync. --- libavformat/matroskadec.c | 14 ++++++++++++++ libavformat/mov.c | 1 + 2 files changed, 15 insertions(+) diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c index 965674bc3b..8825f0fa92 100644 --- a/libavformat/matroskadec.c +++ b/libavformat/matroskadec.c @@ -2972,6 +2972,20 @@ static int mkv_parse_video_codec(MatroskaTrack *track, AVCodecParameters *par, if (track->codec_priv.size == 4) par->codec_tag = AV_RL32(track->codec_priv.data); break; + case AV_CODEC_ID_VC1: + // See also: mov_read_dvc1 + if (track->codec_priv.size < 7) + return AVERROR_INVALIDDATA; + + uint8_t profile_level = track->codec_priv.data[0]; + if ((profile_level & 0xf0) != 0xc0) { + track->codec_priv.size = 0; + break; + } + + *extradata_offset = 7; + + break; case AV_CODEC_ID_VP9: /* we don't need any value stored in CodecPrivate. * make sure that it's not exported as extradata. */ diff --git a/libavformat/mov.c b/libavformat/mov.c index 436ca415c2..69def7aacc 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -2545,6 +2545,7 @@ static int mov_read_dvc1(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (atom.size >= (1<<28) || atom.size < 7) return AVERROR_INVALIDDATA; + // See also: mkv_parse_video_codec profile_level = avio_r8(pb); if ((profile_level & 0xf0) != 0xc0) return 0; -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
