PR #25524 opened by EthanSK URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25524 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25524.patch
## Summary of changes An input seek can miss early audio when the selected automatic reference is a video track that starts later. This draft exposes the existing seek API's stream index through optional input `-seek_stream a:0` (or an absolute stream index). The selector must match exactly one probed input stream. The CLI rescales the existing seek timestamp into its time base, rounding down to preserve the upper seek bound. Omitting the option preserves the existing path. Documentation, a changelog entry and two self-contained FATE tests are included. Related: #25223. Submitted as a small sample implementation for discussion of the interface and scope. ## Validation Built unpatched and patched FFmpeg on macOS arm64 against upstream `86444dd36834875f274f57705762c5448379d644`. The issue's H.264/AAC fixture was generated using that unpatched build. Extracting 1 second of mono 48 kHz audio at `-ss 1` produced: | Command | PCM samples | | --- | ---: | | Unpatched input seek | 0 | | Unpatched read-and-trim | 48,000 | | Patched input seek, option omitted | 0 | | Patched input seek, `-seek_stream a:0` | 48,000 | - All 629 FATE tests enabled by the restricted build passed, including the two added cases. This was not a full-feature build or an external-sample FATE run. - Additional offline synthetic checks covered PCM/MOV and AAC/MP4, delayed video including B-frames, two audio streams with different sample rates/start times, two video streams, nonzero file starts, fractional seeks, `-sseof`, `-seek_timestamp`, `-copyts`, invalid/unmatched/ambiguous selectors, and unchanged default output. Focused loop and video stream-copy comparisons also matched their unpatched counterparts. - `git diff --check` passed. `tools/patcheck` ran with GNU tools; its Makefile-tab, documentation and address-passed-variable hints were inspected. ## Boundaries This only controls the initial `-ss`/`-sseof` seek. `-stream_loop` seeks and output mapping are unchanged. There is no new public API or demuxer change, and this does not implement independent per-track preroll from #24105. The selected reference remains subject to demuxer capabilities and can affect other tracks' decodability. Restored audio length does not guarantee bit-identical output to decoding from the beginning; AAC decoder state and resampling can differ. No performance benchmark or all-demuxer claim is made. >From b88c75759e6f60bab2fc3086395b6e6a69d48b37 Mon Sep 17 00:00:00 2001 From: EthanSK <[email protected]> Date: Tue, 27 Oct 2026 14:20:55 +0000 Subject: [PATCH] fftools/ffmpeg: add an input seek_stream option Allow the initial -ss/-sseof seek to use an explicitly selected input stream. Resolve the existing stream specifier syntax to exactly one stream, and rescale the seek timestamp into that stream's time base, rounding down to preserve the upper seek bound. Keep the automatic reference and stream-loop seeks unchanged when the option is omitted. This exposes existing demuxer seek control; it does not change output mapping or promise independent seeking of all tracks. An MP4 with audio from 0s and video from 3s returns no samples for an input seek to 1s followed by a 1s audio extraction. With -seek_stream a:0, the same command returns the expected 48000 samples. Add self-contained FATE cases for a:0 and an absolute stream index using native MPEG-4 video and PCM audio in MOV, without external samples. Related: https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/25223 Signed-off-by: Ethan Sarif-Kattan <[email protected]> --- Changelog | 1 + doc/ffmpeg.texi | 13 ++++++++ fftools/ffmpeg.h | 1 + fftools/ffmpeg_demux.c | 33 ++++++++++++++++++++- fftools/ffmpeg_opt.c | 3 ++ tests/fate/ffmpeg.mak | 18 +++++++++++ tests/ref/fate/ffmpeg-seek-stream-index | 7 +++++ tests/ref/fate/ffmpeg-seek-stream-specifier | 7 +++++ 8 files changed, 82 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/ffmpeg-seek-stream-index create mode 100644 tests/ref/fate/ffmpeg-seek-stream-specifier diff --git a/Changelog b/Changelog index 08a79e92067..adce811cfef 100644 --- a/Changelog +++ b/Changelog @@ -3,6 +3,7 @@ releases are sorted from youngest to oldest. version <next>: - DTS:X height channels in the DCA decoder +- ffmpeg -seek_stream option to select the initial input seek reference stream version 9.1: - ASTC image encoding and decoding via libastcenc, with .astc and KTX 1.0 muxers and demuxers diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index d31678f8d8c..256885a4711 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -2847,6 +2847,19 @@ to the @option{-ss} option is considered an actual timestamp, and is not offset by the start time of the file. This matters only for files which do not start from timestamp 0, such as transport streams. +@item -seek_stream @var{stream_specifier} (@emph{input}) +Select the reference stream for the initial input seek requested by +@option{-ss} or @option{-sseof}. The stream specifier must match exactly one +stream in this input, for example @code{a:0} for its first audio stream or +@code{1} for its stream with absolute index 1. Invalid specifiers and +specifiers matching no streams or multiple streams cause an error. + +By default, the demuxer selects the reference stream automatically. Selecting +an output stream with @option{-map} does not select the seek reference. +This option does not change the timestamp origin, output stream selection, +or seeks performed by @option{-stream_loop}. Seeking remains subject to the +demuxer's capabilities and may affect the decodability of other streams. + @item -thread_queue_size @var{size} (@emph{output}) This option specifies the maximum number of packets per stream that may be queued to each muxing thread. diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index f51867435da..b40d8bfdc9f 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -155,6 +155,7 @@ typedef struct OptionsContext { float readrate_catchup; double readrate_initial_burst; int accurate_seek; + const char *seek_stream; int thread_queue_size; int input_sync_ref; int find_stream_info; diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 5409dcdc6c0..dcd54673c3f 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -2329,6 +2329,7 @@ int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) const char* subtitle_codec_name = NULL; const char* data_codec_name = NULL; int scan_all_pmts_set = 0; + int seek_stream = -1; int64_t start_time = o->start_time; int64_t start_time_eof = o->start_time_eof; @@ -2517,6 +2518,32 @@ int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) } } + if (o->seek_stream) { + StreamSpecifier ss; + + ret = stream_specifier_parse(&ss, o->seek_stream, 0, d); + if (ret < 0) + return ret; + + for (int i = 0; i < ic->nb_streams; i++) { + if (!stream_specifier_match(&ss, ic, ic->streams[i], d)) + continue; + if (seek_stream >= 0) { + av_log(d, AV_LOG_ERROR, "Seek stream specifier '%s' matches multiple streams.\n", + o->seek_stream); + stream_specifier_uninit(&ss); + return AVERROR(EINVAL); + } + seek_stream = i; + } + stream_specifier_uninit(&ss); + if (seek_stream < 0) { + av_log(d, AV_LOG_ERROR, "Seek stream specifier '%s' matches no streams.\n", + o->seek_stream); + return AVERROR(EINVAL); + } + } + if (start_time != AV_NOPTS_VALUE && start_time_eof != AV_NOPTS_VALUE) { av_log(d, AV_LOG_WARNING, "Cannot use -ss and -sseof both, using -ss\n"); start_time_eof = AV_NOPTS_VALUE; @@ -2558,7 +2585,11 @@ int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) seek_timestamp -= 3*AV_TIME_BASE / 23; } } - ret = avformat_seek_file(ic, -1, INT64_MIN, seek_timestamp, seek_timestamp, 0); + if (seek_stream >= 0) + seek_timestamp = av_rescale_q_rnd(seek_timestamp, AV_TIME_BASE_Q, + ic->streams[seek_stream]->time_base, + AV_ROUND_DOWN); + ret = avformat_seek_file(ic, seek_stream, INT64_MIN, seek_timestamp, seek_timestamp, 0); if (ret < 0) { av_log(d, AV_LOG_WARNING, "could not seek to position %0.3f\n", (double)timestamp / AV_TIME_BASE); diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 265bfba35fe..9339e064ea2 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -1616,6 +1616,9 @@ const OptionDef options[] = { { "accurate_seek", OPT_TYPE_BOOL, OPT_OFFSET | OPT_EXPERT | OPT_INPUT, { .off = OFFSET(accurate_seek) }, "enable/disable accurate seeking with -ss" }, + { "seek_stream", OPT_TYPE_STRING, OPT_OFFSET | OPT_EXPERT | OPT_INPUT, + { .off = OFFSET(seek_stream) }, + "set the reference stream for initial input seeking", "stream_specifier" }, { "isync", OPT_TYPE_INT, OPT_OFFSET | OPT_EXPERT | OPT_INPUT, { .off = OFFSET(input_sync_ref) }, "Indicate the input index for sync reference", "sync ref" }, diff --git a/tests/fate/ffmpeg.mak b/tests/fate/ffmpeg.mak index ff5dfdd4c84..3c038ca3bd0 100644 --- a/tests/fate/ffmpeg.mak +++ b/tests/fate/ffmpeg.mak @@ -16,6 +16,24 @@ fate-ffmpeg-filter_colorkey: CMD = framecrc -auto_conversion_filters -idct simpl FATE_FFMPEG-$(call FILTERFRAMECRC, COLOR) += fate-ffmpeg-lavfi fate-ffmpeg-lavfi: CMD = framecrc -lavfi color=d=1:r=5 -fflags +bitexact +tests/data/seek-stream.mov: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin -y -v error \ + -itsoffset 2 -f lavfi -i "color=s=16x16:r=10:d=1" \ + -f lavfi -i "sine=sample_rate=8000:d=3" \ + -map 0:v -map 1:a -c:v mpeg4 -threads:v 1 -g 10 -bf 0 \ + -flags:v +bitexact -fps_mode passthrough -c:a pcm_s16le \ + -fflags +bitexact -f mov $@ 2>/dev/null + +FATE_SEEK_STREAM = fate-ffmpeg-seek-stream-specifier fate-ffmpeg-seek-stream-index +FATE_FFMPEG-$(call ALLYES, LAVFI_INDEV COLOR_FILTER SINE_FILTER WRAPPED_AVFRAME_DECODER \ + PCM_S16LE_DECODER PCM_S16LE_ENCODER MPEG4_ENCODER MOV_MUXER \ + MOV_DEMUXER FRAMECRC_MUXER ARESAMPLE_FILTER) += $(FATE_SEEK_STREAM) +$(FATE_SEEK_STREAM): tests/data/seek-stream.mov +$(FATE_SEEK_STREAM): CMD = framecrc -ss 0.5 -seek_stream $(SEEK_STREAM) \ + -i $(TARGET_PATH)/tests/data/seek-stream.mov -map 0:a:0 -t 0.1 -c:a pcm_s16le +fate-ffmpeg-seek-stream-specifier: SEEK_STREAM = a:0 +fate-ffmpeg-seek-stream-index: SEEK_STREAM = 1 + FATE_FFMPEG-$(call FILTERFRAMECRC, TESTSRC2 FORMAT SETPARAMS SCALE, LAVFI_INDEV WRAPPED_AVFRAME_DECODER MPEG2VIDEO_ENCODER) += fate-ffmpeg-chroma-location fate-ffmpeg-chroma-location: CMD = framecrc -auto_conversion_filters -f lavfi -i testsrc2=s=64x64:r=25:d=1,format=yuv420p,setparams=chroma_location=topleft -sws_flags +accurate_rnd+bitexact -flags +bitexact -idct simple -dct fastint -threads 1 -c:v mpeg2video -qscale:v 2 -frames:v 1 diff --git a/tests/ref/fate/ffmpeg-seek-stream-index b/tests/ref/fate/ffmpeg-seek-stream-index new file mode 100644 index 00000000000..2907b971c63 --- /dev/null +++ b/tests/ref/fate/ffmpeg-seek-stream-index @@ -0,0 +1,7 @@ +#tb 0: 1/8000 +#media_type 0: audio +#codec_id 0: pcm_s16le +#sample_rate 0: 8000 +#channel_layout_name 0: mono +0, 0, 0, 96, 192, 0x85875b8e +0, 96, 96, 704, 1408, 0x8d8eb30c diff --git a/tests/ref/fate/ffmpeg-seek-stream-specifier b/tests/ref/fate/ffmpeg-seek-stream-specifier new file mode 100644 index 00000000000..2907b971c63 --- /dev/null +++ b/tests/ref/fate/ffmpeg-seek-stream-specifier @@ -0,0 +1,7 @@ +#tb 0: 1/8000 +#media_type 0: audio +#codec_id 0: pcm_s16le +#sample_rate 0: 8000 +#channel_layout_name 0: mono +0, 0, 0, 96, 192, 0x85875b8e +0, 96, 96, 704, 1408, 0x8d8eb30c -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
