PR #23671 opened by WyattBlue URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23671 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23671.patch
With language=auto, whisper reruns language detection on every queue chunk, an extra encoder pass that can also drift to a different language mid-file. Cache the first detected language so subsequent chunks reuse it. Only lock once a chunk produces segments, so a silent or musical opening chunk does set the detected language. Improves multilingual-model performance by ~33% >From 0bf16d65d85ca0b330c39ecb3a11f32c885489fe Mon Sep 17 00:00:00 2001 From: WyattBlue <[email protected]> Date: Wed, 1 Jul 2026 00:33:54 -0400 Subject: [PATCH] avfilter/af_whisper: Reuse detected language across chunks With language=auto, whisper reruns language detection on every queue chunk, an extra encoder pass that can also drift to a different language mid-file. Cache the first detected language so subsequent chunks reuse it. Only lock once a chunk produces segments, so a silent or musical opening chunk does set the detected language. --- libavfilter/af_whisper.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/libavfilter/af_whisper.c b/libavfilter/af_whisper.c index 75103ced0c..77250274b7 100644 --- a/libavfilter/af_whisper.c +++ b/libavfilter/af_whisper.c @@ -233,6 +233,19 @@ static void run_transcription(AVFilterContext *ctx, AVFrame *frame, int samples) const int n_segments = whisper_full_n_segments(wctx->ctx_wsp); char *segments_text = NULL; + if (n_segments > 0 && !av_strcasecmp(wctx->language, "auto")) { + const int lang_id = whisper_full_lang_id(wctx->ctx_wsp); + if (lang_id >= 0) { + char *detected = av_strdup(whisper_lang_str(lang_id)); + if (detected) { + av_freep(&wctx->language_str); + wctx->language_str = detected; + wctx->language = detected; + av_log(ctx, AV_LOG_INFO, "Locked auto-detected language: %s\n", detected); + } + } + } + for (int i = 0; i < n_segments; ++i) { const char *text = whisper_full_get_segment_text(wctx->ctx_wsp, i); if (av_isspace(text[0])) -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
