PR #25486 opened by Martin Storsjö (mstorsjo) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25486 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25486.patch
This is the way Fraunhofer recommends using the library - in single pass mode, there's always a compromise about having to set a global loudness level ahead of time. I'm including a second commit that removes the default value for `-live_loudness_level`, forcing the users to either consider using two pass encoding, or manually specify `-live_loudness_level`. But if people thinks that's too inconvenient for users we can keep the default single pass mode as well. See the first commit message for more elaborate explanation of the rationale for how this is hooked up; AVCodecContext contains fields for importing/exporting two pass stats, but they only work for strings, not opaque binary data. And the ffmpeg CLI tool only supports using these fields for video encoders at the moment. With the chosen approach, it's still possible to wrap it up nicely in the ffmpeg CLI tool later if we want to (libx264 and libx265 currently also use a private option for the two pass input/output file). From 3a304d74358e21a83c178d3c62649174e6dd829c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20Storsj=C3=B6?= <[email protected]> Date: Mon, 26 Oct 2026 14:50:38 +0200 Subject: [PATCH 1/2] avcodec/libfdk-xhe-aacenc: Add support for two pass encoding The so far default one pass encoding requires a hardcoded loudness level parameter (which defaults to -24 dB). By doing two pass encoding, the first pass measures the loudness in order to be able to preserve the dynamics of the stream properly. This uses a private option for the input/output file name rather than the existing "stats_in" and "stats_out" fields in AVCodecContext. Those fields are strings, which are implicitly null terminated, while the loudness data for this encoder is a binary blob, where we can't guarantee that don't contain null bytes. Additionally, the automatic two pass support in the ffmpeg command line binary (for handling the "-pass 1" and "-pass 2" options, for automatically picking a temp file name, storing "stats_out" in that file, and reading from the file into "stats_in") currently only supports this for video encoders, not for audio encoders. This uses the same fields for setting it up (the flags AV_CODEC_FLAG_PASS1 and AV_CODEC_FLAG_PASS2), and uses a private option for the input/output file (just like libx264 and libx265 do), so it's entirely possible to wrap it up more conveniently in the ffmpeg tool later if we want to. --- doc/encoders.texi | 14 +++ libavcodec/libfdk-xhe-aacenc.c | 178 +++++++++++++++++++++++++++++++-- libavcodec/version.h | 4 +- 3 files changed, 188 insertions(+), 8 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index b111ca7c9b0..5fe35b78dd2 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -862,8 +862,22 @@ is embedded into the encoded bitstream. If not set, the encoder tries to generate a hopefully unique ID for the stream. + +@item loudness_file +The path to a file for storing the loudness information for the stream, +for two-pass encoding. @end table +In order to do two pass encoding (where the first pass measures the +loudness of the stream, and the second pass uses that information +for accurately preserving the dynamics of the stream), invoke ffmpeg +e.g. like this: + +@example +ffmpeg -i input.wav -c:a libfdk_xhe_aac -flags:a +pass1 -loudness_file loudness.dat -f null - +ffmpeg -i input.wav -c:a libfdk_xhe_aac -flags:a +pass2 -loudness_file loudness.dat out.m4a +@end example + @anchor{liblc3-enc} @section liblc3 diff --git a/libavcodec/libfdk-xhe-aacenc.c b/libavcodec/libfdk-xhe-aacenc.c index fec9694b7cd..25b89384ed2 100644 --- a/libavcodec/libfdk-xhe-aacenc.c +++ b/libavcodec/libfdk-xhe-aacenc.c @@ -45,6 +45,11 @@ typedef struct XHEAACContext { float *sample_queue; int sample_queue_size; int nb_sample_queue; + + IIS_XHEAACENC_LOUDNESS_INSTANCE_HANDLE loudness_handle; + + char *loudness_file; + int loudness_exported; } XHEAACContext; static const AVOption xhe_aac_enc_options[] = { @@ -52,6 +57,7 @@ static const AVOption xhe_aac_enc_options[] = { { "live_loudness_level", "target loudness level in live encoding mode [LUFS]", offsetof(XHEAACContext, live_loudness_level), AV_OPT_TYPE_FLOAT, { .dbl = -24 }, -31.0, -10.0, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, { "rap_interval", "Random Access Point interval", offsetof(XHEAACContext, rap_interval), AV_OPT_TYPE_DURATION, { .i64 = -1 }, -1, INT64_MAX, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, { "stream_id", "unique ID of this stream within one adaptation set", offsetof(XHEAACContext, stream_id), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 65535, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, + { "loudness_file", "Filename for 2 pass loudness", offsetof(XHEAACContext, loudness_file), AV_OPT_TYPE_STRING, { 0 }, 0, 0, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, FF_AAC_PROFILE_OPTS { NULL } }; @@ -99,12 +105,105 @@ static av_cold int xhe_aac_encode_close(AVCodecContext *avctx) xhe_aac_get_error(err), err); s->handle = NULL; } + if (s->loudness_handle) { + err = IIS_xHEAACEnc_Loudness_Delete(s->loudness_handle); + if (err != IIS_XHEAACENC_NO_ERROR) + av_log(avctx, AV_LOG_ERROR, "Unexpected error on loudness close: %s (%d)\n", + xhe_aac_get_error(err), err); + s->loudness_handle = NULL; + } ff_af_queue_close(&s->afq); av_freep(&s->sample_queue); return 0; } +static av_cold int xhe_aac_write_loudness_file(AVCodecContext *avctx) +{ + XHEAACContext *s = avctx->priv_data; + FILE *out = NULL; + unsigned int size; + const uint8_t *buf = NULL; + int ret; + IIS_XHEAACENC_RETURN_CODE err; + + err = IIS_xHEAACEnc_Loudness_Export(s->loudness_handle, &buf, + &size); + if (err != IIS_XHEAACENC_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unable to export loudness: %s (%d)\n", + xhe_aac_get_error(err), err); + return AVERROR(EINVAL); + } + + out = fopen(s->loudness_file, "wb"); + if (!out) { + ret = AVERROR(errno); + av_log(avctx, AV_LOG_ERROR, "Unable to open %s: %s\n", + s->loudness_file, av_err2str(ret)); + return ret; + } + if (fwrite(buf, 1, size, out) != size) { + ret = AVERROR(EIO); + goto error; + } + + ret = 0; + +error: + if (out) + fclose(out); + return ret; +} + +static av_cold int xhe_aac_read_loudness_file(AVCodecContext *avctx) +{ + XHEAACContext *s = avctx->priv_data; + FILE *in = NULL; + size_t size; + uint8_t *buf = NULL; + int ret; + IIS_XHEAACENC_RETURN_CODE err; + + in = fopen(s->loudness_file, "rb"); + if (!in) { + ret = AVERROR(errno); + av_log(avctx, AV_LOG_ERROR, "Unable to open %s: %s\n", + s->loudness_file, av_err2str(ret)); + return ret; + } + fseek(in, 0, SEEK_END); + size = ftell(in); + fseek(in, 0, SEEK_SET); + + buf = av_malloc(size); + if (!buf) { + ret = AVERROR(ENOMEM); + goto error; + } + if (fread(buf, 1, size, in) != size) { + ret = AVERROR(EIO); + goto error; + } + fclose(in); + in = NULL; + + err = IIS_xHEAACEnc_Loudness_Import(s->loudness_handle, buf, size); + if (err != IIS_XHEAACENC_NO_ERROR) { + ret = AVERROR(EINVAL); + av_log(avctx, AV_LOG_ERROR, "Unable to import loudness data: %s (%d)\n", + xhe_aac_get_error(err), err); + goto error; + } + + ret = 0; + +error: + av_free(buf); + if (in) + fclose(in); + return ret; +} + static av_cold int xhe_aac_encode_init(AVCodecContext *avctx) { XHEAACContext *s = avctx->priv_data; @@ -131,6 +230,43 @@ static av_cold int xhe_aac_encode_init(AVCodecContext *avctx) goto error; } + if (avctx->flags & (AV_CODEC_FLAG_PASS1 | AV_CODEC_FLAG_PASS2)) { + IIS_XHEAACENC_LOUDNESS_SETUP loudness_setup; + + loudness_setup.sampleRate = avctx->sample_rate; + loudness_setup.channelConfig = channel_config; + loudness_setup.audioInputLengthSamples = 0; + loudness_setup.audioInputLengthAvailable = 0; + + err = IIS_xHEAACEnc_Loudness_Open(&s->loudness_handle, loudness_setup); + if (err != IIS_XHEAACENC_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unable to create loudness handle: %s (%d)\n", + xhe_aac_get_error(err), err); + goto error; + } + + if (!s->loudness_file) { + av_log(avctx, AV_LOG_ERROR, + "Must specify loudness file for 2 pass encoding\n"); + goto error; + } + + if (avctx->flags & AV_CODEC_FLAG_PASS1) { + /* Arbitrary frame size for the first pass. */ + avctx->frame_size = avctx->sample_rate * 25 / 1000; + /* For the first pass, we don't need to set up the + * encoder or anything - we just need the loudness handle. */ + return 0; + } + + /* For pass 2, import the loudness data from the previous run. */ + int read_ret = xhe_aac_read_loudness_file(avctx); + if (read_ret != 0) { + ret = read_ret; + goto error; + } + } + err = IIS_xHEAACEnc_Config_Open(&config_handle, avctx->sample_rate, channel_config); if (err != IIS_XHEAACENC_NO_ERROR) { av_log(avctx, AV_LOG_ERROR, "Unable to open config: %s (%d)\n", @@ -237,12 +373,23 @@ static av_cold int xhe_aac_encode_init(AVCodecContext *avctx) goto error; } } - err = IIS_xHEAACEnc_Config_AddParamValueFloat(config_handle, - IIS_XHEAACENC_PARAMETER_LIVE_LOUDNESS_LEVEL, s->live_loudness_level); - if (err != IIS_XHEAACENC_NO_ERROR) { - av_log(avctx, AV_LOG_ERROR, "Unable to set live loudness level: %s (%d)\n", - xhe_aac_get_error(err), err); - goto error; + + if (s->loudness_handle) { + err = IIS_xHEAACEnc_Config_AddParamValuePointer(config_handle, + IIS_XHEAACENC_PARAMETER_LOUDNESS_DATA, s->loudness_handle); + if (err != IIS_XHEAACENC_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unable to set loudness object: %s (%d)\n", + xhe_aac_get_error(err), err); + goto error; + } + } else { + err = IIS_xHEAACEnc_Config_AddParamValueFloat(config_handle, + IIS_XHEAACENC_PARAMETER_LIVE_LOUDNESS_LEVEL, s->live_loudness_level); + if (err != IIS_XHEAACENC_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unable to set live loudness level: %s (%d)\n", + xhe_aac_get_error(err), err); + goto error; + } } err = IIS_xHEAACEnc_Config_Finalize(config_handle); @@ -369,6 +516,25 @@ static int xhe_aac_encode_frame(AVCodecContext *avctx, AVPacket *avpkt, int samples_next; int ret; + if (avctx->flags & AV_CODEC_FLAG_PASS1) { + *got_packet_ptr = 0; + + if (frame) { + err = IIS_xHEAACEnc_Loudness_Measure(s->loudness_handle, + (const float *)frame->data[0], + frame->nb_samples * avctx->ch_layout.nb_channels); + if (err != IIS_XHEAACENC_NO_ERROR) { + av_log(avctx, AV_LOG_ERROR, "Unable to measure loudness: %s (%d)\n", + xhe_aac_get_error(err), err); + return AVERROR(EINVAL); + } + } else if (!s->loudness_exported) { + s->loudness_exported = 1; + return xhe_aac_write_loudness_file(avctx); + } + return 0; + } + if (frame) { int nb_new_samples = frame->nb_samples * avctx->ch_layout.nb_channels; diff --git a/libavcodec/version.h b/libavcodec/version.h index aa3a484c51e..26ee41eb1fd 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,8 +29,8 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 25 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 26 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ -- 2.54.0 From cc9152e06ed998df7f8bb70e33e924a0f169b94c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20Storsj=C3=B6?= <[email protected]> Date: Mon, 26 Oct 2026 15:08:02 +0200 Subject: [PATCH 2/2] avcodec/libfdk-xhe-aacenc: Force users to pick live loudness level If not specified, suggest that users should do two pass encoding, or they need to specify a live loudness level. The previous default value is kept in the error message, to guide users towards a potentially reasonable default value. --- doc/encoders.texi | 6 +++++- libavcodec/libfdk-xhe-aacenc.c | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 5fe35b78dd2..e2a4a22ed76 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -851,7 +851,8 @@ If not set, the default is CBR. @item live_loudness_level Set the expected loudness level of the stream, between -31 and -10 -dB LUFS. Defaults to -24 dB LUFS if not set. +dB LUFS. Must be set if not doing two pass encoding (-24 dB is +a reasonable default value in that case). @item rap_length The duration between encoded Random Access Points in the bitstream. @@ -878,6 +879,9 @@ ffmpeg -i input.wav -c:a libfdk_xhe_aac -flags:a +pass1 -loudness_file loudness. ffmpeg -i input.wav -c:a libfdk_xhe_aac -flags:a +pass2 -loudness_file loudness.dat out.m4a @end example +Doing two pass encoding is the recommended way to use the libfdk_xhe_aac +encoder. + @anchor{liblc3-enc} @section liblc3 diff --git a/libavcodec/libfdk-xhe-aacenc.c b/libavcodec/libfdk-xhe-aacenc.c index 25b89384ed2..576b687baab 100644 --- a/libavcodec/libfdk-xhe-aacenc.c +++ b/libavcodec/libfdk-xhe-aacenc.c @@ -54,7 +54,7 @@ typedef struct XHEAACContext { static const AVOption xhe_aac_enc_options[] = { { "vbr", "VBR mode (0-6)", offsetof(XHEAACContext, vbr), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 6, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, - { "live_loudness_level", "target loudness level in live encoding mode [LUFS]", offsetof(XHEAACContext, live_loudness_level), AV_OPT_TYPE_FLOAT, { .dbl = -24 }, -31.0, -10.0, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, + { "live_loudness_level", "target loudness level in live encoding mode [LUFS]", offsetof(XHEAACContext, live_loudness_level), AV_OPT_TYPE_FLOAT, { .dbl = 0 }, -31.0, -10.0, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, { "rap_interval", "Random Access Point interval", offsetof(XHEAACContext, rap_interval), AV_OPT_TYPE_DURATION, { .i64 = -1 }, -1, INT64_MAX, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, { "stream_id", "unique ID of this stream within one adaptation set", offsetof(XHEAACContext, stream_id), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 65535, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, { "loudness_file", "Filename for 2 pass loudness", offsetof(XHEAACContext, loudness_file), AV_OPT_TYPE_STRING, { 0 }, 0, 0, AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_ENCODING_PARAM }, @@ -383,6 +383,10 @@ static av_cold int xhe_aac_encode_init(AVCodecContext *avctx) goto error; } } else { + if (s->live_loudness_level > -10.0) { + av_log(avctx, AV_LOG_ERROR, "Need to either use two pass encoding, or set -live_loudness_level (-24 is a reasonable default)\n"); + return AVERROR(EINVAL); + } err = IIS_xHEAACEnc_Config_AddParamValueFloat(config_handle, IIS_XHEAACENC_PARAMETER_LIVE_LOUDNESS_LEVEL, s->live_loudness_level); if (err != IIS_XHEAACENC_NO_ERROR) { -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
