PR #24818 opened by kadirlua URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24818 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24818.patch
## Summary Propagate coded color information and static HDR metadata to the Android MediaCodec decoder input `MediaFormat`. ## Problem `libavcodec/mediacodecdec.c` currently sets the decoder MIME type and video dimensions, but does not pass `AVCodecContext` color range, colorspace, transfer characteristics or profile to `MediaFormat`. It also does not pass coded mastering-display/content-light side data as Android's `hdr-static-info` buffer. This matters for direct-to-Surface decoding. Android documents `KEY_HDR_STATIC_INFO` as decoder input for HDR content unless the metadata is already carried in the bitstream and the decoder supports an HDR-capable profile. A reproducible case is a VP9 Profile 2, 10-bit stream with: - limited range - BT.2020 primaries/matrix - SMPTE ST 2084 (PQ) - MaxCLL 1000 / MaxFALL 200 - mastering display metadata, max luminance 1000 nits On a Samsung SM-M526BR / Android 13 using `c2.qti.vp9.decoder`, the decoder falls back to BT.709 color aspects when these fields are absent from the input `MediaFormat`. ## Fix - Reuse the existing MediaCodec AVColor -> Android MediaFormat mapping helpers already used by `mediacodecenc.c` for: - `color-range` - `color-standard` - `color-transfer` - `profile` - Convert coded `AV_PKT_DATA_MASTERING_DISPLAY_METADATA` and `AV_PKT_DATA_CONTENT_LIGHT_LEVEL` side data to the 25-byte little-endian CTA-861.3 Static Metadata Descriptor expected by Android's `hdr-static-info` key. - Leave MediaCodec decoder selection unchanged. - Do not add HDR10+ dynamic metadata handling; this patch is limited to static HDR metadata. ## Validation The issue and the equivalent MediaFormat fix were reproduced on a Samsung SM-M526BR running Android 13 with the Qualcomm `c2.qti.vp9.decoder`. The test stream is VP9 Profile 2, 10-bit, limited range, BT.2020 with SMPTE ST 2084 (PQ), MaxCLL 1000 / MaxFALL 200, and mastering display metadata. Without propagating the color and HDR metadata before `MediaCodec.configure()`, the decoder falls back to BT.709 color aspects. With the equivalent MediaFormat fields supplied before configure, the decoder output reports: ```text android._dataspace = 298188800 color-standard = 6 color-range = 2 color-transfer = 6 >From d2065d9202d6be20fda4b6505900587464510d48 Mon Sep 17 00:00:00 2001 From: Kadir <[email protected]> Date: Wed, 30 Sep 2026 18:19:10 +0300 Subject: [PATCH] avcodec/mediacodecdec: propagate color and HDR metadata to MediaFormat MediaCodec decoders currently configure the input MediaFormat without coded color range, color standard, transfer characteristics, profile, or static HDR metadata available in AVCodecContext. This can cause Android decoders rendering directly to a Surface to fall back to SDR/BT.709 color aspects even when the input is HDR. In particular, a VP9 Profile 2 stream carrying BT.2020/PQ color information and mastering display metadata was decoded by a Qualcomm Codec2 decoder with default BT.709 color aspects. Mirror the existing MediaCodec encoder color/profile mapping on the decoder side and serialize AV_PKT_DATA_MASTERING_DISPLAY_METADATA and AV_PKT_DATA_CONTENT_LIGHT_LEVEL into Android's 25-byte CTA-861.3 "hdr-static-info" MediaFormat buffer. This leaves MediaCodec decoder selection unchanged; the profile used by ff_AMediaCodecList_getCodecNameByType() is not modified. On a Samsung SM-M526BR running Android 13, providing these fields before MediaCodec.configure() changes the VP9 decoder output from fallback BT.709 to BT.2020, limited range and ST2084, with the mastering display and content light metadata present in the output format. Signed-off-by: Kadir <[email protected]> --- libavcodec/mediacodecdec.c | 96 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 89f8ed54a8..54c9e4b032 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -30,6 +30,8 @@ #include "libavutil/mem.h" #include "libavutil/opt.h" #include "libavutil/intreadwrite.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/mathematics.h" #include "libavutil/pixfmt.h" #include "libavutil/internal.h" @@ -307,6 +309,82 @@ static int common_set_extradata(AVCodecContext *avctx, FFAMediaFormat *format) } #endif +static uint16_t mediacodec_hdr_static_value(AVRational value, AVRational unit) +{ + int64_t scaled; + + if (value.num <= 0 || value.den <= 0) + return 0; + + scaled = av_rescale_q(1, value, unit); + return (uint16_t)av_clip64(scaled, 0, UINT16_MAX); +} + +static void mediacodec_set_hdr_static_info(AVCodecContext *avctx, + FFAMediaFormat *format) +{ + const AVPacketSideData *mastering_sd = + ff_get_coded_side_data(avctx, AV_PKT_DATA_MASTERING_DISPLAY_METADATA); + const AVPacketSideData *light_sd = + ff_get_coded_side_data(avctx, AV_PKT_DATA_CONTENT_LIGHT_LEVEL); + const AVMasteringDisplayMetadata *mastering = NULL; + const AVContentLightMetadata *light = NULL; + uint8_t data[25] = { 0 }; + int has_metadata = 0; + + if (mastering_sd && mastering_sd->size >= sizeof(*mastering)) + mastering = (const AVMasteringDisplayMetadata *)mastering_sd->data; + if (light_sd && light_sd->size >= sizeof(*light)) + light = (const AVContentLightMetadata *)light_sd->data; + + if (mastering && mastering->has_primaries) { + AV_WL16(data + 1, + mediacodec_hdr_static_value(mastering->display_primaries[0][0], + (AVRational){ 1, 50000 })); + AV_WL16(data + 3, + mediacodec_hdr_static_value(mastering->display_primaries[0][1], + (AVRational){ 1, 50000 })); + AV_WL16(data + 5, + mediacodec_hdr_static_value(mastering->display_primaries[1][0], + (AVRational){ 1, 50000 })); + AV_WL16(data + 7, + mediacodec_hdr_static_value(mastering->display_primaries[1][1], + (AVRational){ 1, 50000 })); + AV_WL16(data + 9, + mediacodec_hdr_static_value(mastering->display_primaries[2][0], + (AVRational){ 1, 50000 })); + AV_WL16(data + 11, + mediacodec_hdr_static_value(mastering->display_primaries[2][1], + (AVRational){ 1, 50000 })); + AV_WL16(data + 13, + mediacodec_hdr_static_value(mastering->white_point[0], + (AVRational){ 1, 50000 })); + AV_WL16(data + 15, + mediacodec_hdr_static_value(mastering->white_point[1], + (AVRational){ 1, 50000 })); + has_metadata = 1; + } + + if (mastering && mastering->has_luminance) { + AV_WL16(data + 17, + mediacodec_hdr_static_value(mastering->max_luminance, + (AVRational){ 1, 1 })); + AV_WL16(data + 19, + mediacodec_hdr_static_value(mastering->min_luminance, + (AVRational){ 1, 10000 })); + has_metadata = 1; + } + + if (light) { + AV_WL16(data + 21, FFMIN(light->MaxCLL, UINT16_MAX)); + AV_WL16(data + 23, FFMIN(light->MaxFALL, UINT16_MAX)); + has_metadata = 1; + } + + if (has_metadata) + ff_AMediaFormat_setBuffer(format, "hdr-static-info", data, sizeof(data)); +} + static av_cold int mediacodec_decode_init(AVCodecContext *avctx) { int ret; @@ -434,8 +512,26 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) ff_AMediaFormat_setString(format, "mime", codec_mime); if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) { + int value; ff_AMediaFormat_setInt32(format, "width", avctx->width); ff_AMediaFormat_setInt32(format, "height", avctx->height); + value = ff_AMediaFormatColorRange_from_AVColorRange(avctx->color_range); + if (value != COLOR_RANGE_UNSPECIFIED) + ff_AMediaFormat_setInt32(format, "color-range", value); + + value = ff_AMediaFormatColorStandard_from_AVColorSpace(avctx->colorspace); + if (value != COLOR_STANDARD_UNSPECIFIED) + ff_AMediaFormat_setInt32(format, "color-standard", value); + + value = ff_AMediaFormatColorTransfer_from_AVColorTransfer(avctx->color_trc); + if (value != COLOR_TRANSFER_UNSPECIFIED) + ff_AMediaFormat_setInt32(format, "color-transfer", value); + + value = ff_AMediaCodecProfile_getProfileFromAVCodecContext(avctx); + if (value > 0) + ff_AMediaFormat_setInt32(format, "profile", value); + + mediacodec_set_hdr_static_info(avctx, format); } else { ff_AMediaFormat_setInt32(format, "channel-count", avctx->ch_layout.nb_channels); ff_AMediaFormat_setInt32(format, "sample-rate", avctx->sample_rate); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
