PR #23723 opened by omkhar URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23723 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23723.patch
tls_mbedtls checks the certificate verification result (mbedtls_ssl_get_verify_result) only in tls_open(). On the external-socket path (external_sock=1) tls_open() skips the handshake, which runs later via the url_handshake hook, tls_handshake(); that path never checks the result, so with verify=1 an untrusted peer certificate would be accepted. The only in-tree user is the WHIP muxer, which sets verify=0, so this is not reachable today -- the change is defence-in-depth, making tls_handshake() honour verify symmetrically with tls_open(). Verified against an OpenSSL DTLS s_server presenting an untrusted self-signed cert: baseline accepts the connection with verify=1; patched rejects it (AVERROR(EIO), verify flags 8 = BADCERT_NOT_TRUSTED); verify=0 and a trusted cert are unaffected. Applies cleanly at HEAD. >From 71a7c277a4fd12af0ce801e9bd6b92c785d07167 Mon Sep 17 00:00:00 2001 From: Omkhar Arasaratnam <[email protected]> Date: Mon, 6 Jul 2026 12:22:29 -0400 Subject: [PATCH] avformat/tls_mbedtls: check the certificate verification result in tls_handshake() tls_mbedtls verifies the peer certificate manually via mbedtls_ssl_get_verify_result() (it uses MBEDTLS_SSL_VERIFY_OPTIONAL), but only in tls_open(). On the external-socket path (external_sock=1) tls_open() skips the handshake; it then runs later through the url_handshake hook, tls_handshake(), which never checks the verification result, so with verify=1 an untrusted peer certificate would be accepted. The only in-tree user of this path is the WHIP muxer, which sets verify=0 (WebRTC binds the peer via the SDP a=fingerprint, not a CA), so this is not reachable today. This change is defence-in-depth: it makes tls_handshake() honor verify symmetrically with the existing tls_open() check. Signed-off-by: Omkhar Arasaratnam <[email protected]> --- libavformat/tls_mbedtls.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/libavformat/tls_mbedtls.c b/libavformat/tls_mbedtls.c index bfa5103596..8f7ace5e63 100644 --- a/libavformat/tls_mbedtls.c +++ b/libavformat/tls_mbedtls.c @@ -475,6 +475,7 @@ static int tls_handshake(URLContext *h) TLSContext *tls_ctx = h->priv_data; TLSShared *shr = &tls_ctx->tls_shared; URLContext *uc = shr->is_dtls ? shr->udp : shr->tcp; + uint32_t verify_res_flags; int ret; uc->flags &= ~AVIO_FLAG_NONBLOCK; @@ -490,6 +491,18 @@ static int tls_handshake(URLContext *h) } } + if (shr->verify) { + // check the result of the certificate verification + if ((verify_res_flags = mbedtls_ssl_get_verify_result(&tls_ctx->ssl_context)) != 0) { + av_log(h, AV_LOG_ERROR, "mbedtls_ssl_get_verify_result reported problems "\ + "with the certificate verification, returned flags: %"PRIu32"\n", + verify_res_flags); + if (verify_res_flags & MBEDTLS_X509_BADCERT_NOT_TRUSTED) + av_log(h, AV_LOG_ERROR, "The certificate is not correctly signed by the trusted CA.\n"); + return AVERROR(EIO); + } + } + return ret; } -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
