PR #23506 opened by Kacper Michajłow (kasper93)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23506
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/23506.patch
This pull request is an RFC for a libcurl integration into AVFormat as an HTTP
client.
The design is mostly a port from mpv, where switching to libcurl resolved many
limitations of a native HTTP client, improved stability, and brought a lot of
features for "free". libcurl is battle-tested in the wild by a great number of
projects, and in my opinion it is a better fit than a hand-spun client. A
simple HTTP client is fine, but http.c has grown: there are many improvements
glued onto it that make it no longer simple and, without any real tests, hard
to maintain.
This is focused only on the HTTP client part. It does not replace anything and
instead co-exists with http.c, which stays the default. Refactoring http.c and
thinking about migration, with all the backward-compatibility concerns, is for
after we land the first libcurl-based protocol.
#### Notable improvements over http.c:
* Seamless connection reuse, multiplexing, DNS/redirect/TLS caching, scoped to
the AVFormatContext.
* More robust HTTP, including HTTP/2 and HTTP/3.
* SOCKS5 proxy support, a commonly requested feature.
* Better compression support: more algorithms, and compression preferred, which
saves a lot on DASH playback.
* Lower maintenance cost, and future-proofing as the HTTP stack keeps getting
more complex aka. insane, the work to support newer version is basically out of
the scope of ffmpeg core interests imho.
#### Notable limitations over http.c:
* read/client only
* single `max_retries`: http.c has granural `reconnect`, `reconnect_at_eof`,
`reconnect_on_network_error`, `reconnect_on_http_error`, `reconnect_streamed`,
`reconnect_delay_max`, `reconnect_delay_total_max`, `respect_retry_after`. I'm
not sure we need all of this, can be added later.
* `short_seek_size` not supported, up to discussion, if this is still needed.
(btw. it is also not working on http.c currently)
* `url_get_file_handle` not supported, we don't want to mess with sockets under
libcurl
* ICY/Shoutcast metadata not supported. I have implementation working for mpv,
but frankly I not a fun of doing this at transport level and storing result in
AVOption, which will desync by any playback downstream.
#### Points of dicussion
* The scope of curl event loop, currently it's scoped to each AVFC.
* Do we even want libcurl?
* What to do with http.c?
- refactor it into client / server parts
- add tests
- replace client part with libcurl based impl?
- plumbing options compatibility
* Next steps?
#### Design
* HTTP(S) is handled by a new `libcurl` URLProtocol. It coexists with http.c
and is selected explicitly via a `libcurl:` URL prefix or the `prefer_libcurl`
option, so protocol probing and enumeration are unaffected.
* libcurl runs on a dedicated event-loop thread driving curl_multi, calling
threads talk to it through a small command queue and never touch curl handles
directly. This avoids driving curl from url_read(), which is fragile and
notably stalls HTTP/3 when the handle is not pumped while idle.
* The loop is scoped per AVFormatContext (created on demand, shared by all of
the demuxer's transfers), so curl reuses connections and caches across e.g.
HLS/DASH segments (without glue hacks in hls.c). Opens with no owning context
get a private loop. This needs a small back-reference from URLContext to its
AVFormatContext.
* Data flows through a per-transfer FIFO with pause/unpause back pressure,
seeking reconnects (or reuse) with a Range request, and recoverable errors are
retried.
As this is first draft, I have not tested this extensively yet. Done mostly
smoke test, but also the similar implementation is used in mpv, so it's is
proven to work, and I will keep maintaining it.
I refrained from listing the limitations and issues of http.c (and
hls.c/dashdec.c) here, but by switching to libcurl in mpv I closed over 20
issues reported by our users.
**Disclaimer**: LLM was lightly used, to generate AVOptions table, some
mechanic conversions, and did overall "cleanup" pass over the code. The above
description is not LLM, even though it's lots of text dump.
From 2ffc50f3ab11ce0d5e7931116313ae9efe2f61de Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 02:13:16 +0200
Subject: [PATCH 1/6] avformat: thread owning AVFormatContext down to
URLContext
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add a URLContext.fmt_ctx back-reference, set by io_open_default before
url_open() runs. Will be used to access shared resources per avfc.
Signed-off-by: Kacper Michajłow <[email protected]>
---
libavformat/avio.c | 36 ++++++++++++++++++++++++++++--------
libavformat/avio_internal.h | 10 ++++++++++
libavformat/options.c | 3 ++-
libavformat/url.h | 1 +
4 files changed, 41 insertions(+), 9 deletions(-)
diff --git a/libavformat/avio.c b/libavformat/avio.c
index 1e26092f79..4e54cb6e8a 100644
--- a/libavformat/avio.c
+++ b/libavformat/avio.c
@@ -365,16 +365,17 @@ int ffurl_alloc(URLContext **puc, const char *filename,
int flags,
return AVERROR_PROTOCOL_NOT_FOUND;
}
-int ffurl_open_whitelist(URLContext **puc, const char *filename, int flags,
- const AVIOInterruptCB *int_cb, AVDictionary **options,
- const char *whitelist, const char* blacklist,
- URLContext *parent)
+static int url_open_whitelist(URLContext **puc, const char *filename, int
flags,
+ const AVIOInterruptCB *int_cb, AVDictionary
**options,
+ const char *whitelist, const char* blacklist,
+ URLContext *parent, void *fmt_ctx)
{
AVDictionary *tmp_opts = NULL;
AVDictionaryEntry *e;
int ret = ffurl_alloc(puc, filename, flags, int_cb);
if (ret < 0)
return ret;
+ (*puc)->fmt_ctx = fmt_ctx;
if (parent) {
ret = av_opt_copy(*puc, parent);
if (ret < 0)
@@ -415,6 +416,15 @@ fail:
return ret;
}
+int ffurl_open_whitelist(URLContext **puc, const char *filename, int flags,
+ const AVIOInterruptCB *int_cb, AVDictionary **options,
+ const char *whitelist, const char* blacklist,
+ URLContext *parent)
+{
+ return url_open_whitelist(puc, filename, flags, int_cb, options,
+ whitelist, blacklist, parent, NULL);
+}
+
int ffio_fdopen(AVIOContext **sp, URLContext *h)
{
AVIOContext *s;
@@ -474,16 +484,18 @@ int ffio_fdopen(AVIOContext **sp, URLContext *h)
return 0;
}
-int ffio_open_whitelist(AVIOContext **s, const char *filename, int flags,
- const AVIOInterruptCB *int_cb, AVDictionary **options,
- const char *whitelist, const char *blacklist)
+int ffio_open_whitelist2(AVIOContext **s, const char *filename, int flags,
+ const AVIOInterruptCB *int_cb, AVDictionary **options,
+ const char *whitelist, const char *blacklist,
+ void *fmt_ctx)
{
URLContext *h;
int err;
*s = NULL;
- err = ffurl_open_whitelist(&h, filename, flags, int_cb, options,
whitelist, blacklist, NULL);
+ err = url_open_whitelist(&h, filename, flags, int_cb, options, whitelist,
+ blacklist, NULL, fmt_ctx);
if (err < 0)
return err;
err = ffio_fdopen(s, h);
@@ -494,6 +506,14 @@ int ffio_open_whitelist(AVIOContext **s, const char
*filename, int flags,
return 0;
}
+int ffio_open_whitelist(AVIOContext **s, const char *filename, int flags,
+ const AVIOInterruptCB *int_cb, AVDictionary **options,
+ const char *whitelist, const char *blacklist)
+{
+ return ffio_open_whitelist2(s, filename, flags, int_cb, options,
+ whitelist, blacklist, NULL);
+}
+
int avio_open2(AVIOContext **s, const char *filename, int flags,
const AVIOInterruptCB *int_cb, AVDictionary **options)
{
diff --git a/libavformat/avio_internal.h b/libavformat/avio_internal.h
index fadf19dae5..1ae9592f95 100644
--- a/libavformat/avio_internal.h
+++ b/libavformat/avio_internal.h
@@ -255,6 +255,16 @@ int ffio_open_whitelist(AVIOContext **s, const char *url,
int flags,
const AVIOInterruptCB *int_cb, AVDictionary **options,
const char *whitelist, const char *blacklist);
+/**
+ * Like ffio_open_whitelist(), but additionally records @p fmt_ctx on the
+ * underlying URLContext before it is connected, so protocols can use shared
+ * per-format resource. Pass NULL for standalone use.
+ */
+int ffio_open_whitelist2(AVIOContext **s, const char *url, int flags,
+ const AVIOInterruptCB *int_cb, AVDictionary **options,
+ const char *whitelist, const char *blacklist,
+ void *fmt_ctx);
+
/**
* Close a null buffer.
*
diff --git a/libavformat/options.c b/libavformat/options.c
index 6da1b690c6..82b7b94ab3 100644
--- a/libavformat/options.c
+++ b/libavformat/options.c
@@ -153,7 +153,8 @@ static int io_open_default(AVFormatContext *s, AVIOContext
**pb,
av_log(s, loglevel, "Opening \'%s\' for %s\n", url, flags &
AVIO_FLAG_WRITE ? "writing" : "reading");
- return ffio_open_whitelist(pb, url, flags, &s->interrupt_callback,
options, s->protocol_whitelist, s->protocol_blacklist);
+ return ffio_open_whitelist2(pb, url, flags, &s->interrupt_callback,
options,
+ s->protocol_whitelist, s->protocol_blacklist,
s);
}
static int io_close2_default(AVFormatContext *s, AVIOContext *pb)
diff --git a/libavformat/url.h b/libavformat/url.h
index 0784d77b64..f08d18bcb4 100644
--- a/libavformat/url.h
+++ b/libavformat/url.h
@@ -46,6 +46,7 @@ typedef struct URLContext {
const char *protocol_whitelist;
const char *protocol_blacklist;
int min_packet_size; /**< if non zero, the stream is packetized
with this min packet size */
+ void *fmt_ctx; /**< The AVFormatContext that opened this
URLContext, or NULL for standalone use */
} URLContext;
typedef struct URLProtocol {
--
2.52.0
From 1ed030b7d72d7ab16109f6e352529c4d33f997f0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 02:30:08 +0200
Subject: [PATCH 2/6] avformat/libcurl: add libcurl protocol skeleton
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
configure changes, and protocol with ENOSYS impl.
Signed-off-by: Kacper Michajłow <[email protected]>
---
configure | 4 +++
libavformat/Makefile | 1 +
libavformat/libcurl.c | 66 +++++++++++++++++++++++++++++++++++++++++
libavformat/protocols.c | 1 +
4 files changed, 72 insertions(+)
create mode 100644 libavformat/libcurl.c
diff --git a/configure b/configure
index e67aa362ad..9f9a67c8a6 100755
--- a/configure
+++ b/configure
@@ -2086,6 +2086,7 @@ EXTERNAL_LIBRARY_LIST="
libbs2b
libcaca
libcodec2
+ libcurl
libdav1d
libdc1394
libflite
@@ -4140,6 +4141,8 @@ ipns_gateway_protocol_select="https_protocol"
libamqp_protocol_deps="librabbitmq"
libamqp_protocol_select="network"
librist_protocol_deps="librist"
+libcurl_protocol_deps="libcurl"
+libcurl_protocol_select="network"
librist_protocol_select="network"
librtmp_protocol_deps="librtmp"
librtmpe_protocol_deps="librtmp"
@@ -7342,6 +7345,7 @@ enabled libbluray && require_pkg_config libbluray
libbluray libbluray/bl
enabled libbs2b && require_pkg_config libbs2b libbs2b bs2b.h
bs2b_open
enabled libcaca && require_pkg_config libcaca caca caca.h
caca_create_canvas
enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create
-lcodec2
+enabled libcurl && require_pkg_config libcurl "libcurl >= 7.87.0"
curl/curl.h curl_easy_init
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 1.0.0"
"dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0"
davs2.h davs2_decoder_open
enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2
dc1394/dc1394.h dc1394_new
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 0db0c7c2a9..d625244cca 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -733,6 +733,7 @@ OBJS-$(CONFIG_UNIX_PROTOCOL) += unix.o
# external library protocols
OBJS-$(CONFIG_LIBAMQP_PROTOCOL) += libamqp.o
+OBJS-$(CONFIG_LIBCURL_PROTOCOL) += libcurl.o
OBJS-$(CONFIG_LIBRIST_PROTOCOL) += librist.o
OBJS-$(CONFIG_LIBRTMP_PROTOCOL) += librtmp.o
OBJS-$(CONFIG_LIBRTMPE_PROTOCOL) += librtmp.o
diff --git a/libavformat/libcurl.c b/libavformat/libcurl.c
new file mode 100644
index 0000000000..b4a4995873
--- /dev/null
+++ b/libavformat/libcurl.c
@@ -0,0 +1,66 @@
+/*
+ * libcurl based HTTP(S) protocol
+ * Copyright (C) 2026 Kacper Michajłow
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "config_components.h"
+
+#include <curl/curl.h>
+
+#include "libavutil/opt.h"
+
+#include "avformat.h"
+#include "internal.h"
+#include "url.h"
+
+typedef struct CurlContext {
+ const AVClass *class;
+} CurlContext;
+
+static int libcurl_open(URLContext *h, const char *url, int flags,
+ AVDictionary **options)
+{
+ return AVERROR(ENOSYS);
+}
+
+static int libcurl_read(URLContext *h, unsigned char *buf, int size)
+{
+ return AVERROR(ENOSYS);
+}
+
+static int libcurl_close(URLContext *h)
+{
+ return 0;
+}
+
+static const AVClass libcurl_context_class = {
+ .class_name = "libcurl",
+ .item_name = av_default_item_name,
+ .version = LIBAVUTIL_VERSION_INT,
+};
+
+const URLProtocol ff_libcurl_protocol = {
+ .name = "libcurl",
+ .url_open2 = libcurl_open,
+ .url_read = libcurl_read,
+ .url_close = libcurl_close,
+ .priv_data_size = sizeof(CurlContext),
+ .priv_data_class = &libcurl_context_class,
+ .flags = URL_PROTOCOL_FLAG_NETWORK,
+};
diff --git a/libavformat/protocols.c b/libavformat/protocols.c
index 257b41970a..367b40ee2c 100644
--- a/libavformat/protocols.c
+++ b/libavformat/protocols.c
@@ -67,6 +67,7 @@ extern const URLProtocol ff_udp_protocol;
extern const URLProtocol ff_udplite_protocol;
extern const URLProtocol ff_unix_protocol;
extern const URLProtocol ff_libamqp_protocol;
+extern const URLProtocol ff_libcurl_protocol;
extern const URLProtocol ff_librist_protocol;
extern const URLProtocol ff_librtmp_protocol;
extern const URLProtocol ff_librtmpe_protocol;
--
2.52.0
From c7d64874ce7146fec0a56823c7e8b038a8c276fe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 02:52:11 +0200
Subject: [PATCH 3/6] avformat/libcurl: implement event loop and read path
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add the per-AVFormatContext curl_multi worker thread, the command queue,
write/header/xferinfo callbacks and a FIFO with pause/unpause
backpressure. url_open2 probes the response on a dedicated thread.
url_read drains the FIFO and yields to the avio layer so the interrupt
callback is honoured. An explicit "libcurl:" URL prefix forces the
protocol.
Signed-off-by: Kacper Michajłow <[email protected]>
---
configure | 2 +-
libavformat/avformat.c | 5 +
libavformat/internal.h | 13 +
libavformat/libcurl.c | 601 ++++++++++++++++++++++++++++++++++++++++-
4 files changed, 615 insertions(+), 6 deletions(-)
diff --git a/configure b/configure
index 9f9a67c8a6..7dc0670882 100755
--- a/configure
+++ b/configure
@@ -4141,7 +4141,7 @@ ipns_gateway_protocol_select="https_protocol"
libamqp_protocol_deps="librabbitmq"
libamqp_protocol_select="network"
librist_protocol_deps="librist"
-libcurl_protocol_deps="libcurl"
+libcurl_protocol_deps="libcurl threads"
libcurl_protocol_select="network"
librist_protocol_select="network"
librtmp_protocol_deps="librtmp"
diff --git a/libavformat/avformat.c b/libavformat/avformat.c
index dbd4792cd2..19c80dbd56 100644
--- a/libavformat/avformat.c
+++ b/libavformat/avformat.c
@@ -19,6 +19,8 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "config_components.h"
+
#include <math.h>
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
@@ -188,6 +190,9 @@ void avformat_free_context(AVFormatContext *s)
av_freep(&s->chapters);
av_dict_free(&s->metadata);
av_dict_free(&si->id3v2_meta);
+#if CONFIG_LIBCURL_PROTOCOL
+ ff_curl_loop_free(&si->curl_loop);
+#endif
av_packet_free(&si->pkt);
av_packet_free(&si->parse_pkt);
avpriv_packet_list_free(&si->packet_buffer);
diff --git a/libavformat/internal.h b/libavformat/internal.h
index 8bf9444edb..61a14ec4c4 100644
--- a/libavformat/internal.h
+++ b/libavformat/internal.h
@@ -118,6 +118,13 @@ typedef struct FFFormatContext {
AVDictionary *id3v2_meta;
int missing_streams;
+
+ /**
+ * Shared libcurl event loop, created on demand on the first use. Freed on
+ * context free. This allows to share libcurl state across URLContexts,
+ * scoped to this context.
+ */
+ void *curl_loop;
} FFFormatContext;
static av_always_inline FFFormatContext *ffformatcontext(AVFormatContext *s)
@@ -609,6 +616,12 @@ int ff_copy_whiteblacklists(AVFormatContext *dst, const
AVFormatContext *src);
*/
int ff_format_io_close(AVFormatContext *s, AVIOContext **pb);
+/**
+ * Release a libcurl event loop and set *loop to NULL.
+ * No-op when @p loop or *loop is NULL.
+ */
+void ff_curl_loop_free(void **loop);
+
/**
* Utility function to check if the file uses http or https protocol
*
diff --git a/libavformat/libcurl.c b/libavformat/libcurl.c
index b4a4995873..4c16c94f6e 100644
--- a/libavformat/libcurl.c
+++ b/libavformat/libcurl.c
@@ -23,29 +23,620 @@
#include <curl/curl.h>
+#include "libavutil/avstring.h"
+#include "libavutil/error.h"
+#include "libavutil/fifo.h"
+#include "libavutil/macros.h"
+#include "libavutil/mem.h"
#include "libavutil/opt.h"
+#include "libavutil/thread.h"
+#include "libavutil/time.h"
#include "avformat.h"
#include "internal.h"
#include "url.h"
-typedef struct CurlContext {
- const AVClass *class;
-} CurlContext;
+#define CURL_DEFAULT_BUFFER_SIZE (4 << 20)
+/* Blocking waits wake up this often so url_read()/open can poll the interrupt
+ * callback. */
+#define CURL_WAIT_US 100000
+
+typedef struct CurlContext CurlContext;
+
+enum cmd_kind {
+ CMD_ADD, /* add the easy handle to the multi and start the transfer */
+ CMD_REMOVE, /* remove the easy handle from the multi */
+ CMD_UNPAUSE, /* resume a transfer paused because the FIFO was full */
+};
+
+typedef struct CurlCmd {
+ enum cmd_kind kind;
+ CurlContext *ctx;
+ int sync; /* caller waits for completion, flips by done */
+ int done;
+ struct CurlCmd *next;
+} CurlCmd;
+
+typedef struct CurlLoop {
+ pthread_t thread;
+ CURLM *multi;
+
+ pthread_mutex_t mutex; /* guards the command queue, exit and cmd->done */
+ pthread_cond_t cond; /* signaled when a sync command completes */
+ CurlCmd *cmd_head, *cmd_tail;
+ int exit;
+} CurlLoop;
+
+struct CurlContext {
+ const AVClass *class;
+ URLContext *h;
+
+ CurlLoop *loop;
+ int private_loop; /* loop is owned by this context (not
shared) */
+ CURL *easy;
+
+ int64_t buffer_size;
+
+ /* Producer bookkeeping, touched only by the loop thread. */
+ int active; /* currently added to the multi */
+
+ /* Probe result. Set by the loop thread, read by url_open() once probed. */
+ int probed;
+ int stream_ok;
+ int seekable;
+ int64_t content_size;
+
+ /* Shared transfer state, guarded by mutex. */
+ pthread_mutex_t mutex;
+ pthread_cond_t cond;
+ AVFifo *fifo;
+ int paused; /* write callback paused, FIFO was full */
+ int eof; /* producer delivered all data */
+ int error; /* AVERROR for an unrecoverable failure, or 0
*/
+ int aborted; /* transfer should stop (open was
interrupted) */
+};
+
+/* curl_global_init()/cleanup() are refcounted, so technically could be used
+ * directly per each context. However, the thread safety is provided only when
+ * curl_version_info() says so. Do our own refcounting, in global scope.
+ */
+static AVMutex curl_global_lock = AV_MUTEX_INITIALIZER;
+static int curl_global_refs;
+
+static int curl_global_ref(void)
+{
+ int ret = 0;
+ ff_mutex_lock(&curl_global_lock);
+ if (curl_global_refs == 0 && curl_global_init(CURL_GLOBAL_ALL))
+ ret = AVERROR_EXTERNAL;
+ else
+ curl_global_refs++;
+ ff_mutex_unlock(&curl_global_lock);
+ return ret;
+}
+
+static void curl_global_unref(void)
+{
+ ff_mutex_lock(&curl_global_lock);
+ if (--curl_global_refs == 0)
+ curl_global_cleanup();
+ ff_mutex_unlock(&curl_global_lock);
+}
+
+/* Guards lazy creation of a format context's shared loop. */
+static AVMutex curl_loop_lock = AV_MUTEX_INITIALIZER;
+
+static int curlcode_to_averror(CURLcode code)
+{
+ switch (code) {
+ case CURLE_OK: return 0;
+ case CURLE_URL_MALFORMAT:
+ case CURLE_UNSUPPORTED_PROTOCOL: return AVERROR(EINVAL);
+ case CURLE_COULDNT_RESOLVE_PROXY:
+ case CURLE_COULDNT_RESOLVE_HOST: return AVERROR(EHOSTUNREACH);
+ case CURLE_COULDNT_CONNECT: return AVERROR(ECONNREFUSED);
+ case CURLE_OPERATION_TIMEDOUT: return AVERROR(ETIMEDOUT);
+ case CURLE_LOGIN_DENIED:
+ case CURLE_REMOTE_ACCESS_DENIED: return AVERROR(EACCES);
+ case CURLE_OUT_OF_MEMORY: return AVERROR(ENOMEM);
+ case CURLE_PEER_FAILED_VERIFICATION:
+ case CURLE_SSL_CACERT_BADFILE: return AVERROR_INVALIDDATA;
+ default: return AVERROR(EIO);
+ }
+}
+
+static int http_status_to_averror(long status)
+{
+ switch (status) {
+ case 400: return AVERROR_HTTP_BAD_REQUEST;
+ case 401: return AVERROR_HTTP_UNAUTHORIZED;
+ case 403: return AVERROR_HTTP_FORBIDDEN;
+ case 404: return AVERROR_HTTP_NOT_FOUND;
+ }
+ if (status >= 400 && status < 500)
+ return AVERROR_HTTP_OTHER_4XX;
+ if (status >= 500 && status < 600)
+ return AVERROR_HTTP_SERVER_ERROR;
+ return AVERROR(EIO);
+}
+
+/* ------------------------------------------------------------------------- */
+/* curl callbacks (run on the loop thread) */
+/* ------------------------------------------------------------------------- */
+
+static size_t write_callback(char *ptr, size_t size, size_t nmemb, void
*userdata)
+{
+ CurlContext *c = userdata;
+ size_t bytes = size * nmemb;
+ size_t space;
+
+ pthread_mutex_lock(&c->mutex);
+
+ if (c->aborted || !c->stream_ok) {
+ pthread_mutex_unlock(&c->mutex);
+ return CURL_WRITEFUNC_ERROR;
+ }
+
+ space = av_fifo_can_write(c->fifo);
+ if (space < bytes) {
+ /* pause the transfer and wait for the consumer to drain. */
+ c->paused = 1;
+ pthread_mutex_unlock(&c->mutex);
+ return CURL_WRITEFUNC_PAUSE;
+ }
+
+ av_fifo_write(c->fifo, ptr, bytes);
+ c->paused = 0;
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
+
+ return bytes;
+}
+
+static size_t header_callback(char *ptr, size_t size, size_t nitems, void
*userdata)
+{
+ CurlContext *c = userdata;
+ size_t len = size * nitems;
+ size_t n = len;
+ long status = 0;
+
+ /* Act only on the blank line that terminates a header block. */
+ while (n && (ptr[n - 1] == '\r' || ptr[n - 1] == '\n'))
+ n--;
+ if (n)
+ return len;
+
+ curl_easy_getinfo(c->easy, CURLINFO_RESPONSE_CODE, &status);
+
+ /* Redirects produce an intermediate header block, wait for the final one.
*/
+ if (status >= 300 && status < 400)
+ return len;
+
+ pthread_mutex_lock(&c->mutex);
+ if (status >= 200 && status < 300) {
+ c->stream_ok = 1;
+ } else {
+ c->stream_ok = 0;
+ if (!c->error)
+ c->error = http_status_to_averror(status);
+ }
+ c->probed = 1;
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
+
+ return len;
+}
+
+static int xferinfo_callback(void *userdata, curl_off_t dltotal, curl_off_t
dlnow,
+ curl_off_t ultotal, curl_off_t ulnow)
+{
+ CurlContext *c = userdata;
+ int aborted;
+ pthread_mutex_lock(&c->mutex);
+ aborted = c->aborted;
+ pthread_mutex_unlock(&c->mutex);
+ return aborted; /* non-zero aborts the transfer */
+}
+
+/* Transfer finished (or failed) */
+static void on_done(CurlContext *c, CURLcode code)
+{
+ pthread_mutex_lock(&c->mutex);
+ if (!c->probed) {
+ /* Connection died before any usable header arrived. */
+ c->probed = 1;
+ c->stream_ok = 0;
+ if (!c->error)
+ c->error = curlcode_to_averror(code);
+ } else if (code == CURLE_OK && !c->aborted) {
+ c->eof = 1;
+ } else if (!c->aborted && !c->error) {
+ c->error = curlcode_to_averror(code);
+ }
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
+}
+
+/* ------------------------------------------------------------------------- */
+/* event loop thread + command queue */
+/* ------------------------------------------------------------------------- */
+
+static void execute_command(CurlLoop *loop, CurlCmd *cmd)
+{
+ CurlContext *c = cmd->ctx;
+
+ switch (cmd->kind) {
+ case CMD_ADD:
+ c->active = 1;
+ curl_multi_add_handle(loop->multi, c->easy);
+ break;
+ case CMD_REMOVE:
+ if (c->active) {
+ curl_multi_remove_handle(loop->multi, c->easy);
+ c->active = 0;
+ }
+ break;
+ case CMD_UNPAUSE:
+ curl_easy_pause(c->easy, CURLPAUSE_CONT);
+ break;
+ }
+}
+
+static void *curl_worker(void *arg)
+{
+ CurlLoop *loop = arg;
+
+ ff_thread_setname("curl");
+
+ while (1) {
+ CurlCmd *cmd;
+ CURLMsg *msg;
+ int running = 0, left = 0, do_exit;
+
+ pthread_mutex_lock(&loop->mutex);
+ cmd = loop->cmd_head;
+ if (cmd) {
+ loop->cmd_head = cmd->next;
+ if (!loop->cmd_head)
+ loop->cmd_tail = NULL;
+ }
+ do_exit = loop->exit;
+ pthread_mutex_unlock(&loop->mutex);
+
+ if (cmd) {
+ execute_command(loop, cmd);
+ if (cmd->sync) {
+ pthread_mutex_lock(&loop->mutex);
+ cmd->done = 1;
+ pthread_cond_broadcast(&loop->cond);
+ pthread_mutex_unlock(&loop->mutex);
+ } else {
+ av_free(cmd);
+ }
+ continue; /* drain the whole queue before pumping curl */
+ }
+
+ if (do_exit)
+ break;
+
+ curl_multi_perform(loop->multi, &running);
+
+ while ((msg = curl_multi_info_read(loop->multi, &left))) {
+ CurlContext *c = NULL;
+ if (msg->msg != CURLMSG_DONE)
+ continue;
+ curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, &c);
+ curl_multi_remove_handle(loop->multi, msg->easy_handle);
+ if (c) {
+ c->active = 0;
+ on_done(c, msg->data.result);
+ }
+ }
+
+ curl_multi_poll(loop->multi, NULL, 0, 1000, NULL);
+ }
+
+ return NULL;
+}
+
+/* Dispatch a command to the loop. For sync commands the caller blocks until
the
+ * loop thread has executed it. Returns 0 or a negative AVERROR. */
+static int curl_dispatch(CurlLoop *loop, enum cmd_kind kind, CurlContext *c,
int sync)
+{
+ CurlCmd stackcmd = {0};
+ CurlCmd *cmd = sync ? &stackcmd : av_mallocz(sizeof(*cmd));
+
+ if (!cmd)
+ return AVERROR(ENOMEM);
+
+ cmd->kind = kind;
+ cmd->ctx = c;
+ cmd->sync = sync;
+
+ pthread_mutex_lock(&loop->mutex);
+ if (loop->cmd_tail)
+ loop->cmd_tail->next = cmd;
+ else
+ loop->cmd_head = cmd;
+ loop->cmd_tail = cmd;
+ curl_multi_wakeup(loop->multi);
+ if (sync) {
+ while (!cmd->done)
+ pthread_cond_wait(&loop->cond, &loop->mutex);
+ }
+ pthread_mutex_unlock(&loop->mutex);
+
+ return 0;
+}
+
+static CurlLoop *curl_loop_create(void)
+{
+ CurlLoop *loop = av_mallocz(sizeof(*loop));
+ if (!loop)
+ return NULL;
+
+ if (pthread_mutex_init(&loop->mutex, NULL))
+ goto fail;
+ if (pthread_cond_init(&loop->cond, NULL)) {
+ pthread_mutex_destroy(&loop->mutex);
+ goto fail;
+ }
+
+ if (curl_global_ref() < 0)
+ goto fail2;
+
+ loop->multi = curl_multi_init();
+ if (!loop->multi)
+ goto fail3;
+ curl_multi_setopt(loop->multi, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);
+
+ if (pthread_create(&loop->thread, NULL, curl_worker, loop)) {
+ curl_multi_cleanup(loop->multi);
+ goto fail3;
+ }
+
+ return loop;
+
+fail3:
+ curl_global_unref();
+fail2:
+ pthread_cond_destroy(&loop->cond);
+ pthread_mutex_destroy(&loop->mutex);
+fail:
+ av_free(loop);
+ return NULL;
+}
+
+static void curl_loop_destroy(CurlLoop *loop)
+{
+ pthread_mutex_lock(&loop->mutex);
+ loop->exit = 1;
+ curl_multi_wakeup(loop->multi);
+ pthread_mutex_unlock(&loop->mutex);
+
+ pthread_join(loop->thread, NULL);
+
+ curl_multi_cleanup(loop->multi);
+ pthread_cond_destroy(&loop->cond);
+ pthread_mutex_destroy(&loop->mutex);
+ av_free(loop);
+
+ /* Released after the thread is joined and the multi handle is gone. */
+ curl_global_unref();
+}
+
+/* Attach a context to its event loop. With an owning AVFormatContext the loop
is
+ * created lazily, cached on it, and shared across the demuxer's transfers so
curl
+ * reuses connections; it is freed at format teardown. Without one the context
+ * gets a private loop freed on close. */
+static int curl_loop_attach(CurlContext *c, void *fmt_ctx)
+{
+ if (!fmt_ctx) {
+ c->loop = curl_loop_create();
+ c->private_loop = 1;
+ return c->loop ? 0 : AVERROR(ENOMEM);
+ }
+
+ ff_mutex_lock(&curl_loop_lock);
+ c->loop = ffformatcontext(fmt_ctx)->curl_loop;
+ if (!c->loop)
+ c->loop = curl_loop_create();
+ ff_mutex_unlock(&curl_loop_lock);
+
+ return c->loop ? 0 : AVERROR(ENOMEM);
+}
+
+void ff_curl_loop_free(void **loop)
+{
+ if (loop && *loop) {
+ curl_loop_destroy(*loop);
+ *loop = NULL;
+ }
+}
+
+/* ------------------------------------------------------------------------- */
+/* URLProtocol callbacks */
+/* ------------------------------------------------------------------------- */
+
+static int libcurl_close(URLContext *h);
+
+static void setup_curl(CurlContext *c)
+{
+ CURL *e = c->easy;
+ const char *url = c->h->filename;
+
+ /* Drop an optional "libcurl:" prefix that forces this protocol. */
+ av_strstart(url, "libcurl:", &url);
+
+ curl_easy_setopt(e, CURLOPT_URL, url);
+ curl_easy_setopt(e, CURLOPT_PRIVATE, c);
+ curl_easy_setopt(e, CURLOPT_NOSIGNAL, 1L);
+
+ curl_easy_setopt(e, CURLOPT_WRITEFUNCTION, write_callback);
+ curl_easy_setopt(e, CURLOPT_WRITEDATA, c);
+ curl_easy_setopt(e, CURLOPT_HEADERFUNCTION, header_callback);
+ curl_easy_setopt(e, CURLOPT_HEADERDATA, c);
+
+ curl_easy_setopt(e, CURLOPT_NOPROGRESS, 0L);
+ curl_easy_setopt(e, CURLOPT_XFERINFOFUNCTION, xferinfo_callback);
+ curl_easy_setopt(e, CURLOPT_XFERINFODATA, c);
+
+ curl_easy_setopt(e, CURLOPT_FOLLOWLOCATION, 1L);
+ curl_easy_setopt(e, CURLOPT_TCP_KEEPALIVE, 1L);
+ curl_easy_setopt(e, CURLOPT_ACCEPT_ENCODING, "");
+}
+
+/* Block until the transfer has been probed, the stream errored, or the open
was
+ * interrupted. Returns 0, or a negative AVERROR. */
+static int wait_for_probe(CurlContext *c)
+{
+ URLContext *h = c->h;
+ int ret = 0;
+
+ pthread_mutex_lock(&c->mutex);
+ while (!c->probed && !c->error) {
+ int64_t t;
+ struct timespec ts;
+
+ if (ff_check_interrupt(&h->interrupt_callback)) {
+ c->aborted = 1;
+ ret = AVERROR_EXIT;
+ break;
+ }
+ t = av_gettime() + CURL_WAIT_US;
+ ts.tv_sec = t / 1000000;
+ ts.tv_nsec = (t % 1000000) * 1000;
+ pthread_cond_timedwait(&c->cond, &c->mutex, &ts);
+ }
+ if (!ret) {
+ if (!c->stream_ok)
+ ret = c->error ? c->error : AVERROR(EIO);
+ }
+ pthread_mutex_unlock(&c->mutex);
+
+ return ret;
+}
static int libcurl_open(URLContext *h, const char *url, int flags,
AVDictionary **options)
{
- return AVERROR(ENOSYS);
+ CurlContext *c = h->priv_data;
+ int ret;
+
+ c->h = h;
+ c->content_size = -1;
+ if (c->buffer_size <= 0)
+ c->buffer_size = CURL_DEFAULT_BUFFER_SIZE;
+
+ if (pthread_mutex_init(&c->mutex, NULL))
+ return AVERROR(ENOMEM);
+ if (pthread_cond_init(&c->cond, NULL)) {
+ pthread_mutex_destroy(&c->mutex);
+ return AVERROR(ENOMEM);
+ }
+
+ c->fifo = av_fifo_alloc2(c->buffer_size, 1, 0);
+ if (!c->fifo) {
+ ret = AVERROR(ENOMEM);
+ goto fail;
+ }
+
+ ret = curl_loop_attach(c, h->fmt_ctx);
+ if (ret < 0)
+ goto fail;
+
+ c->easy = curl_easy_init();
+ if (!c->easy) {
+ ret = AVERROR(ENOMEM);
+ goto fail;
+ }
+ setup_curl(c);
+
+ ret = curl_dispatch(c->loop, CMD_ADD, c, 0);
+ if (ret < 0)
+ goto fail;
+
+ ret = wait_for_probe(c);
+ if (ret < 0)
+ goto fail;
+
+ h->is_streamed = !c->seekable;
+
+ return 0;
+
+fail:
+ libcurl_close(h);
+ return ret;
}
static int libcurl_read(URLContext *h, unsigned char *buf, int size)
{
- return AVERROR(ENOSYS);
+ CurlContext *c = h->priv_data;
+ int nonblock = h->flags & AVIO_FLAG_NONBLOCK;
+ int ret;
+
+ pthread_mutex_lock(&c->mutex);
+ while (1) {
+ size_t avail = av_fifo_can_read(c->fifo);
+ int64_t t;
+ struct timespec ts;
+
+ if (avail) {
+ int n = FFMIN(avail, (size_t)size);
+ int unpause;
+ av_fifo_read(c->fifo, buf, n);
+ /* Resume a paused transfer once the FIFO is at least half empty.
*/
+ unpause = c->paused && av_fifo_can_write(c->fifo) * 2 >=
c->buffer_size;
+ pthread_mutex_unlock(&c->mutex);
+ if (unpause)
+ curl_dispatch(c->loop, CMD_UNPAUSE, c, 0);
+ return n;
+ }
+ if (c->error) {
+ ret = c->error;
+ break;
+ }
+ if (c->eof) {
+ ret = AVERROR_EOF;
+ break;
+ }
+ if (nonblock) {
+ ret = AVERROR(EAGAIN);
+ break;
+ }
+ t = av_gettime() + CURL_WAIT_US;
+ ts.tv_sec = t / 1000000;
+ ts.tv_nsec = (t % 1000000) * 1000;
+ pthread_cond_timedwait(&c->cond, &c->mutex, &ts);
+ /* Return to the avio layer so it can poll the interrupt callback. */
+ nonblock = 1;
+ }
+ pthread_mutex_unlock(&c->mutex);
+
+ return ret;
}
static int libcurl_close(URLContext *h)
{
+ CurlContext *c = h->priv_data;
+
+ if (c->loop) {
+ if (c->easy) {
+ /* Ensure the handle is out of the multi before we free it. */
+ curl_dispatch(c->loop, CMD_REMOVE, c, 1);
+ curl_easy_cleanup(c->easy);
+ c->easy = NULL;
+ }
+ /* A shared loop outlives the transfer for connection reuse. */
+ if (c->private_loop)
+ curl_loop_destroy(c->loop);
+ c->loop = NULL;
+ }
+
+ av_fifo_freep2(&c->fifo);
+ pthread_cond_destroy(&c->cond);
+ pthread_mutex_destroy(&c->mutex);
+
return 0;
}
--
2.52.0
From 3ff7786064737a55b5d5765d62ff8f100c3eb4cf Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 03:03:57 +0200
Subject: [PATCH 4/6] avformat/libcurl: add seek, size and retry
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Probe Accept-Ranges/Content-Range/Content-Encoding to determine
seekability and total size, issue ranged requests via CURLOPT_RANGE,
implement url_seek (including AVSEEK_SIZE) as a reconnect at the new
offset, and resume seekable transfers from the next missing byte after a
recoverable error.
Note that we prefer the compression over seekability, as servers are
unlikely to compress media files in practice, as this would be huge
performance cost for zero gain. However downloading text playlists
benefits a lot from compression. YouTube for example can serve text
files that have 30MB+ which is huge bandwidth usage when not compressed.
Signed-off-by: Kacper Michajłow <[email protected]>
---
libavformat/libcurl.c | 216 ++++++++++++++++++++++++++++++++++++++----
1 file changed, 200 insertions(+), 16 deletions(-)
diff --git a/libavformat/libcurl.c b/libavformat/libcurl.c
index 4c16c94f6e..8b68d9c904 100644
--- a/libavformat/libcurl.c
+++ b/libavformat/libcurl.c
@@ -22,6 +22,9 @@
#include "config_components.h"
#include <curl/curl.h>
+#include <inttypes.h>
+#include <stdlib.h>
+#include <string.h>
#include "libavutil/avstring.h"
#include "libavutil/error.h"
@@ -47,12 +50,14 @@ enum cmd_kind {
CMD_ADD, /* add the easy handle to the multi and start the transfer */
CMD_REMOVE, /* remove the easy handle from the multi */
CMD_UNPAUSE, /* resume a transfer paused because the FIFO was full */
+ CMD_SEEK, /* restart the transfer at a new byte offset */
};
typedef struct CurlCmd {
enum cmd_kind kind;
CurlContext *ctx;
- int sync; /* caller waits for completion, flips by done */
+ int64_t pos; /* CMD_SEEK target offset */
+ int sync; /* caller waits for completion */
int done;
struct CurlCmd *next;
} CurlCmd;
@@ -76,9 +81,20 @@ struct CurlContext {
CURL *easy;
int64_t buffer_size;
+ int max_retries;
+
+ int64_t logical_pos; /* next byte url_read() will return, caller
side */
/* Producer bookkeeping, touched only by the loop thread. */
- int active; /* currently added to the multi */
+ int active; /* currently added to the multi */
+ uint64_t request_start; /* absolute offset the current request
began at */
+ uint64_t request_received;/* bytes delivered in the current request
*/
+ int retry_count; /* consecutive recoverable failures */
+
+ /* Per-response-block header scratch, loop thread only. */
+ int hdr_accept_ranges;
+ int hdr_compressed;
+ int64_t hdr_content_total;
/* Probe result. Set by the loop thread, read by url_open() once probed. */
int probed;
@@ -145,6 +161,24 @@ static int curlcode_to_averror(CURLcode code)
}
}
+static int is_recoverable(CURLcode code)
+{
+ switch (code) {
+ case CURLE_RECV_ERROR:
+ case CURLE_SEND_ERROR:
+ case CURLE_PARTIAL_FILE:
+ case CURLE_OPERATION_TIMEDOUT:
+ case CURLE_GOT_NOTHING:
+ case CURLE_COULDNT_CONNECT:
+ case CURLE_COULDNT_RESOLVE_HOST:
+ case CURLE_HTTP2:
+ case CURLE_HTTP2_STREAM:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
static int http_status_to_averror(long status)
{
switch (status) {
@@ -187,12 +221,23 @@ static size_t write_callback(char *ptr, size_t size,
size_t nmemb, void *userdat
av_fifo_write(c->fifo, ptr, bytes);
c->paused = 0;
+ c->request_received += bytes;
pthread_cond_broadcast(&c->cond);
pthread_mutex_unlock(&c->mutex);
return bytes;
}
+/* Parse the total length out of a "Content-Range: bytes a-b/total" value.
+ * Returns the total, or -1 if unknown ("*") or unparsable. */
+static int64_t parse_content_range_total(const char *v)
+{
+ const char *slash = strchr(v, '/');
+ if (!slash || slash[1] == '*')
+ return -1;
+ return strtoll(slash + 1, NULL, 10);
+}
+
static size_t header_callback(char *ptr, size_t size, size_t nitems, void
*userdata)
{
CurlContext *c = userdata;
@@ -200,7 +245,26 @@ static size_t header_callback(char *ptr, size_t size,
size_t nitems, void *userd
size_t n = len;
long status = 0;
- /* Act only on the blank line that terminates a header block. */
+ if (av_strncasecmp(ptr, "HTTP/", 5) == 0) {
+ c->hdr_accept_ranges = 0;
+ c->hdr_compressed = 0;
+ c->hdr_content_total = -1;
+ return len;
+ }
+ if (av_strncasecmp(ptr, "Accept-Ranges:", 14) == 0) {
+ c->hdr_accept_ranges = !!av_stristr(ptr + 14, "bytes");
+ return len;
+ }
+ if (av_strncasecmp(ptr, "Content-Encoding:", 17) == 0) {
+ c->hdr_compressed = !av_stristr(ptr + 17, "identity");
+ return len;
+ }
+ if (av_strncasecmp(ptr, "Content-Range:", 14) == 0) {
+ c->hdr_content_total = parse_content_range_total(ptr + 14);
+ return len;
+ }
+
+ /* Otherwise act only on the blank line that terminates the header block.
*/
while (n && (ptr[n - 1] == '\r' || ptr[n - 1] == '\n'))
n--;
if (n)
@@ -215,6 +279,22 @@ static size_t header_callback(char *ptr, size_t size,
size_t nitems, void *userd
pthread_mutex_lock(&c->mutex);
if (status >= 200 && status < 300) {
c->stream_ok = 1;
+ /* A compressed body is addressed in encoded form, so byte offsets are
+ * meaningless: not seekable. Note that we prefer compression over
+ * seekability, servers doesn't offer media in compressed form, so it
+ * gives us free compression for other payloads like text playlist. */
+ c->seekable = !c->hdr_compressed &&
+ (status == 206 || c->hdr_accept_ranges);
+ if (c->seekable) {
+ int64_t total = c->hdr_content_total;
+ if (total < 0 && status != 206) {
+ curl_off_t cl = -1;
+ if (curl_easy_getinfo(c->easy,
CURLINFO_CONTENT_LENGTH_DOWNLOAD_T,
+ &cl) == CURLE_OK && cl >= 0)
+ total = cl;
+ }
+ c->content_size = total;
+ }
} else {
c->stream_ok = 0;
if (!c->error)
@@ -238,23 +318,71 @@ static int xferinfo_callback(void *userdata, curl_off_t
dltotal, curl_off_t dlno
return aborted; /* non-zero aborts the transfer */
}
+/* (Re)issue the request for the current offset and add it to the multi. Loop
+ * thread only. */
+static void start_request(CurlContext *c)
+{
+ if (!c->probed || c->seekable) {
+ char range[32];
+ snprintf(range, sizeof(range), "%"PRIu64"-", c->request_start);
+ curl_easy_setopt(c->easy, CURLOPT_RANGE, range);
+ } else {
+ curl_easy_setopt(c->easy, CURLOPT_RANGE, NULL);
+ }
+ c->request_received = 0;
+ c->active = 1;
+ curl_multi_add_handle(c->loop->multi, c->easy);
+}
+
/* Transfer finished (or failed) */
static void on_done(CurlContext *c, CURLcode code)
{
+ int aborted;
+
pthread_mutex_lock(&c->mutex);
+ aborted = c->aborted;
+ /* Advance past delivered bytes so a retry or seek resumes at the right
offset. */
+ c->request_start += c->request_received;
+ c->request_received = 0;
+ pthread_mutex_unlock(&c->mutex);
+
if (!c->probed) {
/* Connection died before any usable header arrived. */
- c->probed = 1;
+ pthread_mutex_lock(&c->mutex);
+ c->probed = 1;
c->stream_ok = 0;
if (!c->error)
c->error = curlcode_to_averror(code);
- } else if (code == CURLE_OK && !c->aborted) {
- c->eof = 1;
- } else if (!c->aborted && !c->error) {
- c->error = curlcode_to_averror(code);
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
+ return;
+ }
+
+ if (code == CURLE_OK && !aborted && c->stream_ok) {
+ pthread_mutex_lock(&c->mutex);
+ c->eof = 1;
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
+ return;
+ }
+
+ /* Resume seekable transfers after a recoverable error. */
+ if (!aborted && c->seekable && is_recoverable(code) &&
+ c->retry_count < c->max_retries) {
+ c->retry_count++;
+ av_log(c->h, AV_LOG_WARNING, "%s, retrying (#%d) from %"PRIu64"\n",
+ curl_easy_strerror(code), c->retry_count, c->request_start);
+ start_request(c);
+ return;
+ }
+
+ if (!aborted) {
+ pthread_mutex_lock(&c->mutex);
+ if (!c->error)
+ c->error = curlcode_to_averror(code);
+ pthread_cond_broadcast(&c->cond);
+ pthread_mutex_unlock(&c->mutex);
}
- pthread_cond_broadcast(&c->cond);
- pthread_mutex_unlock(&c->mutex);
}
/* ------------------------------------------------------------------------- */
@@ -267,8 +395,7 @@ static void execute_command(CurlLoop *loop, CurlCmd *cmd)
switch (cmd->kind) {
case CMD_ADD:
- c->active = 1;
- curl_multi_add_handle(loop->multi, c->easy);
+ start_request(c);
break;
case CMD_REMOVE:
if (c->active) {
@@ -279,6 +406,21 @@ static void execute_command(CurlLoop *loop, CurlCmd *cmd)
case CMD_UNPAUSE:
curl_easy_pause(c->easy, CURLPAUSE_CONT);
break;
+ case CMD_SEEK:
+ if (c->active) {
+ curl_multi_remove_handle(loop->multi, c->easy);
+ c->active = 0;
+ }
+ pthread_mutex_lock(&c->mutex);
+ av_fifo_reset2(c->fifo);
+ c->paused = 0;
+ c->eof = 0;
+ c->error = 0;
+ pthread_mutex_unlock(&c->mutex);
+ c->request_start = cmd->pos;
+ c->retry_count = 0;
+ start_request(c);
+ break;
}
}
@@ -341,7 +483,8 @@ static void *curl_worker(void *arg)
/* Dispatch a command to the loop. For sync commands the caller blocks until
the
* loop thread has executed it. Returns 0 or a negative AVERROR. */
-static int curl_dispatch(CurlLoop *loop, enum cmd_kind kind, CurlContext *c,
int sync)
+static int curl_dispatch(CurlLoop *loop, enum cmd_kind kind, CurlContext *c,
+ int64_t pos, int sync)
{
CurlCmd stackcmd = {0};
CurlCmd *cmd = sync ? &stackcmd : av_mallocz(sizeof(*cmd));
@@ -351,6 +494,7 @@ static int curl_dispatch(CurlLoop *loop, enum cmd_kind
kind, CurlContext *c, int
cmd->kind = kind;
cmd->ctx = c;
+ cmd->pos = pos;
cmd->sync = sync;
pthread_mutex_lock(&loop->mutex);
@@ -525,6 +669,7 @@ static int libcurl_open(URLContext *h, const char *url, int
flags,
c->h = h;
c->content_size = -1;
+ c->max_retries = 5;
if (c->buffer_size <= 0)
c->buffer_size = CURL_DEFAULT_BUFFER_SIZE;
@@ -552,7 +697,7 @@ static int libcurl_open(URLContext *h, const char *url, int
flags,
}
setup_curl(c);
- ret = curl_dispatch(c->loop, CMD_ADD, c, 0);
+ ret = curl_dispatch(c->loop, CMD_ADD, c, 0, 0);
if (ret < 0)
goto fail;
@@ -587,9 +732,10 @@ static int libcurl_read(URLContext *h, unsigned char *buf,
int size)
av_fifo_read(c->fifo, buf, n);
/* Resume a paused transfer once the FIFO is at least half empty.
*/
unpause = c->paused && av_fifo_can_write(c->fifo) * 2 >=
c->buffer_size;
+ c->logical_pos += n;
pthread_mutex_unlock(&c->mutex);
if (unpause)
- curl_dispatch(c->loop, CMD_UNPAUSE, c, 0);
+ curl_dispatch(c->loop, CMD_UNPAUSE, c, 0, 0);
return n;
}
if (c->error) {
@@ -616,6 +762,43 @@ static int libcurl_read(URLContext *h, unsigned char *buf,
int size)
return ret;
}
+static int64_t libcurl_seek(URLContext *h, int64_t pos, int whence)
+{
+ CurlContext *c = h->priv_data;
+ int64_t newpos;
+
+ if (whence == AVSEEK_SIZE)
+ return c->content_size >= 0 ? c->content_size : AVERROR(ENOSYS);
+
+ if (!c->seekable)
+ return AVERROR(ENOSYS);
+
+ switch (whence) {
+ case SEEK_SET:
+ newpos = pos;
+ break;
+ case SEEK_CUR:
+ newpos = c->logical_pos + pos;
+ break;
+ case SEEK_END:
+ if (c->content_size < 0)
+ return AVERROR(ENOSYS);
+ newpos = c->content_size + pos;
+ break;
+ default:
+ return AVERROR(EINVAL);
+ }
+ if (newpos < 0)
+ return AVERROR(EINVAL);
+
+ /* Restart the transfer at the new offset. Any failure of the new request
+ * surfaces on the following url_read(). */
+ curl_post(c->loop, CMD_SEEK, c, newpos, 1);
+ c->logical_pos = newpos;
+
+ return newpos;
+}
+
static int libcurl_close(URLContext *h)
{
CurlContext *c = h->priv_data;
@@ -623,7 +806,7 @@ static int libcurl_close(URLContext *h)
if (c->loop) {
if (c->easy) {
/* Ensure the handle is out of the multi before we free it. */
- curl_dispatch(c->loop, CMD_REMOVE, c, 1);
+ curl_dispatch(c->loop, CMD_REMOVE, c, 0, 1);
curl_easy_cleanup(c->easy);
c->easy = NULL;
}
@@ -650,6 +833,7 @@ const URLProtocol ff_libcurl_protocol = {
.name = "libcurl",
.url_open2 = libcurl_open,
.url_read = libcurl_read,
+ .url_seek = libcurl_seek,
.url_close = libcurl_close,
.priv_data_size = sizeof(CurlContext),
.priv_data_class = &libcurl_context_class,
--
2.52.0
From 496fba3ae05bbcf0c5b6c4ee12894258a12c88f7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 03:34:03 +0200
Subject: [PATCH 5/6] avformat/libcurl: add options, TLS, proxy, cookies and
headers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add the AVOption table and map it onto libcurl. Verbose curl output is
routed to av_log at debug level. This mirrors subset of http.c options.
Signed-off-by: Kacper Michajłow <[email protected]>
---
doc/protocols.texi | 88 ++++++++++++++++
libavformat/libcurl.c | 231 ++++++++++++++++++++++++++++++++++++++++--
2 files changed, 311 insertions(+), 8 deletions(-)
diff --git a/doc/protocols.texi b/doc/protocols.texi
index 0ac28e78b5..4bdd24f6c0 100644
--- a/doc/protocols.texi
+++ b/doc/protocols.texi
@@ -1060,6 +1060,94 @@ The Real-Time Messaging Protocol tunneled through HTTPS
(RTMPTS) is used
for streaming multimedia content within HTTPS requests to traverse
firewalls.
+@section libcurl
+
+HTTP and HTTPS access through the libcurl library, available as an alternative
to
+the native @code{http} protocol when FFmpeg is built with
@code{--enable-libcurl}.
+It runs transfers on a dedicated event-loop thread shared per
+@code{AVFormatContext}, so all of a demuxer's connections (for example HLS
+segments) reuse libcurl's connection cache and HTTP/{2,3} multiplexing.
+
+The native @code{http} protocol stays the default; to use libcurl explicitly,
+prefix the URL with @code{libcurl:}, for example
+@code{libcurl:https://example.com/video.mp4}.
+
+The libcurl protocol accepts the following options.
+
+@table @option
+@item user_agent
+Override the @code{User-Agent} header. Defaults to @code{Lavf/<version>}.
+
+@item referer
+Set the @code{Referer} header.
+
+@item headers
+Set custom HTTP headers, overriding internal defaults. Multiple headers are
+separated by newlines.
+
+@item http_proxy
+Set the HTTP proxy to tunnel through.
+
+@item cookies
+Cookies to be sent in future requests, given as newline-delimited
+@code{Set-Cookie}-style values.
+
+@item location
+The actual location of the data after following redirects. This is an output
+option, read with @command{av_opt_get}; demuxers use it to resolve relative
URLs.
+
+@item offset
+Initial byte offset of the request. Default is @code{0}.
+
+@item end_offset
+If greater than @code{0}, limit the request to the bytes preceding this offset.
+Together with @option{offset} this requests a sub-range of the resource, as
used
+for HLS byte-range segments.
+
+@item seekable
+Control seekability of the connection. If set to @code{0} the stream is treated
+as non-seekable, @code{1} forces seekable, and @code{-1} (the default) lets it
be
+detected from the server response.
+
+@item tls_verify
+Verify the peer certificate and host name. Enabled by default.
+
+@item ca_file
+Path to a certificate authority bundle that overrides the system store.
+
+@item cert_file
+@item key_file
+Client certificate and private key files for mutual TLS.
+
+@item connect_timeout
+Connection timeout in seconds. @code{0} (the default) uses libcurl's own
timeout.
+
+@item max_redirects
+Maximum number of redirects to follow. Default is @code{16}.
+
+@item multiple_requests
+Reuse the connection across requests (HTTP keep-alive). Enabled by default;
+setting it to @code{0} uses a fresh connection per transfer.
+
+@item http_version
+HTTP version to use: @code{auto} (default), @code{1.0}, @code{1.1}, @code{2},
+@code{2tls}, @code{2-prior-knowledge}, @code{3} or @code{3only}. Versions newer
+than the libcurl FFmpeg was built against fall back to @code{auto}.
+
+@item buffer_size
+Size in bytes of the receive buffer. Default is 4 MiB.
+
+@item request_size
+If greater than @code{0}, split a seekable transfer into ranged requests of at
+most this many bytes each. Default is @code{0} (a single request).
+
+@item max_retries
+Maximum number of retries after a recoverable error on a seekable transfer.
+Default is @code{5}.
+@end table
+
+For more information see: @url{https://curl.se/libcurl/}.
+
@section libsmbclient
libsmbclient permits one to manipulate CIFS/SMB network resources.
diff --git a/libavformat/libcurl.c b/libavformat/libcurl.c
index 8b68d9c904..132631be8a 100644
--- a/libavformat/libcurl.c
+++ b/libavformat/libcurl.c
@@ -23,12 +23,14 @@
#include <curl/curl.h>
#include <inttypes.h>
+#include <limits.h>
#include <stdlib.h>
#include <string.h>
#include "libavutil/avstring.h"
#include "libavutil/error.h"
#include "libavutil/fifo.h"
+#include "libavutil/log.h"
#include "libavutil/macros.h"
#include "libavutil/mem.h"
#include "libavutil/opt.h"
@@ -38,8 +40,17 @@
#include "avformat.h"
#include "internal.h"
#include "url.h"
+#include "version.h"
+#define DEFAULT_USER_AGENT "Lavf/" AV_STRINGIFY(LIBAVFORMAT_VERSION)
#define CURL_DEFAULT_BUFFER_SIZE (4 << 20)
+
+#ifndef CURL_HTTP_VERSION_3
+#define CURL_HTTP_VERSION_3 CURL_HTTP_VERSION_NONE
+#endif
+#ifndef CURL_HTTP_VERSION_3ONLY
+#define CURL_HTTP_VERSION_3ONLY CURL_HTTP_VERSION_NONE
+#endif
/* Blocking waits wake up this often so url_read()/open can poll the interrupt
* callback. */
#define CURL_WAIT_US 100000
@@ -79,8 +90,28 @@ struct CurlContext {
CurlLoop *loop;
int private_loop; /* loop is owned by this context (not
shared) */
CURL *easy;
+ struct curl_slist *header_list;
+ /* AVOptions. */
+ char *user_agent;
+ char *referer;
+ char *headers;
+ char *http_proxy;
+ char *cookies;
+ char *ca_file;
+ char *cert_file;
+ char *key_file;
+ char *location; /* effective URL after redirects (output) */
+ int64_t off; /* initial byte offset */
+ int64_t end_off; /* exclusive upper byte bound (0 = none) */
+ int tls_verify;
+ int seekable_opt;
+ int connect_timeout;
+ int max_redirects;
+ int multiple_requests;
+ int http_version;
int64_t buffer_size;
+ int64_t request_size;
int max_retries;
int64_t logical_pos; /* next byte url_read() will return, caller
side */
@@ -279,6 +310,19 @@ static size_t header_callback(char *ptr, size_t size,
size_t nitems, void *userd
pthread_mutex_lock(&c->mutex);
if (status >= 200 && status < 300) {
c->stream_ok = 1;
+ /* Capture the post-redirect URL, this is exposed as "location"
AVOption
+ * for compatibility with http.c. */
+ if (!c->probed) {
+ const char *eff = NULL;
+ if (curl_easy_getinfo(c->easy, CURLINFO_EFFECTIVE_URL, &eff) ==
CURLE_OK
+ && eff) {
+ char *dup = av_strdup(eff);
+ if (dup) {
+ av_free(c->location);
+ c->location = dup;
+ }
+ }
+ }
/* A compressed body is addressed in encoded form, so byte offsets are
* meaningless: not seekable. Note that we prefer compression over
* seekability, servers doesn't offer media in compressed form, so it
@@ -323,8 +367,20 @@ static int xferinfo_callback(void *userdata, curl_off_t
dltotal, curl_off_t dlno
static void start_request(CurlContext *c)
{
if (!c->probed || c->seekable) {
- char range[32];
- snprintf(range, sizeof(range), "%"PRIu64"-", c->request_start);
+ uint64_t start = c->request_start;
+ char range[48];
+ if (c->request_size > 0 || c->end_off > 0) {
+ uint64_t end = UINT64_MAX;
+ if (c->request_size > 0)
+ end = start + c->request_size - 1;
+ if (c->content_size > 0)
+ end = FFMIN(end, (uint64_t)c->content_size - 1);
+ if (c->end_off > 0)
+ end = FFMIN(end, (uint64_t)c->end_off - 1);
+ snprintf(range, sizeof(range), "%"PRIu64"-%"PRIu64, start, end);
+ } else {
+ snprintf(range, sizeof(range), "%"PRIu64"-", start);
+ }
curl_easy_setopt(c->easy, CURLOPT_RANGE, range);
} else {
curl_easy_setopt(c->easy, CURLOPT_RANGE, NULL);
@@ -337,12 +393,14 @@ static void start_request(CurlContext *c)
/* Transfer finished (or failed) */
static void on_done(CurlContext *c, CURLcode code)
{
+ uint64_t received;
int aborted;
pthread_mutex_lock(&c->mutex);
- aborted = c->aborted;
+ aborted = c->aborted;
+ received = c->request_received;
/* Advance past delivered bytes so a retry or seek resumes at the right
offset. */
- c->request_start += c->request_received;
+ c->request_start += received;
c->request_received = 0;
pthread_mutex_unlock(&c->mutex);
@@ -359,6 +417,17 @@ static void on_done(CurlContext *c, CURLcode code)
}
if (code == CURLE_OK && !aborted && c->stream_ok) {
+ c->retry_count = 0;
+ if (c->seekable && c->request_size > 0) {
+ int64_t file_end = c->end_off > 0 ? c->end_off : c->content_size;
+ int more = file_end > 0
+ ? (int64_t)c->request_start < file_end
+ : received >= (uint64_t)c->request_size;
+ if (more) {
+ start_request(c);
+ return;
+ }
+ }
pthread_mutex_lock(&c->mutex);
c->eof = 1;
pthread_cond_broadcast(&c->cond);
@@ -604,6 +673,58 @@ void ff_curl_loop_free(void **loop)
static int libcurl_close(URLContext *h);
+static int debug_callback(CURL *easy, curl_infotype type, char *data,
+ size_t size, void *userdata)
+{
+ CurlContext *c = userdata;
+ const char *prefix, *p = data, *end = data + size;
+
+ switch (type) {
+ case CURLINFO_TEXT: prefix = "* "; break;
+ case CURLINFO_HEADER_IN: prefix = "< "; break;
+ case CURLINFO_HEADER_OUT: prefix = "> "; break;
+ default: return 0;
+ }
+
+ /* Split multiline payload into each log. */
+ while (p < end) {
+ const char *nl = memchr(p, '\n', end - p);
+ size_t len = (nl ? nl : end) - p;
+ while (len && p[len - 1] == '\r')
+ len--;
+ av_log(c->h, AV_LOG_DEBUG, "%s%.*s\n", prefix, (int)len, p);
+ if (!nl)
+ break;
+ p = nl + 1;
+ }
+ return 0;
+}
+
+/* Build the custom request header list from the referer and headers options.
*/
+static struct curl_slist *build_headers(CurlContext *c)
+{
+ struct curl_slist *list = NULL;
+
+ if (c->referer && c->referer[0]) {
+ char *h = av_asprintf("Referer: %s", c->referer);
+ if (h) {
+ list = curl_slist_append(list, h);
+ av_free(h);
+ }
+ }
+ if (c->headers && c->headers[0]) {
+ char *copy = av_strdup(c->headers);
+ char *line, *saveptr = NULL;
+ if (copy) {
+ for (line = av_strtok(copy, "\r\n", &saveptr); line;
+ line = av_strtok(NULL, "\r\n", &saveptr))
+ list = curl_slist_append(list, line);
+ av_free(copy);
+ }
+ }
+ return list;
+}
+
static void setup_curl(CurlContext *c)
{
CURL *e = c->easy;
@@ -625,9 +746,58 @@ static void setup_curl(CurlContext *c)
curl_easy_setopt(e, CURLOPT_XFERINFOFUNCTION, xferinfo_callback);
curl_easy_setopt(e, CURLOPT_XFERINFODATA, c);
+ if (av_log_get_level() >= AV_LOG_DEBUG) {
+ curl_easy_setopt(e, CURLOPT_VERBOSE, 1L);
+ curl_easy_setopt(e, CURLOPT_DEBUGFUNCTION, debug_callback);
+ curl_easy_setopt(e, CURLOPT_DEBUGDATA, c);
+ }
+
curl_easy_setopt(e, CURLOPT_FOLLOWLOCATION, 1L);
- curl_easy_setopt(e, CURLOPT_TCP_KEEPALIVE, 1L);
+ curl_easy_setopt(e, CURLOPT_MAXREDIRS, (long)c->max_redirects);
+ curl_easy_setopt(e, CURLOPT_HTTP_VERSION, (long)c->http_version);
+ curl_easy_setopt(e, CURLOPT_TCP_KEEPALIVE, c->multiple_requests ? 1L : 0L);
+ curl_easy_setopt(e, CURLOPT_FORBID_REUSE, c->multiple_requests ? 0L : 1L);
+ curl_easy_setopt(e, CURLOPT_HSTS_CTRL, (long)CURLHSTS_ENABLE);
curl_easy_setopt(e, CURLOPT_ACCEPT_ENCODING, "");
+ if (c->connect_timeout > 0)
+ curl_easy_setopt(e, CURLOPT_CONNECTTIMEOUT_MS,
+ (long)c->connect_timeout * 1000);
+
+ if (c->user_agent && c->user_agent[0])
+ curl_easy_setopt(e, CURLOPT_USERAGENT, c->user_agent);
+ if (c->http_proxy && c->http_proxy[0])
+ curl_easy_setopt(e, CURLOPT_PROXY, c->http_proxy);
+
+ curl_easy_setopt(e, CURLOPT_SSL_OPTIONS, (long)CURLSSLOPT_NATIVE_CA);
+ curl_easy_setopt(e, CURLOPT_SSL_VERIFYPEER, c->tls_verify ? 1L : 0L);
+ curl_easy_setopt(e, CURLOPT_SSL_VERIFYHOST, c->tls_verify ? 2L : 0L);
+ if (c->ca_file)
+ curl_easy_setopt(e, CURLOPT_CAINFO, c->ca_file);
+ if (c->cert_file)
+ curl_easy_setopt(e, CURLOPT_SSLCERT, c->cert_file);
+ if (c->key_file)
+ curl_easy_setopt(e, CURLOPT_SSLKEY, c->key_file);
+
+ if (c->cookies && c->cookies[0]) {
+ char *copy = av_strdup(c->cookies);
+ char *line, *saveptr = NULL;
+ curl_easy_setopt(e, CURLOPT_COOKIEFILE, ""); /* enable the cookie
engine */
+ if (copy) {
+ for (line = av_strtok(copy, "\r\n", &saveptr); line;
+ line = av_strtok(NULL, "\r\n", &saveptr)) {
+ char *sc = av_asprintf("Set-Cookie: %s", line);
+ if (sc) {
+ curl_easy_setopt(e, CURLOPT_COOKIELIST, sc);
+ av_free(sc);
+ }
+ }
+ av_free(copy);
+ }
+ }
+
+ c->header_list = build_headers(c);
+ if (c->header_list)
+ curl_easy_setopt(e, CURLOPT_HTTPHEADER, c->header_list);
}
/* Block until the transfer has been probed, the stream errored, or the open
was
@@ -665,13 +835,18 @@ static int libcurl_open(URLContext *h, const char *url,
int flags,
AVDictionary **options)
{
CurlContext *c = h->priv_data;
+ const char *eff_url = h->filename;
int ret;
c->h = h;
c->content_size = -1;
- c->max_retries = 5;
- if (c->buffer_size <= 0)
- c->buffer_size = CURL_DEFAULT_BUFFER_SIZE;
+ c->request_start = c->off;
+ c->logical_pos = c->off;
+
+ /* Report the request URL until header_callback replaces it post-redirect.
*/
+ av_strstart(eff_url, "libcurl:", &eff_url);
+ av_freep(&c->location);
+ c->location = av_strdup(eff_url);
if (pthread_mutex_init(&c->mutex, NULL))
return AVERROR(ENOMEM);
@@ -705,6 +880,8 @@ static int libcurl_open(URLContext *h, const char *url, int
flags,
if (ret < 0)
goto fail;
+ if (c->seekable_opt == 0)
+ c->seekable = 0;
h->is_streamed = !c->seekable;
return 0;
@@ -816,6 +993,8 @@ static int libcurl_close(URLContext *h)
c->loop = NULL;
}
+ if (c->header_list)
+ curl_slist_free_all(c->header_list);
av_fifo_freep2(&c->fifo);
pthread_cond_destroy(&c->cond);
pthread_mutex_destroy(&c->mutex);
@@ -823,9 +1002,45 @@ static int libcurl_close(URLContext *h)
return 0;
}
+#define OFFSET(x) offsetof(CurlContext, x)
+#define D AV_OPT_FLAG_DECODING_PARAM
+#define E AV_OPT_FLAG_ENCODING_PARAM
+static const AVOption options[] = {
+ { "user_agent", "override User-Agent header", OFFSET(user_agent),
AV_OPT_TYPE_STRING, { .str = DEFAULT_USER_AGENT }, 0, 0, D },
+ { "referer", "override Referer header", OFFSET(referer),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D },
+ { "headers", "set custom HTTP headers, can override built in default
headers", OFFSET(headers), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "http_proxy", "set HTTP proxy to tunnel through", OFFSET(http_proxy),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "cookies", "set cookies to be sent in applicable future requests, use
newline delimited Set-Cookie HTTP field value syntax", OFFSET(cookies),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D },
+ { "location", "the actual location of the data received",
OFFSET(location), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "offset", "initial byte offset", OFFSET(off), AV_OPT_TYPE_INT64, { .i64
= 0 }, 0, INT64_MAX, D },
+ { "end_offset", "try to limit the request to bytes preceding this offset",
OFFSET(end_off), AV_OPT_TYPE_INT64, { .i64 = 0 }, 0, INT64_MAX, D },
+ { "seekable", "control seekability of connection", OFFSET(seekable_opt),
AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, D },
+ { "tls_verify", "verify the peer certificate and hostname",
OFFSET(tls_verify), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, D | E },
+ { "ca_file", "certificate authority bundle file", OFFSET(ca_file),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "cert_file", "client certificate file", OFFSET(cert_file),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "key_file", "client private key file", OFFSET(key_file),
AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D | E },
+ { "connect_timeout", "connection timeout in seconds (0 = libcurl
default)", OFFSET(connect_timeout), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX,
D | E },
+ { "max_redirects", "maximum number of redirects to follow",
OFFSET(max_redirects), AV_OPT_TYPE_INT, { .i64 = 16 }, 0, INT_MAX, D },
+ { "multiple_requests", "reuse the connection across requests (HTTP
keep-alive)", OFFSET(multiple_requests), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1,
D | E },
+ { "max_retries", "maximum number of retries after a recoverable error",
OFFSET(max_retries), AV_OPT_TYPE_INT, { .i64 = 5 }, 0, INT_MAX, D },
+ { "buffer_size", "receive buffer size in bytes", OFFSET(buffer_size),
AV_OPT_TYPE_INT64, { .i64 = CURL_DEFAULT_BUFFER_SIZE }, 1024, INT64_MAX, D },
+ { "request_size", "split a transfer into ranged requests of at most this
many bytes (0 = unlimited)", OFFSET(request_size), AV_OPT_TYPE_INT64, { .i64 =
0 }, 0, INT64_MAX, D },
+ { "http_version", "HTTP version to use", OFFSET(http_version),
AV_OPT_TYPE_INT, { .i64 = CURL_HTTP_VERSION_NONE }, 0, INT_MAX, D, .unit =
"http_version" },
+ { "auto", "negotiate the best supported version", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_NONE }, 0, 0, D,
.unit = "http_version" },
+ { "1.0", "HTTP/1.0", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_1_0 }, 0, 0, D,
.unit = "http_version" },
+ { "1.1", "HTTP/1.1", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_1_1 }, 0, 0, D,
.unit = "http_version" },
+ { "2", "HTTP/2", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_2 }, 0, 0, D,
.unit = "http_version" },
+ { "2tls", "HTTP/2 over TLS only", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_2TLS }, 0, 0, D,
.unit = "http_version" },
+ { "2-prior-knowledge", "HTTP/2 without an upgrade handshake", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE }, 0, 0, D,
.unit = "http_version" },
+ { "3", "HTTP/3, fall back to earlier versions", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_3 }, 0, 0, D,
.unit = "http_version" },
+ { "3only", "HTTP/3 only", 0,
AV_OPT_TYPE_CONST, { .i64 = CURL_HTTP_VERSION_3ONLY }, 0, 0, D,
.unit = "http_version" },
+ { NULL }
+};
+
static const AVClass libcurl_context_class = {
.class_name = "libcurl",
.item_name = av_default_item_name,
+ .option = options,
.version = LIBAVUTIL_VERSION_INT,
};
--
2.52.0
From d2b7ffd4d26bc520c45eae55f72fdffb2b632243 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kacper=20Michaj=C5=82ow?= <[email protected]>
Date: Mon, 15 Jun 2026 03:44:00 +0200
Subject: [PATCH 6/6] avformat: add prefer_libcurl option to route http(s) via
libcurl
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Register a prefer_libcurl IO option that, when set, opens http(s) URLs
with the libcurl protocol instead of the native one. The native http
protocol stays the default.
Signed-off-by: Kacper Michajłow <[email protected]>
---
doc/protocols.texi | 21 +++++++++++++++++++--
libavformat/avio.c | 35 ++++++++++++++++++++++++++++++++++-
libavformat/url.h | 1 +
3 files changed, 54 insertions(+), 3 deletions(-)
diff --git a/doc/protocols.texi b/doc/protocols.texi
index 4bdd24f6c0..1ac6cdfdf3 100644
--- a/doc/protocols.texi
+++ b/doc/protocols.texi
@@ -1068,10 +1068,27 @@ It runs transfers on a dedicated event-loop thread
shared per
@code{AVFormatContext}, so all of a demuxer's connections (for example HLS
segments) reuse libcurl's connection cache and HTTP/{2,3} multiplexing.
-The native @code{http} protocol stays the default; to use libcurl explicitly,
-prefix the URL with @code{libcurl:}, for example
+The native @code{http} protocol stays the default. There are two ways to use
+libcurl instead:
+
+@itemize @bullet
+@item
+Prefix the URL with @code{libcurl:} to force it explicitly, for example
@code{libcurl:https://example.com/video.mp4}.
+@item
+Set the @option{prefer_libcurl} option to route @code{http://} and
+@code{https://} URLs through libcurl transparently.
+@end itemize
+
+@table @option
+@item prefer_libcurl
+When set to @code{1}, open @code{http(s)} URLs with the libcurl protocol
instead
+of the native one. This is a generic IO option rather than a libcurl-specific
+one, so it also applies on builds without libcurl (where it has no effect).
+Default is @code{0}.
+@end table
+
The libcurl protocol accepts the following options.
@table @option
diff --git a/libavformat/avio.c b/libavformat/avio.c
index 4e54cb6e8a..34b48924ea 100644
--- a/libavformat/avio.c
+++ b/libavformat/avio.c
@@ -19,6 +19,10 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "config_components.h"
+
+#include <stdlib.h>
+
#include "libavutil/avstring.h"
#include "libavutil/dict.h"
#include "libavutil/mem.h"
@@ -61,6 +65,7 @@ static const AVOption urlcontext_options[] = {
{"protocol_whitelist", "List of protocols that are allowed to be used",
OFFSET(protocol_whitelist), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, D },
{"protocol_blacklist", "List of protocols that are not allowed to be
used", OFFSET(protocol_blacklist), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0,
D },
{"rw_timeout", "Timeout for IO operations (in microseconds)",
offsetof(URLContext, rw_timeout), AV_OPT_TYPE_INT64, { .i64 = 0 }, 0,
INT64_MAX, AV_OPT_FLAG_ENCODING_PARAM | AV_OPT_FLAG_DECODING_PARAM },
+ {"prefer_libcurl", "use the libcurl protocol for http(s) URLs when
available", OFFSET(prefer_libcurl), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, D },
{ NULL }
};
@@ -365,6 +370,24 @@ int ffurl_alloc(URLContext **puc, const char *filename,
int flags,
return AVERROR_PROTOCOL_NOT_FOUND;
}
+#if CONFIG_LIBCURL_PROTOCOL
+extern const URLProtocol ff_libcurl_protocol;
+
+/* Decide whether an http(s) URL should be routed to the libcurl protocol
instead
+ * of the native one. Controlled by the per-open "prefer_libcurl" option. */
+static int prefer_libcurl(const char *filename, AVDictionary **options)
+{
+ AVDictionaryEntry *e;
+
+ if (!options || !(e = av_dict_get(*options, "prefer_libcurl", NULL, 0)) ||
+ !atoi(e->value))
+ return 0;
+
+ return av_strstart(filename, "http://";, NULL) ||
+ av_strstart(filename, "https://";, NULL);
+}
+#endif
+
static int url_open_whitelist(URLContext **puc, const char *filename, int
flags,
const AVIOInterruptCB *int_cb, AVDictionary
**options,
const char *whitelist, const char* blacklist,
@@ -372,7 +395,17 @@ static int url_open_whitelist(URLContext **puc, const char
*filename, int flags,
{
AVDictionary *tmp_opts = NULL;
AVDictionaryEntry *e;
- int ret = ffurl_alloc(puc, filename, flags, int_cb);
+ int ret;
+
+#if CONFIG_LIBCURL_PROTOCOL
+ /* The option itself is applied to the URLContext further down; here it
only
+ * picks the protocol, before the context exists. */
+ if (prefer_libcurl(filename, options))
+ ret = url_alloc_for_protocol(puc, &ff_libcurl_protocol, filename,
flags,
+ int_cb);
+ else
+#endif
+ ret = ffurl_alloc(puc, filename, flags, int_cb);
if (ret < 0)
return ret;
(*puc)->fmt_ctx = fmt_ctx;
diff --git a/libavformat/url.h b/libavformat/url.h
index f08d18bcb4..c6ecdb286b 100644
--- a/libavformat/url.h
+++ b/libavformat/url.h
@@ -47,6 +47,7 @@ typedef struct URLContext {
const char *protocol_blacklist;
int min_packet_size; /**< if non zero, the stream is packetized
with this min packet size */
void *fmt_ctx; /**< The AVFormatContext that opened this
URLContext, or NULL for standalone use */
+ int prefer_libcurl; /**< route http(s) opens through the libcurl
protocol */
} URLContext;
typedef struct URLProtocol {
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
