PR #25765 opened by Kilian (lusoris) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25765 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25765.patch
With `window_size` set, `dash_flush()` deleted segments that left the window before the new manifests were published. If publishing the MPD or an HLS media playlist then failed, the last published manifest still listed segments that were already deleted. This is the separate change for Minor 1 of the #25735 review. It is stacked on #25619, which already returns and checks the manifest and playlist write errors this change relies on; please review the top commit only. It supersedes nothing. Changes: - Eviction moves from `dash_flush()` to the end of a successful `write_manifest()`, including the path that skips the master playlist refresh. - With `hls_playlist`, eviction stops at the first segment of the last published media playlist, so LHLS playlists, which are updated separately, never list a deleted segment. - With `ignore_io_errors`, a failed close is reported as success. `write_manifest()` now tracks whether every manifest of the call was really published and skips the eviction otherwise. The write sequence and the returned values are unchanged. Overlap with #25735: it adds a one-line `ret >= 0 &&` guard to the eviction loop in `dash_flush()` that this change removes. `dash_flush()` calls `write_manifest()` only when `ret >= 0`, so the guard is no longer needed here; whichever of the two lands second drops that block. A merge of both has this one conflict and no other. Overlap with #25475: it also changes `dashenc.c` and already conflicts with #25619 in 12 places. This commit adds 3 more, two in `write_hls_media_playlist()` and one in `write_manifest()`. I will rebase whichever of them lands later. Overlap with #25749 (mine): it changes when `dashenc_delete_segment_file()` and `dashenc_delete_media_segments()` delete files. This commit moves the call to `dashenc_delete_media_segments()` and adds one `hls_start_index` line at its end. The two merge without conflicts, and the merged `dashenc.c` builds. Testing (on this branch, #25619 plus this commit): - An `LD_PRELOAD` shim failed the third and every later rename of one manifest (MPD, media playlist or master playlist) in normal, streaming and LHLS mode, with one and two streams. On #25619 alone, 10 of 18 cases left a manifest that listed a deleted segment; with this commit none do. - The same shim with `-ignore_io_errors 1` failed the third and every later close of one manifest, in the same 18 cases. Without the `ignore_io_errors` part of this change, 10 of 18 cases left a manifest that listed a deleted segment; with it none do. - Two of those 18 cases still fail the same way on #25619 alone and with this commit: with two streams in streaming or LHLS mode and a failing MPD rename, `master.m3u8` lists media playlists that are never published. That is a separate problem in #25619 and will be fixed there. - Without the injected failure, all 12 runs keep at most `window_size + extra_window_size` segments. - `make fate-dashenc fate-source` passes, and GCC reports the same 7 `-Wformat-truncation` warnings for `dashenc.c` with and without this commit. >From b070c248eb455272ca333749f853a8d54d783d43 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Fri, 30 Oct 2026 15:21:26 +0100 Subject: [PATCH 1/3] avformat/hlsplaylist: return playlist write errors The entry writer returned success even when AVIO had recorded a write error. Return out->error so callers can detect the failure. In hls_window, the existing warning "ff_hls_write_file_entry get error" now fires. The fail label still overwrites ret with hlsenc_io_close's result, as on master; close, retry and trailer behavior are unchanged. An ENOSPC callback now returns ENOSPC instead of zero. A /dev/full VOD playlist exceeding the AVIO buffer logs the warning and still exits zero with either ignore_io_errors value. HLS close stubs and the four DASH and nine sample-free HLS FATE tests pass. Signed-off-by: lusoris <[email protected]> --- libavformat/hlsplaylist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/hlsplaylist.c b/libavformat/hlsplaylist.c index 774bc5ac84b..55cfa294953 100644 --- a/libavformat/hlsplaylist.c +++ b/libavformat/hlsplaylist.c @@ -193,7 +193,7 @@ int ff_hls_write_file_entry(AVIOContext *out, int insert_discont, avio_printf(out, "%s", baseurl); avio_printf(out, "%s\n", filename); - return 0; + return out->error; } void ff_hls_write_end_list(AVIOContext *out) -- 2.54.0 >From 2b5b1b7f8fc257d6070fb7fa4e19f44ae458280f Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Fri, 30 Oct 2026 15:22:32 +0100 Subject: [PATCH 2/3] avformat/dashenc: propagate output close errors dashenc_io_close discarded close and HTTP shutdown results, allowing failed output writes to appear successful. Return these errors and buffered AVIO errors after shutdown, then check them in the manifest, playlist, initialization and media segment writers and their callers. Check AVIO write and flush errors on single-file output contexts during initialization, segment flushes and trailer writing, ensuring delayed write failures propagate to the trailer when dashenc_io_close is skipped. Return the results of av_write_trailer and of the single-file av_write_frame flush, and check streaming segment writes after each flush, so a reset connection stops muxing at the failing packet. Finish MPD publication before writing HLS media playlists, so a playlist failure cannot truncate the DASH manifest. Continue program-date bookkeeping after ignored entry write errors so later windows retain their wall-clock timestamps. Keep ignore_io_errors effective, including media playlist rename failures whose result was previously discarded. Attempt the media-playlist rename after successful writing or an ignored write failure; suppress its error only when ignore_io_errors is enabled. Other rename sites retain their existing error handling, including when returned by the trailer. After an ignored streaming write failure, close the failed output so the next segment opens a fresh one; with http_persistent, a failed flush or shutdown closes the persistent connection the same way instead of reusing its sticky error. After an ignored single-file write failure, log it once and drop the failed segment from the MPD and HLS byte ranges. On seekable output, clear the sticky AVIO error and seek back to the end of the last completed write, so every later advertised range matches the file offsets. On non-seekable output, with global_sidx, or when that seek fails, keep the sticky error and stop writing that representation. With global_sidx the mov muxer has already recorded the failed fragment for the global index, so resuming would leave the index pointing past the written data. A representation whose initialization segment failed is left out of the MPD and of all three HLS master playlist loops. Skip media-segment publication during teardown when initialization failed before the media path was established. With lavfi input and /dev/full targets, the base missed output errors. The fixed executable fails with ignore_io_errors=0 and exits zero with ignore_io_errors=1. A directory at media_0.m3u8 gives exit 235 or zero, respectively, with rename attempted in both cases. Delayed single-file stream write errors propagate non-zero exit unless ignore_io_errors is set. Standard, streaming and low-latency HLS checks pass, as do offline close/flush/shutdown stubs. With a write failure injected into a seekable single-file output, every initialization and media byte range in the MPD and HLS playlists starts at a box in the written file; a FIFO output stops writing after the failure. With global_sidx, the same injection previously left a global sidx whose last reference ended 1946 bytes past the end of the file; the representation now stops and no index is written. fate-dash-mpd-timing, fate-lavf-mp4 and the 19 HLS FATE tests pass. Against a loopback HTTP server that resets the connection once during the first media upload with http_persistent=1, a build without this handling never wrote segment 2 although the MPD lists it; now segment 2 arrives in full on a new connection. With an init-only write failure on one of two video, one of two audio, a sole audio or one of two audio-only representations, the HLS master playlist previously still listed the failed media playlist; now it lists only the surviving ones, the first surviving audio rendition becomes the default, and a failed sole audio leaves no audio group. When every video representation is dropped that way, the master playlist previously kept the video layout and listed no variant at all; it now falls back to the audio-only layout, so the surviving audio stays playable. The master-playlist bitrate is computed only for listed streams and only from a positive total duration: an output whose video stream received no packets previously died with SIGFPE in the trailer and now finishes with exit 0. A healthy video plus audio master playlist is byte-identical before and after. Signed-off-by: lusoris <[email protected]> --- libavformat/dashenc.c | 266 ++++++++++++++++++++++++++++++++---------- 1 file changed, 203 insertions(+), 63 deletions(-) diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index 36f39bd083a..b36d5792445 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -130,6 +130,7 @@ typedef struct OutputStream { char codec_str[100]; int written_len; + int io_error; char filename[1024]; char full_path[1024]; char temp_path[1024]; @@ -225,23 +226,33 @@ static int dashenc_io_open(AVFormatContext *s, AVIOContext **pb, char *filename, return err; } -static void dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) { +static int dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) +{ DASHContext *c = s->priv_data; int http_base_proto = filename ? ff_is_http_proto(filename) : 0; + int ret = 0; if (!*pb) - return; + return 0; if (!http_base_proto || !c->http_persistent) { - ff_format_io_close(s, pb); + ret = ff_format_io_close(s, pb); #if CONFIG_HTTP_PROTOCOL } else { URLContext *http_url_context = ffio_geturlcontext(*pb); av_assert0(http_url_context); avio_flush(*pb); - ffurl_shutdown(http_url_context, AVIO_FLAG_WRITE); + ret = ffurl_shutdown(http_url_context, AVIO_FLAG_WRITE); + if (ret >= 0) + ret = (*pb)->error; + if (ret < 0) + ff_format_io_close(s, pb); #endif } + if (ret < 0) + av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, + "Unable to write %s: %s\n", filename, av_err2str(ret)); + return c->ignore_io_errors ? 0 : ret; } static const char *get_format_str(SegmentType segment_type) @@ -315,16 +326,46 @@ static int init_segment_types(AVFormatContext *s) return 0; } +static void ignore_single_file_error(DASHContext *c, OutputStream *os) +{ + AVIOContext *pb = os->ctx->pb; + int err = pb->error; + int64_t end = ffiocontext(pb)->written_output_size; + + av_log(os->ctx, AV_LOG_WARNING, "Unable to write stream %td: %s\n", + os - c->streams, av_err2str(err)); + os->io_error = err; + os->packets_written = 0; + os->total_pkt_size = 0; + os->total_pkt_duration = 0; + if (!c->global_sidx && (pb->seekable & AVIO_SEEKABLE_NORMAL)) { + pb->error = 0; + if (avio_seek(pb, end, SEEK_SET) < 0) { + pb->error = err; + return; + } + os->pos = end; + /* An incomplete initialization segment cannot be recovered. */ + if (os->init_range_length) + os->io_error = 0; + } +} + static int flush_dynbuf(DASHContext *c, OutputStream *os, int *range_length) { uint8_t *buffer; + int ret, err; if (!os->ctx->pb) { return AVERROR(EINVAL); } + if (os->io_error) { + *range_length = 0; + return 0; + } // flush - av_write_frame(os->ctx, NULL); + ret = av_write_frame(os->ctx, NULL); avio_flush(os->ctx->pb); if (!c->single_file) { @@ -337,10 +378,17 @@ static int flush_dynbuf(DASHContext *c, OutputStream *os, int *range_length) av_free(buffer); // re-open buffer - return avio_open_dyn_buf(&os->ctx->pb); + err = avio_open_dyn_buf(&os->ctx->pb); + return ret < 0 ? ret : err; } else { *range_length = avio_tell(os->ctx->pb) - os->pos; - return 0; + if (c->ignore_io_errors && os->ctx->pb->error < 0) { + if (ret == os->ctx->pb->error) + ret = 0; + ignore_single_file_error(c, os); + *range_length = 0; + } + return ret < 0 ? ret : 0; } } @@ -375,16 +423,16 @@ static void get_start_index_number(OutputStream *os, DASHContext *c, } } -static void write_hls_media_playlist(OutputStream *os, AVFormatContext *s, - int representation_id, int final, - char *prefetch_url) { +static int write_hls_media_playlist(OutputStream *os, AVFormatContext *s, + int representation_id, int final, + char *prefetch_url) { DASHContext *c = s->priv_data; int timescale = os->ctx->streams[0]->time_base.den; char temp_filename_hls[1024]; char filename_hls[1024]; AVDictionary *http_opts = NULL; int target_duration = 0; - int ret = 0; + int ret = 0, err; const char *proto = avio_find_protocol_name(c->dirname); int use_rename = proto && !strcmp(proto, "file"); int i, start_index, start_number; @@ -394,7 +442,7 @@ static void write_hls_media_playlist(OutputStream *os, AVFormatContext *s, if (!c->hls_playlist || start_index >= os->nb_segments || os->segment_type != SEGMENT_TYPE_MP4) - return; + return 0; get_hls_playlist_name(filename_hls, sizeof(filename_hls), c->dirname, representation_id); @@ -404,10 +452,8 @@ static void write_hls_media_playlist(OutputStream *os, AVFormatContext *s, set_http_options(&http_opts, c); ret = dashenc_io_open(s, &c->m3u8_out, temp_filename_hls, &http_opts); av_dict_free(&http_opts); - if (ret < 0) { - handle_io_open_error(s, ret, temp_filename_hls); - return; - } + if (ret < 0) + return handle_io_open_error(s, ret, temp_filename_hls); for (i = start_index; i < os->nb_segments; i++) { Segment *seg = os->segments[i]; double duration = (double) seg->duration / timescale; @@ -437,9 +483,8 @@ static void write_hls_media_playlist(OutputStream *os, AVFormatContext *s, seg->range_length, seg->start_pos, NULL, c->single_file ? os->initfile : seg->file, &prog_date_time, 0); - if (ret < 0) { - av_log(os->ctx, AV_LOG_WARNING, "ff_hls_write_file_entry get error\n"); - } + if (ret < 0 && !c->ignore_io_errors) + goto fail; } if (prefetch_url) @@ -448,26 +493,40 @@ static void write_hls_media_playlist(OutputStream *os, AVFormatContext *s, if (final) ff_hls_write_end_list(c->m3u8_out); - dashenc_io_close(s, &c->m3u8_out, temp_filename_hls); +fail: + err = dashenc_io_close(s, &c->m3u8_out, temp_filename_hls); + if (ret >= 0) + ret = err; + if (ret < 0 && !c->ignore_io_errors) + return ret; if (use_rename) - ff_rename(temp_filename_hls, filename_hls, os->ctx); + ret = ff_rename(temp_filename_hls, filename_hls, os->ctx); + return c->ignore_io_errors ? 0 : ret; } static int flush_init_segment(AVFormatContext *s, OutputStream *os) { DASHContext *c = s->priv_data; int ret, range_length; + char filename[1024]; ret = flush_dynbuf(c, os, &range_length); if (ret < 0) return ret; + if (os->io_error) + return 0; os->pos = os->init_range_length = range_length; - if (!c->single_file) { - char filename[1024]; - snprintf(filename, sizeof(filename), "%s%s", c->dirname, os->initfile); - dashenc_io_close(s, &os->out, filename); + snprintf(filename, sizeof(filename), "%s%s", c->dirname, os->initfile); + if (!c->single_file) + return dashenc_io_close(s, &os->out, filename); + + if (os->ctx && os->ctx->pb && os->ctx->pb->error < 0) { + ret = os->ctx->pb->error; + av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, + "Unable to write %s: %s\n", filename, av_err2str(ret)); + return c->ignore_io_errors ? 0 : ret; } return 0; } @@ -515,7 +574,7 @@ static void dash_free(AVFormatContext *s) } static void output_segment_list(OutputStream *os, AVIOContext *out, AVFormatContext *s, - int representation_id, int final) + int final) { DASHContext *c = s->priv_data; int i, start_index, start_number; @@ -583,10 +642,6 @@ static void output_segment_list(OutputStream *os, AVIOContext *out, AVFormatCont } avio_printf(out, "\t\t\t\t</SegmentList>\n"); } - if (!c->lhls || final) { - write_hls_media_playlist(os, s, representation_id, final, NULL); - } - } static char *xmlescape(const char *str) { @@ -680,6 +735,8 @@ static int write_adaptation_set(AVFormatContext *s, AVIOContext *out, int as_ind if (os->as_idx - 1 != as_index) continue; + if (os->io_error && !os->init_range_length) + continue; if (os->bit_rate > 0) snprintf(bandwidth_str, sizeof(bandwidth_str), " bandwidth=\"%d\"", os->bit_rate); @@ -724,7 +781,7 @@ static int write_adaptation_set(AVFormatContext *s, AVIOContext *out, int as_ind if (!final && c->ldash && os->gop_size && os->frag_type != FRAG_TYPE_NONE && !(c->profile & MPD_PROFILE_DVB) && (os->frag_type != FRAG_TYPE_DURATION || os->frag_duration != os->seg_duration)) avio_printf(out, "\t\t\t\t<Resync dT=\"%"PRId64"\" type=\"1\"/>\n", os->gop_size); - output_segment_list(os, out, s, i, final); + output_segment_list(os, out, s, final); avio_printf(out, "\t\t\t</Representation>\n"); } avio_printf(out, "\t\t</AdaptationSet>\n"); @@ -983,6 +1040,25 @@ end: return 0; } +static int has_hls_video(AVFormatContext *s) +{ + DASHContext *c = s->priv_data; + int i; + + for (i = 0; i < s->nb_streams; i++) { + OutputStream *os = &c->streams[i]; + + if (s->streams[i]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) + continue; + if (os->io_error && !os->init_range_length) + continue; + if (os->segment_type != SEGMENT_TYPE_MP4) + continue; + return 1; + } + return 0; +} + static int write_manifest(AVFormatContext *s, int final) { DASHContext *c = s->priv_data; @@ -1100,13 +1176,25 @@ static int write_manifest(AVFormatContext *s, int final) avio_printf(out, "</MPD>\n"); avio_flush(out); - dashenc_io_close(s, &c->mpd_out, temp_filename); + ret = dashenc_io_close(s, &c->mpd_out, temp_filename); + if (ret < 0) + return ret; if (use_rename) { if ((ret = ff_rename(temp_filename, s->url, s)) < 0) return ret; } + // Publish HLS media playlists after completing the MPD, so their errors + // cannot interrupt MPD serialization. + if (!c->lhls || final) { + for (i = 0; i < s->nb_streams; i++) { + ret = write_hls_media_playlist(&c->streams[i], s, i, final, NULL); + if (ret < 0) + return ret; + } + } + if (c->hls_playlist) { char filename_hls[1024]; @@ -1131,7 +1219,7 @@ static int write_manifest(AVFormatContext *s, int final) ff_hls_write_playlist_version(c->m3u8_out, 7); - if (c->has_video) { + if (has_hls_video(s)) { // treat audio streams as alternative renditions for video streams const char *audio_group = "A1"; char audio_codec_str[128] = "\0"; @@ -1144,6 +1232,8 @@ static int write_manifest(AVFormatContext *s, int final) OutputStream *os = &c->streams[i]; if (st->codecpar->codec_type != AVMEDIA_TYPE_AUDIO) continue; + if (os->io_error && !os->init_range_length) + continue; if (os->segment_type != SEGMENT_TYPE_MP4) continue; get_hls_playlist_name(playlist_file, sizeof(playlist_file), NULL, i); @@ -1166,17 +1256,21 @@ static int write_manifest(AVFormatContext *s, int final) AVStream *st = s->streams[i]; OutputStream *os = &c->streams[i]; const char *agroup = NULL; - int stream_bitrate = os->muxer_overhead; - if (os->bit_rate > 0) - stream_bitrate += os->bit_rate; - else if (final) - stream_bitrate += os->pos * 8 * AV_TIME_BASE / c->total_duration; - else if (os->first_segment_bit_rate > 0) - stream_bitrate += os->first_segment_bit_rate; + int stream_bitrate; + if (st->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) continue; + if (os->io_error && !os->init_range_length) + continue; if (os->segment_type != SEGMENT_TYPE_MP4) continue; + stream_bitrate = os->muxer_overhead; + if (os->bit_rate > 0) + stream_bitrate += os->bit_rate; + else if (final && c->total_duration > 0) + stream_bitrate += os->pos * 8 * AV_TIME_BASE / c->total_duration; + else if (os->first_segment_bit_rate > 0) + stream_bitrate += os->first_segment_bit_rate; av_strlcpy(codec_str, os->codec_str, sizeof(codec_str)); if (max_audio_bitrate) { agroup = audio_group; @@ -1198,17 +1292,21 @@ static int write_manifest(AVFormatContext *s, int final) char codec_str[128]; AVStream *st = s->streams[i]; OutputStream *os = &c->streams[i]; - int stream_bitrate = os->muxer_overhead; - if (os->bit_rate > 0) - stream_bitrate += os->bit_rate; - else if (final) - stream_bitrate += os->pos * 8 * AV_TIME_BASE / c->total_duration; - else if (os->first_segment_bit_rate > 0) - stream_bitrate += os->first_segment_bit_rate; + int stream_bitrate; + if (st->codecpar->codec_type != AVMEDIA_TYPE_AUDIO) continue; + if (os->io_error && !os->init_range_length) + continue; if (os->segment_type != SEGMENT_TYPE_MP4) continue; + stream_bitrate = os->muxer_overhead; + if (os->bit_rate > 0) + stream_bitrate += os->bit_rate; + else if (final && c->total_duration > 0) + stream_bitrate += os->pos * 8 * AV_TIME_BASE / c->total_duration; + else if (os->first_segment_bit_rate > 0) + stream_bitrate += os->first_segment_bit_rate; av_strlcpy(codec_str, os->codec_str, sizeof(codec_str)); get_hls_playlist_name(playlist_file, sizeof(playlist_file), NULL, i); ff_hls_write_stream_info(st, c->m3u8_out, stream_bitrate, 0, @@ -1217,7 +1315,9 @@ static int write_manifest(AVFormatContext *s, int final) } } - dashenc_io_close(s, &c->m3u8_out, temp_filename); + ret = dashenc_io_close(s, &c->m3u8_out, temp_filename); + if (ret < 0) + return ret; if (use_rename) if ((ret = ff_rename(temp_filename, filename_hls, s)) < 0) return ret; @@ -1802,7 +1902,10 @@ static int dash_flush(AVFormatContext *s, int final, int stream) int range_length, index_length = 0; int64_t duration; - if (!os->packets_written) + if (!os->packets_written || os->io_error) + continue; + + if (!c->single_file && !os->temp_path[0]) continue; // Flush the single stream that got a keyframe right now. @@ -1827,11 +1930,23 @@ static int dash_flush(AVFormatContext *s, int final, int stream) if (ret < 0) break; os->packets_written = 0; + if (c->single_file && c->ignore_io_errors && !range_length) + continue; if (c->single_file) { + ret = (os->ctx && os->ctx->pb) ? os->ctx->pb->error : 0; + if (ret < 0) { + av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, + "Unable to write %s: %s\n", os->full_path, av_err2str(ret)); + if (!c->ignore_io_errors) + break; + ret = 0; + } find_index_range(s, os->full_path, os->pos, &index_length); } else { - dashenc_io_close(s, &os->out, os->temp_path); + ret = dashenc_io_close(s, &os->out, os->temp_path); + if (ret < 0) + break; if (use_rename) { ret = ff_rename(os->temp_path, os->full_path, os->ctx); @@ -1870,9 +1985,17 @@ static int dash_flush(AVFormatContext *s, int final, int stream) if (final) { for (i = 0; i < s->nb_streams; i++) { OutputStream *os = &c->streams[i]; - if (os->ctx && os->ctx_inited) { + if (os->ctx && os->ctx_inited && !os->io_error) { int64_t file_size = avio_tell(os->ctx->pb); - av_write_trailer(os->ctx); + int err = av_write_trailer(os->ctx); + if (c->single_file && os->ctx->pb && os->ctx->pb->error < 0) + err = os->ctx->pb->error; + if (err < 0) { + av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, + "Unable to write %s: %s\n", os->full_path, av_err2str(err)); + if (!c->ignore_io_errors && ret >= 0) + ret = err; + } if (c->global_sidx) { int j, start_index, start_number; int64_t sidx_size = avio_tell(os->ctx->pb) - file_size; @@ -1941,6 +2064,9 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) int64_t seg_end_duration, elapsed_duration; int ret; + if (os->io_error) + return 0; + ret = update_stream_extradata(s, os, pkt, &st->avg_frame_rate); if (ret < 0) return ret; @@ -2054,7 +2180,7 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) sizeof(os->producer_reference_time_str), os->producer_reference_time.wallclock); - if ((ret = dash_flush(s, 0, pkt->stream_index)) < 0) + if ((ret = dash_flush(s, 0, pkt->stream_index)) < 0 || os->io_error) return ret; } @@ -2099,16 +2225,21 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) c->max_gop_size = FFMAX(c->max_gop_size, os->gop_size); } - if ((ret = ff_write_chained(os->ctx, 0, pkt, s, 0)) < 0) + if ((ret = ff_write_chained(os->ctx, 0, pkt, s, 0)) < 0) { + if (c->single_file && c->ignore_io_errors && ret == os->ctx->pb->error) { + ignore_single_file_error(c, os); + return 0; + } return ret; + } os->packets_written++; os->total_pkt_size += pkt->size; os->total_pkt_duration += pkt->duration; os->last_flags = pkt->flags; - if (!os->init_range_length) - flush_init_segment(s, os); + if (!os->init_range_length && (ret = flush_init_segment(s, os)) < 0) + return ret; //open the output context when the first frame of a segment is ready if (!c->single_file && os->packets_written == 1) { @@ -2135,13 +2266,14 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) // in streaming mode, the segments are available for playing // before fully written but the manifest is needed so that // clients and discover the segment filenames. - if (c->streaming) { - write_manifest(s, 0); - } + if (c->streaming && (ret = write_manifest(s, 0)) < 0) + return ret; if (c->lhls) { char *prefetch_url = use_rename ? NULL : os->filename; - write_hls_media_playlist(os, s, pkt->stream_index, 0, prefetch_url); + ret = write_hls_media_playlist(os, s, pkt->stream_index, 0, prefetch_url); + if (ret < 0) + return ret; } } @@ -2154,6 +2286,14 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) if (os->out) { avio_write(os->out, buf + os->written_len, len - os->written_len); avio_flush(os->out); + if (os->out->error < 0) { + int err = os->out->error; + av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, + "Unable to write %s: %s\n", os->temp_path, av_err2str(err)); + if (!c->ignore_io_errors) + return err; + ff_format_io_close(s, &os->out); + } } os->written_len = len; } @@ -2164,7 +2304,7 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) static int dash_write_trailer(AVFormatContext *s) { DASHContext *c = s->priv_data; - int i; + int i, ret; if (s->nb_streams > 0) { OutputStream *os = &c->streams[0]; @@ -2178,7 +2318,7 @@ static int dash_write_trailer(AVFormatContext *s) s->streams[0]->time_base, AV_TIME_BASE_Q); } - dash_flush(s, 1, -1); + ret = dash_flush(s, 1, -1); if (c->remove_at_exit) { for (i = 0; i < s->nb_streams; ++i) { @@ -2200,7 +2340,7 @@ static int dash_write_trailer(AVFormatContext *s) } } - return 0; + return ret; } static int dash_check_bitstream(AVFormatContext *s, AVStream *st, -- 2.54.0 >From 8e511766e16bd5abdfb90d3e892b772f1e59a8c1 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Wed, 4 Nov 2026 23:11:33 +0100 Subject: [PATCH 3/3] avformat/dashenc: evict segments only after the manifest is written With window_size set, dash_flush() deleted segments that fell out of the window before the new manifests were published. When publishing the MPD or an HLS media playlist then failed, the last published manifest still listed segments that were already gone from disk. Move eviction to the end of a successful write_manifest(), including the path that skips refreshing the master playlist. For HLS media playlists, also limit it to the first segment of the last published playlist, so LHLS playlists, which are updated separately, never reference a deleted segment. Adjust the index when removing entries and keep segments that were not evicted for dash_free(). With ignore_io_errors, a failed close is reported as success, so track whether every manifest of this call was really published and skip the eviction otherwise. The write sequence and the returned values are unchanged. Tested by failing the third and every later rename of one manifest (MPD, media playlist or master playlist; normal, streaming and LHLS mode; one and two streams), and with ignore_io_errors by failing the third and every later close of one manifest. Before this change, 10 of 18 cases in each set left a manifest that referenced a deleted segment; after it, none do. Runs without the injected failure still keep at most window_size + extra_window_size segments (12 of 12). fate-dashenc and fate-source pass. Signed-off-by: lusoris <[email protected]> --- libavformat/dashenc.c | 91 +++++++++++++++++++++++++++++++------------ 1 file changed, 66 insertions(+), 25 deletions(-) diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c index b36d5792445..af1547ff467 100644 --- a/libavformat/dashenc.c +++ b/libavformat/dashenc.c @@ -112,6 +112,7 @@ typedef struct OutputStream { int64_t init_start_pos, pos; int init_range_length; int nb_segments, segments_size, segment_index; + int hls_start_index; /* First segment in the last published media playlist. */ int64_t seg_duration; int64_t frag_duration; int64_t last_duration; @@ -226,7 +227,7 @@ static int dashenc_io_open(AVFormatContext *s, AVIOContext **pb, char *filename, return err; } -static int dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) +static int dashenc_io_close_raw(AVFormatContext *s, AVIOContext **pb, char *filename) { DASHContext *c = s->priv_data; int http_base_proto = filename ? ff_is_http_proto(filename) : 0; @@ -252,6 +253,14 @@ static int dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename if (ret < 0) av_log(s, c->ignore_io_errors ? AV_LOG_WARNING : AV_LOG_ERROR, "Unable to write %s: %s\n", filename, av_err2str(ret)); + return ret; +} + +static int dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filename) +{ + DASHContext *c = s->priv_data; + int ret = dashenc_io_close_raw(s, pb, filename); + return c->ignore_io_errors ? 0 : ret; } @@ -425,14 +434,14 @@ static void get_start_index_number(OutputStream *os, DASHContext *c, static int write_hls_media_playlist(OutputStream *os, AVFormatContext *s, int representation_id, int final, - char *prefetch_url) { + char *prefetch_url, int *published) { DASHContext *c = s->priv_data; int timescale = os->ctx->streams[0]->time_base.den; char temp_filename_hls[1024]; char filename_hls[1024]; AVDictionary *http_opts = NULL; int target_duration = 0; - int ret = 0, err; + int ret = 0, err, write_failed = 0; const char *proto = avio_find_protocol_name(c->dirname); int use_rename = proto && !strcmp(proto, "file"); int i, start_index, start_number; @@ -440,9 +449,13 @@ static int write_hls_media_playlist(OutputStream *os, AVFormatContext *s, get_start_index_number(os, c, &start_index, &start_number); + if (published) + *published = 1; if (!c->hls_playlist || start_index >= os->nb_segments || os->segment_type != SEGMENT_TYPE_MP4) return 0; + if (published) + *published = 0; get_hls_playlist_name(filename_hls, sizeof(filename_hls), c->dirname, representation_id); @@ -483,8 +496,11 @@ static int write_hls_media_playlist(OutputStream *os, AVFormatContext *s, seg->range_length, seg->start_pos, NULL, c->single_file ? os->initfile : seg->file, &prog_date_time, 0); - if (ret < 0 && !c->ignore_io_errors) - goto fail; + if (ret < 0) { + write_failed = 1; + if (!c->ignore_io_errors) + goto fail; + } } if (prefetch_url) @@ -494,7 +510,9 @@ static int write_hls_media_playlist(OutputStream *os, AVFormatContext *s, ff_hls_write_end_list(c->m3u8_out); fail: - err = dashenc_io_close(s, &c->m3u8_out, temp_filename_hls); + err = dashenc_io_close_raw(s, &c->m3u8_out, temp_filename_hls); + if (err < 0) + write_failed = 1; if (ret >= 0) ret = err; if (ret < 0 && !c->ignore_io_errors) @@ -502,6 +520,11 @@ fail: if (use_rename) ret = ff_rename(temp_filename_hls, filename_hls, os->ctx); + if (ret >= 0 && !write_failed) { + os->hls_start_index = start_index; + if (published) + *published = 1; + } return c->ignore_io_errors ? 0 : ret; } @@ -1059,12 +1082,14 @@ static int has_hls_video(AVFormatContext *s) return 0; } +static inline void dashenc_delete_media_segments(AVFormatContext *s, OutputStream *os, int remove_count); + static int write_manifest(AVFormatContext *s, int final) { DASHContext *c = s->priv_data; AVIOContext *out; char temp_filename[1024]; - int ret, i; + int ret, i, published = 1; const char *proto = avio_find_protocol_name(s->url); int use_rename = proto && !strcmp(proto, "file"); static unsigned int warned_non_file = 0; @@ -1176,9 +1201,12 @@ static int write_manifest(AVFormatContext *s, int final) avio_printf(out, "</MPD>\n"); avio_flush(out); - ret = dashenc_io_close(s, &c->mpd_out, temp_filename); - if (ret < 0) - return ret; + ret = dashenc_io_close_raw(s, &c->mpd_out, temp_filename); + if (ret < 0) { + if (!c->ignore_io_errors) + return ret; + published = 0; + } if (use_rename) { if ((ret = ff_rename(temp_filename, s->url, s)) < 0) @@ -1189,9 +1217,13 @@ static int write_manifest(AVFormatContext *s, int final) // cannot interrupt MPD serialization. if (!c->lhls || final) { for (i = 0; i < s->nb_streams; i++) { - ret = write_hls_media_playlist(&c->streams[i], s, i, final, NULL); + int media_published; + + ret = write_hls_media_playlist(&c->streams[i], s, i, final, NULL, &media_published); if (ret < 0) return ret; + if (!media_published) + published = 0; } } @@ -1201,7 +1233,7 @@ static int write_manifest(AVFormatContext *s, int final) // Publish master playlist only the configured rate if (c->master_playlist_created && (!c->master_publish_rate || c->streams[0].segment_index % c->master_publish_rate)) - return 0; + goto manifests_written; if (*c->dirname) snprintf(filename_hls, sizeof(filename_hls), "%s%s", c->dirname, c->hls_master_name); @@ -1315,15 +1347,32 @@ static int write_manifest(AVFormatContext *s, int final) } } - ret = dashenc_io_close(s, &c->m3u8_out, temp_filename); - if (ret < 0) - return ret; + ret = dashenc_io_close_raw(s, &c->m3u8_out, temp_filename); + if (ret < 0) { + if (!c->ignore_io_errors) + return ret; + published = 0; + } if (use_rename) if ((ret = ff_rename(temp_filename, filename_hls, s)) < 0) return ret; c->master_playlist_created = 1; } +manifests_written: + if (c->window_size && published) { + for (i = 0; i < s->nb_streams; i++) { + OutputStream *os = &c->streams[i]; + int remove_count = os->nb_segments - c->window_size - c->extra_window_size; + + /* LHLS media playlists can lag behind the MPD. */ + if (c->hls_playlist && os->segment_type == SEGMENT_TYPE_MP4) + remove_count = FFMIN(remove_count, os->hls_start_index); + if (remove_count > 0) + dashenc_delete_media_segments(s, os, remove_count); + } + } + return 0; } @@ -1869,6 +1918,7 @@ static inline void dashenc_delete_media_segments(AVFormatContext *s, OutputStrea } os->nb_segments -= remove_count; + os->hls_start_index = FFMAX(os->hls_start_index - remove_count, 0); memmove(os->segments, os->segments + remove_count, os->nb_segments * sizeof(*os->segments)); } @@ -1973,15 +2023,6 @@ static int dash_flush(AVFormatContext *s, int final, int stream) os->pos += range_length; } - if (c->window_size) { - for (i = 0; i < s->nb_streams; i++) { - OutputStream *os = &c->streams[i]; - int remove_count = os->nb_segments - c->window_size - c->extra_window_size; - if (remove_count > 0) - dashenc_delete_media_segments(s, os, remove_count); - } - } - if (final) { for (i = 0; i < s->nb_streams; i++) { OutputStream *os = &c->streams[i]; @@ -2271,7 +2312,7 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt) if (c->lhls) { char *prefetch_url = use_rename ? NULL : os->filename; - ret = write_hls_media_playlist(os, s, pkt->stream_index, 0, prefetch_url); + ret = write_hls_media_playlist(os, s, pkt->stream_index, 0, prefetch_url, NULL); if (ret < 0) return ret; } -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
