#11034: Stream copy with -use_editlist 0 introduces delay by default, causes A/V
desync without warning
-------------------------------------+-------------------------------------
Reporter: pchu | Type: defect
Status: new | Priority: normal
Component: | Version: git-
undetermined | master
Keywords: | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Summary of the bug:
At first, I was trying to fix some mp4 files' problematic editlists
(preferably get rid of them), only to find that if I extract and remux the
audio/video streams, the resulting mp4 files still and always have an
editlist.
Later, I tested on mp4 files that don't have editlists in the first place.
It seems that FFmpeg copy-codec introduces editlists by default, unless
force {{{-use_editlist 0}}}.
I notice that, with the default {{{-use_editlist auto}}} option,
{{{ffprobe -v trace out.mp4}}} shows a non-zero media_time in the video
stream editlist, meaning that A/V would desync (if FFmpeg really copies
the streams without any modification, which is not true as I'll show
below).
Instead, with {{{-use_editlist 0}}} option, {{{ffprobe -show_streams
out.mp4}}} shows a non-zero {{{start_pts}}} and {{{start_time}}} in the
video stream, again meaning that A/V would desync (which is real).
How to reproduce:
{{{
ffmpeg -i "elst_test.mp4" -codec copy -copyts -copytb 1 -fps_mode
passthrough -avoid_negative_ts disabled "out_with_elst.mp4"
ffmpeg -i "elst_test.mp4" -codec copy -copyts -copytb 1 -fps_mode
passthrough -avoid_negative_ts disabled -use_editlist 0
"out_without_elst.mp4"
}}}
All those options {{{-copyts -copytb 1 -fps_mode passthrough
-avoid_negative_ts disabled}}} are me trying to tell FFmpeg not to process
input timestamps, although they have no effect. The output files are the
same with or without these options. The resulting files always have
timestamps modified.
I inspect the first frames/packets with {{{ffprobe -select_streams v
-read_intervals 0%0.5 -show_packets elst_test.mp4}}} and {{{out.mp4}}},
with and without {{{-ignore_editlist 1}}} option. To summarize, their
timestamps are:
{{{
% elst_test.mp4 (no editlist):
% time_base=1/600, start_pts=0
frame I P B B B ...
DTS -40 -20 0 20 40 ...
CTS 0 80 40 20 60 ...
PTS 0 80 40 20 60 ...
% out_with_elst.mp4 (with an editlist saying media_time=1280):
% time_base=1/19200, start_pts=0
frame I P B B B ...
DTS 0 640 1280 1920 2560 ... (not counting editlist shift)
CTS 1280 3840 2560 1920 3200 ... (not counting editlist shift)
PTS 0 2560 1280 640 1920 ... (counting editlist shift, A/V synched)
% out_without_elst.mp4 (no editlist):
% time_base=1/19200, start_pts=1280
frame I P B B B ...
DTS 0 640 1280 1920 2560 ...
CTS 1280 3840 2560 1920 3200 ...
PTS 1280 3840 2560 1920 3200 ... (out-of-sync by 1280/19200 seconds)
}}}
Finally, I googled a lot and was able to narrow down the reason why FFmpeg
insisted on shifting all timestamps forward and making start_pts non-zero
: version 0 of the CTTS box doesn't accept negative CTS-DTS offsets. With
this constraint, {{{out_without_elst.mp4}}} has to start at PTS = 1280.
I realized at this point that {{{ffprobe -show_packets}}} did not report
'correct' or 'raw' DTS:
{{{
% elst_test.mp4 (no editlist):
% time_base=1/600, start_pts=0
frame I P B B B ...
DTS 0 20 40 60 80 ... ('raw' DTS from STTS box)
CTS 0 80 40 20 60 ... (calculated from DTS and CTTS box)
DTS -40 -20 0 20 40 ... (then shift DTS by 40) <- reported by
ffprobe
}}}
My workaround to produce no-delay v1 CTTS video:
{{{
ffmpeg -i "elst_test.mp4" -codec copy -movflags +negative_cts_offsets
"out_with_elst_CTTSv1.mp4"
ffmpeg -i "elst_test.mp4" -codec copy -movflags +negative_cts_offsets
-use_editlist 0 "out_without_elst_CTTSv1.mp4"
}}}
But there're still problems:
1. No warning is shown when FFmpeg chooses to follow v0 CTTS box
specification and introduces unwanted delays.
2. The only method {{{-movflags +negative_cts_offsets}}} that allows
FFmpeg to use v1 spec (to keep input timestamps untouched) is not
documented/mentioned neither under {{{-copyts}}} option section nor under
{{{-vsync/-fps_mode}}} or {{{-avoid_negative_ts}}} option sections, so
that I had to dig deeper to find it.
3. I also wonder why {{{-avoid_negative_ts disabled}}} does not imply the
use of v1 spec (**which is also the input file timestamps spec! **). Why
{{{-copyts}}} does not imply using the same version of CTTS box in output
as in input file?
4. When no other options except for {{{-use_editlist 0}}} is given,
shouldn't the default behaviour be using v1 CTTS (same as input) with a
"using-v1" warning shown in cmdline, instead of using v0 CTTS with an
"A/V-desync" warning?
5. ISO/IEC 14496-12:2015(E) explicitely states that "The DT axis has a
zero origin". Then why {{{ffprobe -show_packets}}} deliberately shifts DTS
to a non-zero origin in order to keep CTS-DTS offsets non-negative,
pretending to be a v0 CTTS box (which is not)?
6. When {{{negative_cts_offsets}}} flag is set,
{{{out_with_elst_CTTSv1.mp4}}} no longer needs an editlist to shift its
start_pts, but the implicit option {{{-use_editlist auto}}} still creates
an editlist for no reason, with a media_time = 0. What's the point?
7. {{{-copytb 1}}} does not stop FFmpeg from modifying the
{{{time_base=1/600 -> 1/19200}}}. How can I stop this and why is it doing
so?
--
Ticket URL: <https://trac.ffmpeg.org/ticket/11034>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker_______________________________________________
FFmpeg-trac mailing list
[email protected]
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
[email protected] with subject "unsubscribe".
