Francois Visagie <[email protected]> added the comment:
On 2009-05-14.23:12:04 cehoyos wrote:
>
> I'm sorry, but again, I did not read the complete
issue!
>
> A few comment nonetheless:
> Only latest svn is accepted for bug-reports.
Thanks. I'll come back to that at the end.
The most important point that I'll come back to below
also, is that these results DO show beyond any doubt
problem behaviour of the -maxrate option.
> The first sentence of your report does not match
ffmpeg's
> output ("DV AVI" vs "avs raw")
I understand the apparent contradiction. The source is
BFF DV AVI, but because the FFmpeg DV decoder fails
with "AC EOB marker is absent" messages, the DV AVI is
being input via the AviSynth script "Underflow clip
1.avi.37.avs":
DirectShowSource("Underflow clip 1.avi")
and with the FFmpeg command line option
-i "Underflow clip 1.avi.37.avs"
> Apart from the fact that my
> ffmpeg version does not accept "-pass %PASS%" as
command line
> option:
%PASS% is the syntax for a replacable command line
parameter (or whatever it's called) in Windows. I.e.
the -pass <int> option is being specified.
> Are you sure that aspect, top and pass are needed to
> reproduce the issue? Imo, the shortest possible (and
working)
> command line makes debugging much easier.
Yes, not all options are needed to reproduce buffer
underflow. Errors occur on pass 2 only with this input
sample. Here are the minimum two-pass command lines
fully written out (no replacable parameters):
ffmpeg -i "Underflow clip 1.avi.37.avs" -target pal-
dvd -muxrate 9928000 -acodec ac3 -ab 128000 -flags
ildct+ilme -g 15 -maxrate 9800000 -vcodec mpeg2video -
b 9000000 -pass 1 "Underflow clip 1.avi.37.mpg"
ffmpeg -i "Underflow clip 1.avi.37.avs" -target pal-
dvd -muxrate 9928000 -acodec ac3 -ab 128000 -flags
ildct+ilme -g 15 -maxrate 9800000 -vcodec mpeg2video -
b 9000000 -pass 2 "Underflow clip 1.avi.37.mpg"
--- AND ---
On 2009-05-15.00:43:47 michaelni wrote:
>
> On Thu, May 14, 2009 at 08:02:38PM +0000, Francois
Visagie wrote:
> > SITUATION AND PROBLEM DESCRIPTION
> > ---------------------------------
> >
> > Under certain conditions, lavc mpeg2video's -
maxrate option
> produces
> > buffer underflow when transcoding from DV AVI to
> DVD-compliant MPEG-2.
> > I am using a Windows XP SP3 system with FFmpeg 0.5.
> >
> > COMMAND LINE 1
> > --------------
> >
> > I can 100% reliably recreate the buffer underflow
condition
> with the
> > command line:
> >
> > ffmpeg -top 0 -async 0 -nr 100 -i %AVSCRIPT% -
aspect
> > 4:3 -target pal-dvd -muxrate 9928000 -acodec ac3 -
ab 128000 -flags
> > ildct+ilme -g 15 -maxrate 9800000 - vcodec
mpeg2video -b
> 9000000 -pass
> > %PASS% %OUTPUT%
> >
> > Although that bitrate is high, it's still legal and
>
> no, it is not, you have a audio bitrate of 128k and
a video
> bitrate of 9800kbit that is too much for a muxrate
of
> 9928kbit, mpeg-ps is not just raw audio + raw
video, there
> are also mpeg-ps specific things that need bits.
>
> > was chosen to force the lavc mpeg2video buffer
underflow
> issue with my
> > test clip. Much lower bitrates (e.g. 5800) also
produce buffer
> > underflows on production length videos, but then
it's much more
> > difficult to find and create a short test clip
that reproduces the
> > problem. (And encoding the production video takes
~55h each time!)
> >
> > -muxrate was chosen to indicate exceeded mux rate
as soon as video
> > exceeds the allowed bitrate.
> >
> > With the chosen settings this command line does
not exceed
> -maxrate by
> > impressively much for the sample clip below.
However, on production
> > length videos, or by adding -mbd 2 -trellis and
other high-quality
> > settings to this command line, video bitrate can
overshoot -maxrate
> > spectacularly, to 11000kbps and beyond in most
cases.
> >
>
> > This command line does not include -bufsize as the
documentation
> > suggests. This was done on purpose to demonstrate
that -maxrate by
> > itself causes buffer underflow. This test was
repeated with
> -bufsize,
> > and also without -maxrate, and results are below.
>
> -target pal-dvd
> sets a wide varity of parameters, amongth them
bufsize so
> your ommision of the bufsize parameter does nothing
>
> >
> > This command line and sample clip should therefore
be treated as a
> > valid instance of faulty buffer underflow. More
example
> command lines
> > and their results follow below. Pay particular
attention to
> peak and
> > average video bitrate values and warnings either
reported or not.
> >
> > SAMPLE DATA
> > -----------
> >
> > A sample video file which reproduces the buffer
underflow condition
> > with the above command line has been uploaded to
> > <ftp://upload.ffmpeg.org/MPlayer/incoming/lavc
> > mepg2video buffer underflow>. The file is
called "Underflow clip
> > 1.rar", it was split into roughly 10MB
chunks "Underflow clip
> > 1.rar.chunk1...20"
> > and uploaded. On Windows systems the video file
can be
> reconstituted
> > with the uploaded batch file Unde_Merge.bat, or
the
> constituent files
> > "Underflow clip 1.rar.chunk1...20" can be
similarly
> concatenated with
> > the cat command on ...x systems.
> >
> > COMMAND LINE 1 RESULTS
> > ----------------------
> > No "impossible bitrate constraints, this will
fail"
> > warning
> > Buffer underflow
>
> > Video Peak 9845 Average 8398
>
> What tool was used to find these 2 values?
"MPEG Bitrate Viewer" (bitrate.exe) 1.5.054
Although not quite as detailed, the histogram from
Avidemux 2.4.4 and "Show Information" of CyberLink
PowerDVD 6 confirmed these values.
>
> >
> > COMMAND LINE 1 CONSOLE OUTPUT
> > -----------------------------
> > FFmpeg version 0.5, Copyright (c) 2000-2009
Fabrice Bellard, et al.
> > configuration: --enable-gpl --enable-postproc --
enable-swscale
> > --enable-avfilt er --enable-avfilter-lavf --enable-
pthreads
> --enable-
> > avisynth --enable-libfaac - -enable-libfaad --
enable-libmp3lame
> > --enable-libspeex - -enable-libtheora --enabl e-
libvorbis
> > --enable-libxvid --enable-libx264 --enable-
memalign-hack
> > libavutil 49.15. 0 / 49.15. 0
> > libavcodec 52.20. 0 / 52.20. 0
> > libavformat 52.31. 0 / 52.31. 0
> > libavdevice 52. 1. 0 / 52. 1. 0
> > libavfilter 0. 4. 0 / 0. 4. 0
> > libswscale 0. 7. 1 / 0. 7. 1
> > libpostproc 51. 2. 0 / 51. 2. 0
> > built on Mar 16 2009 16:09:18, gcc: 4.2.4
[Sherpya] Input
> #0, avs,
> > from 'INPUT.AVS':
> > Duration: 00:01:12.76, start: 0.000000, bitrate:
0 kb/s
> > Stream #0.0: Video: rawvideo, yuyv422, 720x576,
> > 165888 kb/s, 25 tbr, 25 tbn,
> > 25 tbc
> > Stream #0.1: Audio: pcm_s16le, 48000 Hz,
stereo, s16, 1536 kb/s
> > Output #0, dvd, to 'C:\Documents and
Settings\fvisagie\My
> Documents\My
> > Videos\Ho me Videos\Testing\Underflow clip 1
\Underflow clip
> > 1.avi.37.mpg':
> > Stream #0.0: Video: mpeg2video, yuv420p,
720x576 [PAR 16:15 DAR
> > 4:3], q=2-31 , pass 1, 9000 kb/s, 90k tbn, 25 tbc
> > Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16,
> > 128 kb/s
> > Stream mapping:
> > Stream #0.0 -> #0.0
> > Stream #0.1 -> #0.1
> > Press [q] to stop encoding
> > frame= 1819 fps= 30 q=1.6 Lsize= 72900kB
time=72.72
> > bitrate=8212.3kbits/s
> > video:70456kB audio:1136kB global headers:0kB
muxing overhead
> > 1.826325% FFmpeg version 0.5, Copyright (c) 2000-
2009
> Fabrice Bellard,
> > et al.
> > configuration: --enable-gpl --enable-postproc --
enable-swscale
> > --enable-avfilt er --enable-avfilter-lavf --enable-
pthreads
> --enable-
> > avisynth --enable-libfaac - -enable-libfaad --
enable-libmp3lame
> > --enable-libspeex - -enable-libtheora --enabl e-
libvorbis
> > --enable-libxvid --enable-libx264 --enable-
memalign-hack
> > libavutil 49.15. 0 / 49.15. 0
> > libavcodec 52.20. 0 / 52.20. 0
> > libavformat 52.31. 0 / 52.31. 0
> > libavdevice 52. 1. 0 / 52. 1. 0
> > libavfilter 0. 4. 0 / 0. 4. 0
> > libswscale 0. 7. 1 / 0. 7. 1
> > libpostproc 51. 2. 0 / 51. 2. 0
> > built on Mar 16 2009 16:09:18, gcc: 4.2.4
[Sherpya] Input
> #0, avs,
> > from 'INPUT.AVS':
> > Duration: 00:01:12.76, start: 0.000000, bitrate:
0 kb/s
> > Stream #0.0: Video: rawvideo, yuyv422, 720x576,
> > 165888 kb/s, 25 tbr, 25 tbn,
> > 25 tbc
> > Stream #0.1: Audio: pcm_s16le, 48000 Hz,
stereo, s16, 1536 kb/s
> > Output #0, dvd, to 'C:\Documents and
Settings\fvisagie\My
> Documents\My
> > Videos\Ho me Videos\Testing\Underflow clip 1
\Underflow clip
> > 1.avi.37.mpg':
> > Stream #0.0: Video: mpeg2video, yuv420p,
720x576 [PAR 16:15 DAR
> > 4:3], q=2-31 , pass 2, 9000 kb/s, 90k tbn, 25 tbc
> > Stream #0.1: Audio: ac3, 48000 Hz, stereo, s16,
> > 128 kb/s
> > Stream mapping:
> > Stream #0.0 -> #0.0
> > Stream #0.1 -> #0.1
> > [mpeg2video @ 0x1733140][lavc rc] Using all of
requested bitrate is
> > not necessar y for this video with these
parameters.
> > Press [q] to stop encoding
> > [dvd @ 0x1748010]buffer underflow i=0 bufi=58292
> > size=61037=9896.7kbits/s [dvd @ 0x1748010]buffer
underflow i=0
> > bufi=60316
> > size=61037
>
> [...]
> > COMMAND LINE 2
> > --------------
> >
> > Command line 1 with -bufsize 1835000 (DVD spec)
added:
> >
> > ffmpeg -top 0 -async 0 -nr 100 -i %AVSCRIPT% -
aspect
> > 4:3 -target pal-dvd -muxrate 9928000 -acodec ac3 -
ab 128000 -flags
> > ildct+ilme -g 15 -maxrate 9800000 - bufsize
1835000 -vcodec
> mpeg2video
> > -b 9000000 -pass % PASS% %OUTPUT%
> >
> > COMMAND LINE 2 RESULTS
> > ----------------------
> > No "impossible bitrate constraints, this will
fail"
> > warning
> > Buffer underflow
> > Video Peak 9845 Average 8398
> >
>
> > COMMAND LINE 3
> > --------------
> >
> > Command line 2 with -maxrate 9800000 removed:
>
> as long as you keep -target pal-dvd there is a
maxrate
>
> >
> > ffmpeg -top 0 -async 0 -nr 100 -i %AVSCRIPT% -
aspect
> > 4:3 -target pal-dvd -muxrate 9928000 -acodec ac3 -
ab 128000 -flags
> > ildct+ilme -g 15 -bufsize 1835000 - vcodec
mpeg2video -b
> 9000000 -pass
> > %PASS% %OUTPUT%
> >
> > COMMAND LINE 3 RESULTS
> > ----------------------
> >
> > "impossible bitrate constraints, this will fail"
> > warning displayed
> > No buffer underflow
> > Video Peak 9113 Average 7739
> > Encode is ~50% slower than above two command lines!
What the results of command 3 show:
* WITHOUT -maxrate (command line 3), no problem is
experienced and both the video peak and average values
are well within the -muxrate constraint specified. The
video bitrate peak value shows the specified -muxrate
constraint works for this particular clip: there is
enough room for video bitrate, audio bitrate, muxing
overhead (which is typically < 2% for this clip) etc.
within the specified -muxrate. It also shows that the
specified -maxrate 9800000 works for this clip: it is
substantially more than the actual video bitrate peak.
* However, WITH -maxrate 9800000 (command line 1),
buffer underflows are experienced on the same video
clip and with the same other options that gave no
problems without -maxrate. Both video peak and video
average bitrates are worsened, just by ADDING a valid
and reasonable -maxrate to an otherwise working encode.
* So whether a particular encode with -maxrate leads
to eventual buffer underflows or not, these results
show that the -maxrate option has the ability to break
an otherwise perfectly working encode. Therefore some
detail of the -maxrate implementation must be wrong.
> >
> > WHAT WE KNOW SO FAR
> > -------------------
> >
> > a) Buffer underflows seem to occur more severely
with "high
> quality"
> > encodes that involve e.g. -mbd 2 - trellis. Both
average and peak
> > bitrate are also higher.
>
> > b) Hervé found that reducing -buf_size avoids
excessive
> bitrate, but
> > at the expense of DVD-Video compliance of course.
It's
> unclear whether
> > this was in the presence of the -maxrate option.
>
> what i know is that you guys dont know too much.
That is quite correct in my case :-).
> a increase in bufsize breaks compliance, a decrease
does not
> normally also iam not convinced that there is any
bug in
> libav* at all at least you have not provided a non
> contradictionary case that produced any issues
>
> [...]
> > CLOSE
> > -----
> >
> > I am under lots of pressure to finish my current
DVD magnum
> opus soon,
> > and if there's any prospect of investigating and
hopefully
> fixing this
> > bug in the near future, please let me know where I
can help with
> > testing and/or more information.
>
> given that you managed to feed ffmpeg with
contradictionary
> parameters the buffer underflows you report are just
a case
> of garbage in, garbage out.
>
> If you can provide a minimal command line with non
> contradictionary paremeters (you need a gap between
muxrate
> and the sum of all video and audio max rates), how
large that
> gap has to be, i dont know but not 0, and also not
0.1%,
> mpeg-ps is a mildly bloated format it does need some
space
> for its bloat!
Yes.
The very compelling evidence is on my hard drive. It
involves an encode with more real-world bitrates:
ffmpeg -i "Movie.avi.37.avs" -target pal-dvd -muxrate
10080000 -acodec ac3 -ab 128000 -flags ildct+ilme -g
15 -vcodec mpeg2video -b 6300000 -pass
1 "Movie.avi.37.mpg"
ffmpeg -i "Movie.avi.37.avs" -target pal-dvd -muxrate
10080000 -acodec ac3 -ab 128000 -flags ildct+ilme -g
15 -vcodec mpeg2video -b 6300000 -pass
2 "Movie.avi.37.mpg"
That encode works.
When I add -maxrate 9800000, the average and peak
video bitrates shoot up and (on pass 2) buffer
underflows result.
Much more believable evidence? ;-)
The problem is that movie file is 36GB.
The test clip contains the section from that movie
file where buffer underflows occur. Unfortunately,
when I encode the test clip with the same bitrate
specifications no buffer underflows result.
So it seems we can proceed in one of two ways:
1) If you agree that the above command lines 1 - 3
show conclusively that -maxrate at least wrongly
increases bitrate, we can proceed on that basis and I
can perhaps assist you with further testing you
recommend, and/or
2) You and I need to find a way to get that 36GB file
to you. Then everybody would be perfectly happy, but I
have neither fast nor much bandwidth! If you have any
suggestions, please let me know.
LATEST SVN
----------
Because I am finding FFmpeg useful, as a token of
gratitude I am trying to contribute to the greater
effort what I can. What I can contribute is helping to
idenfity and correct code and documentation errors. My
programming has become too rusty for anything else.
Now, I understand the advantages of reporting against
the latest SVN. I am sure you also understand the
disadvantage that it will exclude people like myself
who have neither the skills nor the tools to make
their own build of the latest SVN. If we could find a
way for such people to always have access to the
latest SVN build, everybody would be perfectly happy.
For that I have a suggestion: please script an
automatic daily build of the current SVN. In a
previous role as development manager of a huge
software system I did that (in DOS batch files) and it
worked beautifully (it could even restart
automatically from where it had failed once the error
was corrected).
If that can be done, the following advantages will
result:
* everybody will always have access to builds with the
very latest improvements
* developers will be in a position to always receive
feedback on the very latest version from everybody
using it
* a regular build is the best configuration control
method of ensuring proper software development
synchronisation and consistency
I'll suggest this as a separate software improvement
for greater visibility and traceability.
Thank you for attending to this, and kind regards,
Francois
----------
nosy: +fvisagie
status: closed -> open
substatus: invalid -> needs_more_info
_____________________________________________________
FFmpeg issue tracker <[email protected]>
<https://roundup.ffmpeg.org/roundup/ffmpeg/issue1081>
_____________________________________________________
