Re: [FFmpeg-devel] [RFC] How to deal with libavfilter buffer overflows?
Le decadi 30 vendémiaire, an CCXXIII, Stefano Sabatini a écrit : $ ffmpeg -f lavfi -i testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat -y outtest.mp4 [...] Input #0, lavfi, from 'testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat': Duration: N/A, start: 0.00, bitrate: N/A Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1000k tbn, 1000k tbc [...] Output #0, mp4, to 'outtest.mp4': Metadata: encoder : Lavf56.9.101 Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc Metadata: encoder : Lavc56.8.102 libx264 Stream mapping: Stream #0:0 - #0:0 (rawvideo (native) - h264 (libx264)) Press [q] to stop, [?] for help [out @ 0x3140560] 100 buffers queued in out, something may be wrong. frame= 125 fps=0.0 q=-1.0 Lsize= 25kB time=00:00:04.92 bitrate= 42.4kbits/s video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.620967% I frequently stumble upon situations when buffer overflows happen, and they are usually a showstopper in many applications. For example when recombining streams with the concat filter, or when overlaying. The output is usually playable but misses audio or video frames. For example in the case above, the command is trying to switch order of the two input parts, but only the second part is displayed and the user is left with the vague message something may be wrong. So the current buffering system sometimes fails, and the user has no clue to understand where the problem lies. I want at least to improve diagnostics, but I want to listen about other developers experience first, and if you have ideas to tackle such issues. Hi. I am well aware of the problem too, and actually I have recently started writing a summary to try address it, along with the other problems of the current lavfi API. There are several underlying issues that are raised by your example: * You have AB, and you want to split and merge it into BA, but you only get B: it looks like it is just a bug. The issue disappears if I test with ffmpeg -lavfi filter instead of ffmpeg -f lavfi -i filter. I have yet to fully understand if the bug is in lavd/lavfi or in lavfi itself, but even in the former case, there is still the problem that the API is almost impossible to use correctly. * Apart from this bug, this command works, but that is a bit of luck. Someone raised the input buffer queue for concat to 256, and your graph requires 128. The warning you see is just a warning. But if you were to raise the duration of the first segment to something above 256 frames, it would indeed fail. The message to look for is Buffer queue overflow, dropping., not 100 buffers queued in output stream. * You have AB from a non-replayable input and you want BA. That can not work reliably. If your input were replayable, you could replay it as A1B1 and A2B2 and produce B2A1. But since it is not replayable, you need to buffer all A as decoded frames in memory. In this particular case, that is 5×25×320×240×3 = 27 Mo, still acceptable. But imagine the same thing with a full-HD credits sequence: 5×90×1920×1080×3/2 = 1.3 Go. When it has already eaten half a gigaoctet of memory, ffmpeg/lavfi can only guess whether this is expected and the memory consumption will soon stop or if the graph has a bug (arithmetic mistake in setpts, links connected to the wrong pad) that makes it eat memory indefinitely. Since the second case is likely to lead to memory and swap exhaustion, making the host machine unusable, ffmpeg should not attempt it without explicit user consent. You can usually give that user consent by inserting the fifo filter somewhere. In this particular case, inserting fifo at the s1 link does the trick. We could enhance the Buffer queue overflow message to print the exact link name where it happens, so the user can know where the fifo is needed. But that is not a very good solution because fifos are dangerous too. The solution I consider to propose involves moving the FIFO service directly into the framework. That would allow a few goodies, such as global allocation of the queue and statistics. Regards, -- Nicolas George signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [RFC] How to deal with libavfilter buffer overflows?
On Wed, Oct 22, 2014 at 01:39:07PM +0200, Nicolas George wrote: Le decadi 30 vendémiaire, an CCXXIII, Stefano Sabatini a écrit : $ ffmpeg -f lavfi -i testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat -y outtest.mp4 [...] Input #0, lavfi, from 'testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat': Duration: N/A, start: 0.00, bitrate: N/A Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1000k tbn, 1000k tbc [...] Output #0, mp4, to 'outtest.mp4': Metadata: encoder : Lavf56.9.101 Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc Metadata: encoder : Lavc56.8.102 libx264 Stream mapping: Stream #0:0 - #0:0 (rawvideo (native) - h264 (libx264)) Press [q] to stop, [?] for help [out @ 0x3140560] 100 buffers queued in out, something may be wrong. frame= 125 fps=0.0 q=-1.0 Lsize= 25kB time=00:00:04.92 bitrate= 42.4kbits/s video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.620967% I frequently stumble upon situations when buffer overflows happen, and they are usually a showstopper in many applications. For example when recombining streams with the concat filter, or when overlaying. The output is usually playable but misses audio or video frames. For example in the case above, the command is trying to switch order of the two input parts, but only the second part is displayed and the user is left with the vague message something may be wrong. So the current buffering system sometimes fails, and the user has no clue to understand where the problem lies. I want at least to improve diagnostics, but I want to listen about other developers experience first, and if you have ideas to tackle such issues. Hi. I am well aware of the problem too, and actually I have recently started writing a summary to try address it, along with the other problems of the current lavfi API. There are several underlying issues that are raised by your example: * You have AB, and you want to split and merge it into BA, but you only get B: it looks like it is just a bug. The issue disappears if I test with ffmpeg -lavfi filter instead of ffmpeg -f lavfi -i filter. I have yet to fully understand if the bug is in lavd/lavfi or in lavfi itself, but even in the former case, there is still the problem that the API is almost impossible to use correctly. * Apart from this bug, this command works, but that is a bit of luck. Someone raised the input buffer queue for concat to 256, and your graph requires 128. The warning you see is just a warning. But if you were to raise the duration of the first segment to something above 256 frames, it would indeed fail. The message to look for is Buffer queue overflow, dropping., not 100 buffers queued in output stream. * You have AB from a non-replayable input and you want BA. That can not work reliably. If your input were replayable, you could replay it as A1B1 and A2B2 and produce B2A1. But since it is not replayable, you need to buffer all A as decoded frames in memory. In this particular case, that is 5×25×320×240×3 = 27 Mo, still acceptable. But imagine the same thing with a full-HD credits sequence: 5×90×1920×1080×3/2 = 1.3 Go. When it has already eaten half a gigaoctet of memory, ffmpeg/lavfi can only guess whether this is expected and the memory consumption will soon stop or if the graph has a bug (arithmetic mistake in setpts, links connected to the wrong pad) that makes it eat memory indefinitely. Since the second case is likely to lead to memory and swap exhaustion, making the host machine unusable, ffmpeg should not attempt it without explicit user consent. the fifo/buffer que should grow the ability to buffer frames to disk and to do so using lossless and lossy compression that way much larger delays can be achived with such buffers without exhaustig memory true seek support should be added to the source filters so that one can just specify which sections to play in which order without the need to use large buffers [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The misfortune of the wise is better than the prosperity of the fool. -- Epicurus signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [RFC] How to deal with libavfilter buffer overflows?
$ ffmpeg -f lavfi -i testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat -y outtest.mp4 [...] Input #0, lavfi, from 'testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat': Duration: N/A, start: 0.00, bitrate: N/A Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1000k tbn, 1000k tbc [...] Output #0, mp4, to 'outtest.mp4': Metadata: encoder : Lavf56.9.101 Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc Metadata: encoder : Lavc56.8.102 libx264 Stream mapping: Stream #0:0 - #0:0 (rawvideo (native) - h264 (libx264)) Press [q] to stop, [?] for help [out @ 0x3140560] 100 buffers queued in out, something may be wrong. frame= 125 fps=0.0 q=-1.0 Lsize= 25kB time=00:00:04.92 bitrate= 42.4kbits/s video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.620967% I frequently stumble upon situations when buffer overflows happen, and they are usually a showstopper in many applications. For example when recombining streams with the concat filter, or when overlaying. The output is usually playable but misses audio or video frames. For example in the case above, the command is trying to switch order of the two input parts, but only the second part is displayed and the user is left with the vague message something may be wrong. So the current buffering system sometimes fails, and the user has no clue to understand where the problem lies. I want at least to improve diagnostics, but I want to listen about other developers experience first, and if you have ideas to tackle such issues. I can provide more use cases if required. -- FFmpeg = Foolish and Fanciful Multimedia Powerful Elected Gladiator ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
