[issue2584] VLC stack overflow in libavcodec with vc1 file
Reinhard Tartler siret...@tauware.de added the comment: patch in https://roundup.ffmpeg.org/msg13620 committed as http://git.ffmpeg.org/? p=ffmpeg.git;a=commit;h=2bbec1eda46d907605772a8b6e8263caa4bc4c82 Patch results in the following valgrind issues fixed: --- vc1-overread-old.log2011-02-19 13:03:29.0 +0100 +++ vc1-overread.log2011-02-19 12:57:50.0 +0100 @@ -2,13 +2,12 @@ Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al. Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info Command: ./ffplay_g guess_mv_stack_overflow.vc1 -Parent PID: 17119 +Parent PID: 10311 Thread 4: Invalid read of size 4 - at 0x83BF935: vc1_decode_ac_coeff (bswap.h:42) - by 0x83C36E3: vc1_decode_i_blocks_adv (vc1dec.c:1693) - by 0x83CE2AB: vc1_decode_frame (vc1dec.c:2989) + at 0x83C3A21: vc1_decode_i_blocks_adv (bswap.h:42) + by 0x83CE2FB: vc1_decode_frame (vc1dec.c:2989) by 0x83B63AD: avcodec_decode_video2 (utils.c:667) by 0x807BDAC: input_request_frame (ffplay.c:1539) by 0x808945E: avfilter_request_frame (avfilter.c:369) @@ -18,11 +17,11 @@ by 0x41DA25C: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) by 0x421696D: start_thread (pthread_create.c:300) by 0x42F7A4D: clone (clone.S:130) - Address 0x48bcc87 is 1,687 bytes inside a block of size 1,690 alloc'd + Address 0x628cfec is 3,436 bytes inside a block of size 3,439 alloc'd at 0x4024106: memalign (vg_replace_malloc.c:581) by 0x4024163: posix_memalign (vg_replace_malloc.c:709) - by 0x859C157: av_mallocz (mem.c:83) - by 0x83CDB0A: vc1_decode_frame (vc1dec.c:3187) + by 0x859C1D7: av_mallocz (mem.c:83) + by 0x83CDB5A: vc1_decode_frame (vc1dec.c:3187) by 0x83B63AD: avcodec_decode_video2 (utils.c:667) by 0x807BDAC: input_request_frame (ffplay.c:1539) by 0x808945E: avfilter_request_frame (avfilter.c:369) @@ -33,8 +32,9 @@ by 0x421696D: start_thread (pthread_create.c:300) Invalid read of size 4 - at 0x83C39D1: vc1_decode_i_blocks_adv (bswap.h:42) - by 0x83CE2AB: vc1_decode_frame (vc1dec.c:2989) + at 0x83BF933: vc1_decode_ac_coeff (bswap.h:42) + by 0x83C3733: vc1_decode_i_blocks_adv (vc1dec.c:1693) + by 0x83CE2FB: vc1_decode_frame (vc1dec.c:2989) by 0x83B63AD: avcodec_decode_video2 (utils.c:667) by 0x807BDAC: input_request_frame (ffplay.c:1539) by 0x808945E: avfilter_request_frame (avfilter.c:369) @@ -44,90 +44,11 @@ by 0x41DA25C: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) by 0x421696D: start_thread (pthread_create.c:300) by 0x42F7A4D: clone (clone.S:130) - Address 0x48bcc88 is 1,688 bytes inside a block of size 1,690 alloc'd + Address 0x628cfec is 3,436 bytes inside a block of size 3,439 alloc'd at 0x4024106: memalign (vg_replace_malloc.c:581) by 0x4024163: posix_memalign (vg_replace_malloc.c:709) - by 0x859C157: av_mallocz (mem.c:83) - by 0x83CDB0A: vc1_decode_frame (vc1dec.c:3187) - by 0x83B63AD: avcodec_decode_video2 (utils.c:667) - by 0x807BDAC: input_request_frame (ffplay.c:1539) - by 0x808945E: avfilter_request_frame (avfilter.c:369) - by 0x808063F: get_filtered_video_frame (cmdutils.c:853) - by 0x807C59E: video_thread (ffplay.c:1828) - by 0x418F9CD: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x41DA25C: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x421696D: start_thread (pthread_create.c:300) - -Invalid read of size 1 - at 0x83BF9B3: vc1_decode_ac_coeff (get_bits.h:319) - by 0x83C36E3: vc1_decode_i_blocks_adv (vc1dec.c:1693) - by 0x83CE2AB: vc1_decode_frame (vc1dec.c:2989) - by 0x83B63AD: avcodec_decode_video2 (utils.c:667) - by 0x807BDAC: input_request_frame (ffplay.c:1539) - by 0x808945E: avfilter_request_frame (avfilter.c:369) - by 0x808063F: get_filtered_video_frame (cmdutils.c:853) - by 0x807C59E: video_thread (ffplay.c:1828) - by 0x418F9CD: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x41DA25C: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x421696D: start_thread (pthread_create.c:300) - by 0x42F7A4D: clone (clone.S:130) - Address 0x48bcc8a is 0 bytes after a block of size 1,690 alloc'd - at 0x4024106: memalign (vg_replace_malloc.c:581) - by 0x4024163: posix_memalign (vg_replace_malloc.c:709) - by 0x859C157: av_mallocz (mem.c:83) - by 0x83CDB0A: vc1_decode_frame (vc1dec.c:3187) - by 0x83B63AD: avcodec_decode_video2 (utils.c:667) - by 0x807BDAC: input_request_frame (ffplay.c:1539) - by 0x808945E: avfilter_request_frame (avfilter.c:369) - by 0x808063F: get_filtered_video_frame (cmdutils.c:853) - by 0x807C59E: video_thread (ffplay.c:1828) - by 0x418F9CD: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x41DA25C: ??? (in /usr/lib/libSDL-1.2.so.0.11.3) - by 0x421696D: start_thread (pthread_create.c:300) - -Invalid read of size 4 - at 0x83C330A: vc1_decode_i_blocks_adv (bswap.h:42) - by 0x83CE2AB: vc1_decode_frame (vc1dec.c:2989) - by 0x83B63AD: avcodec_decode_video2 (utils.c:667) - by 0x807BDAC: input_request_frame (ffplay.c:1539) - by
[issue2584] VLC stack overflow in libavcodec with vc1 file
Reimar Döffinger b...@reimardoeffinger.de added the comment: On Sat, Feb 19, 2011 at 12:27:12PM +, Reinhard Tartler wrote: Reinhard Tartler siret...@tauware.de added the comment: patch in https://roundup.ffmpeg.org/msg13620 committed as http://git.ffmpeg.org/? p=ffmpeg.git;a=commit;h=2bbec1eda46d907605772a8b6e8263caa4bc4c82 Patch results in the following valgrind issues fixed: Increase the padding requirement in the next major bump and the rest should be gone. FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Reimar Döffinger b...@reimardoeffinger.de added the comment: On Fri, Feb 04, 2011 at 01:16:36AM +, drosenbe wrote: The attached vc1 file causes a stack overflow in VLC media player 1.1.6 when running on top of ffmpeg 0.5.3. The issue appears to also affect ffmpeg 0.6.1, but the corruption causes a crash in a different location. No crash is triggered in ffmpeg or ffplay, but VLC developers have indicated that this is an ffmpeg issue. This is not enough information by far. The issue is not reproducible using the latest FFmpeg, there is no description of the crash here nor on whether it is reproducible with some older version nor how the VLC developers determined it as a FFmpeg issue. FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Ronald S. Bultje rsbul...@gmail.com added the comment: ==56585== Invalid read of size 4 ==56585==at 0x10036F181: vc1_decode_ac_coeff (in ./ffmpeg_g) ==56585==by 0x100373E51: vc1_decode_i_blocks_adv (in ./ffmpeg_g)==56585==by 0x300017: ??? ==56585==by 0x1010EA52F: ??? ==56585==by 0x7FFF000C: ??? ==56585==by 0x40019: ??? ==56585==by 0x16: ??? ==56585==by 0x7FFF5FBFE45B: ??? ==56585==by 0x7FFF5FBFE457: ??? ==56585==by 0x7FFF5FBFE453: ??? ==56585==by 0x10106FFF2: ??? ==56585==by 0x5BFF: ??? ==56585== Address 0x101059597 is 1,687 bytes inside a block of size 1,690 alloc 'd ==56585==at 0x100CDFD06: memalign (vg_replace_malloc.c:581) ==56585==by 0x100CDFD5F: posix_memalign (vg_replace_malloc.c:709) ==56585==by 0x1004B50DC: av_mallocz (in ./ffmpeg_g) ==56585==by 0x7FFF5FBFE91F: ??? [..] ==56585== Invalid read of size 4 ==56585==at 0x100373A6A: vc1_decode_i_blocks_adv (in ./ffmpeg_g) ==56585==by 0x300017: ??? ==56585==by 0x1010EA52F: ??? ==56585==by 0x7FFF000C: ??? ==56585==by 0x40019: ??? ==56585==by 0x16: ??? ==56585==by 0x7FFF5FBFE45B: ??? ==56585==by 0x7FFF5FBFE457: ???==56585==by 0x7FFF5FBFE453: ??? ==56585==by 0x10106FFF2: ??? ==56585==by 0x5BFF: ??? ==56585==by 0x11: ??? ==56585== Address 0x101059598 is 1,688 bytes inside a block of size 1,690 alloc 'd ==56585==at 0x100CDFD06: memalign (vg_replace_malloc.c:581) ==56585==by 0x100CDFD5F: posix_memalign (vg_replace_malloc.c:709) ==56585==by 0x1004B50DC: av_mallocz (in ./ffmpeg_g) ==56585==by 0x7FFF5FBFE91F: ??? disass of second address: 0x000100373a1b vc1_decode_i_blocks_adv+1195: mov 0x35d0(%rbp),%ebx 0x000100373a21 vc1_decode_i_blocks_adv+1201: mov %ebx,0x94(%rsp) 0x000100373a28 vc1_decode_i_blocks_adv+1208: mov 0x35d4(%rbp),%eax 0x000100373a2e vc1_decode_i_blocks_adv+1214: mov %eax,0x90(%rsp) 0x000100373a35 vc1_decode_i_blocks_adv+1221: mov 0x90(%rbp),%r13d 0x000100373a3c vc1_decode_i_blocks_adv+1228: movslq 0x3d24(%rbp),%rax 0x000100373a43 vc1_decode_i_blocks_adv+1235: lea (%rax,%rax,2),%rax 0x000100373a47 vc1_decode_i_blocks_adv+1239: shl$0x3,%rax 0x000100373a4b vc1_decode_i_blocks_adv+1243: add 0x2bd6ae(%rip),%rax# 0x100631100 0x000100373a52 vc1_decode_i_blocks_adv+1250: mov 0x8(%rax),%r8 0x000100373a56 get_vlc2+0:mov0x3d68(%rbp),%esi 0x000100373a5c get_vlc2+6:mov0x3d58(%rbp),%r9 0x000100373a63 get_vlc2+13: mov%esi,%eax 0x000100373a65 get_vlc2+15: shr$0x3,%eax 0x000100373a68 get_vlc2+18: mov%eax,%eax 0x000100373a6a get_vlc2+20: mov(%r9,%rax,1),%eax 0x000100373a6e av_bswap32+0: bswap %eax 0x000100373a70 NEG_USR32+0: mov%esi,%ecx 0x000100373a72 NEG_USR32+2: and$0x7,%ecx 0x000100373a75 NEG_USR32+5: shl%cl,%eax 0x000100373a77 NEG_USR32+7: shr$0xf7,%eax disass of first: 0x00010036f15a vc1_decode_ac_coeff+42:lea 0x82c6df(%rip),%rdi# 0x100b9b840 ff_vc1_ac_coeff_table 0x00010036f161 vc1_decode_ac_coeff+49:movslq %r8d,%r8 0x00010036f164 vc1_decode_ac_coeff+52:lea(%r8,%r8,2),%rax 0x00010036f168 vc1_decode_ac_coeff+56:mov 0x8(%rdi,%rax,8),%r10 0x00010036f16d vc1_decode_ac_coeff+61:mov0x3d68(%rbx),%esi 0x00010036f173 vc1_decode_ac_coeff+67:mov0x3d58(%rbx),%rbp 0x00010036f17a vc1_decode_ac_coeff+74:mov%esi,%eax 0x00010036f17c vc1_decode_ac_coeff+76:shr$0x3,%eax 0x00010036f17f vc1_decode_ac_coeff+79:mov%eax,%eax 0x00010036f181 vc1_decode_ac_coeff+81:mov 0x0(%rbp,%rax,1),%eax 0x00010036f185 av_bswap32+0: bswap %eax 0x00010036f187 NEG_USR32+0: mov%esi,%ecx 0x00010036f189 NEG_USR32+2: and$0x7,%ecx 0x00010036f18c NEG_USR32+5: shl%cl,%eax 0x00010036f18e NEG_USR32+7: shr$0xf7,%eax 0x00010036f191 vc1_decode_ac_coeff+97:mov%eax,%eax 0x00010036f193 vc1_decode_ac_coeff+99:lea (%r10,%rax,4),%rax 0x00010036f197 vc1_decode_ac_coeff+103: movswl (%rax),%r9d 0x00010036f19b vc1_decode_ac_coeff+107: movswl 0x2(%rax),%edx 0x00010036f19f vc1_decode_ac_coeff+111: test %edx,%edx FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Reimar Döffinger b...@reimardoeffinger.de added the comment:
On Sat, Feb 05, 2011 at 07:49:34PM +, Ronald S. Bultje wrote:
==56585== Invalid read of size 4
==56585== Address 0x101059597 is 1,687 bytes inside a block of size
1,690 alloc
'd
==56585==at 0x100CDFD06: memalign (vg_replace_malloc.c:581)
==56585==by 0x100CDFD5F: posix_memalign (vg_replace_malloc.c:709)
==56585==by 0x1004B50DC: av_mallocz (in ./ffmpeg_g)
==56585==by 0x7FFF5FBFE91F: ???
[..]
==56585== Invalid read of size 4
==56585== Address 0x101059598 is 1,688 bytes inside a block of size
1,690 alloc
'd
==56585==at 0x100CDFD06: memalign (vg_replace_malloc.c:581)
==56585==by 0x100CDFD5F: posix_memalign (vg_replace_malloc.c:709)
==56585==by 0x1004B50DC: av_mallocz (in ./ffmpeg_g)
==56585==by 0x7FFF5FBFE91F: ???
Those are invalid reads and not even on stack memory, so
I see no relation with this issue.
And they are probably fixed by the patch I sent quite some time ago.
Probably it's this one:
Index: libavcodec/vc1dec.c
===
--- libavcodec/vc1dec.c (revision 26402)
+++ libavcodec/vc1dec.c (working copy)
@@ -1375,7 +1375,7 @@
if (index != vc1_ac_sizes[codingset] - 1) {
run = vc1_index_decode_table[codingset][index][0];
level = vc1_index_decode_table[codingset][index][1];
-lst = index = vc1_last_decode_table[codingset];
+lst = index = vc1_last_decode_table[codingset] || get_bits_left(gb)
0;
if(get_bits1(gb))
level = -level;
} else {
FFmpeg issue tracker iss...@roundup.ffmpeg.org
https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Ronald S. Bultje rsbul...@gmail.com added the comment: It fixes some, but not all. wc -l of valgrind ffmpeg goes from ~2000 to ~400, but still more warnings remain: ==61513== Invalid read of size 4 ==61513==at 0x10036E8BA: vc1_decode_i_blocks_adv (in ./ffmpeg_g) ==61513==by 0x300017: ??? ==61513==by 0x1010DC41F: ??? ==61513==by 0x7FFF000C: ??? ==61513==by 0x30019: ??? ==61513==by 0x16: ??? ==61513==by 0x7FFF5FBFE46B: ??? ==61513==by 0x7FFF5FBFE467: ??? ==61513==by 0x7FFF5FBFE463: ??? ==61513==by 0x10105FFF2: ??? ==61513==by 0x58FF: ??? ==61513==by 0x11: ??? ==61513== Address 0x1010beedc is 3,436 bytes inside a block of size 3,439 alloc'd 0x00010036e897 vc1_decode_i_blocks_adv+1239: shl$0x3,%rax 0x00010036e89b vc1_decode_i_blocks_adv+1243: add 0x2bc85e(%rip),%rax# 0x10062b100 0x00010036e8a2 vc1_decode_i_blocks_adv+1250: mov 0x8(%rax),%r8 0x00010036e8a6 get_vlc2+0:mov0x3cf8(%rbp),%esi 0x00010036e8ac get_vlc2+6:mov0x3ce8(%rbp),%r9 0x00010036e8b3 get_vlc2+13: mov%esi,%eax 0x00010036e8b5 get_vlc2+15: shr$0x3,%eax 0x00010036e8b8 get_vlc2+18: mov%eax,%eax 0x00010036e8ba get_vlc2+20: mov(%r9,%rax,1),%eax 0x00010036e8be av_bswap32+0: bswap %eax 0x00010036e8c0 NEG_USR32+0: mov%esi,%ecx 0x00010036e8c2 NEG_USR32+2: and$0x7,%ecx 0x00010036e8c5 NEG_USR32+5: shl%cl,%eax 0x00010036e8c7 NEG_USR32+7: shr$0xf7,%eax ==61513== Invalid read of size 4 ==61513==at 0x100369FF1: vc1_decode_ac_coeff (in ./ffmpeg_g) ==61513==by 0x10036ECA1: vc1_decode_i_blocks_adv (in ./ffmpeg_g) ==61513==by 0x300017: ??? ==61513==by 0x1010E22FF: ??? ==61513==by 0x7FFF000C: ??? ==61513==by 0x30019: ??? ==61513==by 0x16: ??? ==61513==by 0x7FFF5FBFE46B: ??? ==61513==by 0x7FFF5FBFE467: ??? ==61513==by 0x7FFF5FBFE463: ??? ==61513==by 0x10105FFF2: ??? ==61513==by 0x58FF: ??? ==61513== Address 0x1010beedc is 3,436 bytes inside a block of size 3,439 alloc'd (gdb) disass 0x100369FF1 Dump of assembler code for function vc1_decode_ac_coeff: 0x000100369fa0 vc1_decode_ac_coeff+0: mov%rbx,-0x30(%rsp) 0x000100369fa5 vc1_decode_ac_coeff+5: mov%rbp,-0x28(%rsp) 0x000100369faa vc1_decode_ac_coeff+10:mov%r12,-0x20(%rsp) 0x000100369faf vc1_decode_ac_coeff+15:mov%r13,-0x18(%rsp) 0x000100369fb4 vc1_decode_ac_coeff+20:mov%r14,-0x10(%rsp) 0x000100369fb9 vc1_decode_ac_coeff+25:mov%r15,-0x8(%rsp) 0x000100369fbe vc1_decode_ac_coeff+30:mov%rdi,%rbx 0x000100369fc1 vc1_decode_ac_coeff+33:mov%rsi,%r13 0x000100369fc4 vc1_decode_ac_coeff+36:mov%rdx,%r14 0x000100369fc7 vc1_decode_ac_coeff+39:mov%rcx,%r15 0x000100369fca vc1_decode_ac_coeff+42:lea 0x828c4f(%rip),%rdi# 0x100b92c20 ff_vc1_ac_coeff_table 0x000100369fd1 vc1_decode_ac_coeff+49:movslq %r8d,%r8 0x000100369fd4 vc1_decode_ac_coeff+52:lea(%r8,%r8,2),%rax 0x000100369fd8 vc1_decode_ac_coeff+56:mov 0x8(%rdi,%rax,8),%r10 0x000100369fdd vc1_decode_ac_coeff+61:mov0x3cf8(%rbx),%esi 0x000100369fe3 vc1_decode_ac_coeff+67:mov0x3ce8(%rbx),%rbp 0x000100369fea vc1_decode_ac_coeff+74:mov%esi,%eax 0x000100369fec vc1_decode_ac_coeff+76:shr$0x3,%eax 0x000100369fef vc1_decode_ac_coeff+79:mov%eax,%eax 0x000100369ff1 vc1_decode_ac_coeff+81:mov 0x0(%rbp,%rax,1),%eax 0x000100369ff5 av_bswap32+0: bswap %eax i.e. this is the get_vlc2() 4 lines above the one your patch touches. Also during playback a lot of these warnings: [vc1 @ 0x100f8bbc0] Luma scaling is not supported, expect wrong picture [vc1 @ 0x100f8bbc0] Chroma scaling is not supported, expect wrong picture FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Reimar Döffinger b...@reimardoeffinger.de added the comment: On Sat, Feb 05, 2011 at 08:29:06PM +, Ronald S. Bultje wrote: It fixes some, but not all. wc -l of valgrind ffmpeg goes from ~2000 to ~400, but still more warnings remain: ==61513== Invalid read of size 4 ==61513==at 0x10036E8BA: vc1_decode_i_blocks_adv (in ./ffmpeg_g) ==61513==by 0x300017: ??? ==61513==by 0x1010DC41F: ??? ==61513==by 0x7FFF000C: ??? ==61513==by 0x30019: ??? ==61513==by 0x16: ??? ==61513==by 0x7FFF5FBFE46B: ??? ==61513==by 0x7FFF5FBFE467: ??? ==61513==by 0x7FFF5FBFE463: ??? ==61513==by 0x10105FFF2: ??? ==61513==by 0x58FF: ??? ==61513==by 0x11: ??? ==61513== Address 0x1010beedc is 3,436 bytes inside a block of size 3,439 alloc'd *shrug* that's a one-byte overread, I do not care that much about that. I don't think this is really the right place to discuss this, the bug report was about a stack overflow, assuming that means a stack buffer overflow (as in, writes+on stack) that is really, really serious and needs to be fixed ASAP. These overreads do not. FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
New submission from drosenbe dan.j.rosenb...@gmail.com: The attached vc1 file causes a stack overflow in VLC media player 1.1.6 when running on top of ffmpeg 0.5.3. The issue appears to also affect ffmpeg 0.6.1, but the corruption causes a crash in a different location. No crash is triggered in ffmpeg or ffplay, but VLC developers have indicated that this is an ffmpeg issue. -- messages: 13600 priority: normal status: new substatus: new title: VLC stack overflow in libavcodec with vc1 file type: bug FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
[issue2584] VLC stack overflow in libavcodec with vc1 file
Kees Cook k...@outflux.net added the comment: This has been assigned CVE-2011-0723 FFmpeg issue tracker iss...@roundup.ffmpeg.org https://roundup.ffmpeg.org/issue2584
