#10852: sws_scale overflows buffer for some resolutions using ssse3 instructions
---------------------------------+--------------------------------------
Reporter: Jerome | Type: defect
Status: new | Priority: important
Component: swscale | Version: git-master
Keywords: memory | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
---------------------------------+--------------------------------------
Summary of the bug:
This bug occurs when using the sws_scale function to convert the color
space of an image from yuv420 to bgr24 (it might occur for other color
spaces, I haven't verified).
It started happening with version 5.0 and is still happening with the
latest master. It does not systematically happen, only for some image
resolution. When writing the output image, sws_scale goes beyond the image
buffer by 8 bytes. This memory violation often triggers a segfault.
Valgrind output:
==1050090== Memcheck, a memory error detector
==1050090== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et
al.
==1050090== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright
info
==1050090== Command: ./test_memcheck
==1050090==
[swscaler @ 0x62cb680] deprecated pixel format used, make sure you did set
range correctly
Convert unscaled 3240
==1050090== Invalid write of size 8
==1050090== at 0x48EEAF6: ??? (libavutil/x86/x86inc.asm:1274)
==1050090== by 0x48ED6F2: yuv420_bgr24_ssse3 (yuv2rgb_template.c:193)
==1050090== by 0x48B5B5F: scale_internal (swscale.c:1042)
==1050090== by 0x48B6668: sws_scale (swscale.c:1214)
==1050090== by 0x109393: main (main_memcheck.cpp:40)
==1050090== Address 0x69aac40 is 0 bytes after a block of size 6,220,800
alloc'd
==1050090== at 0x4847203: operator new[](unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050090== by 0x10931A: main (main_memcheck.cpp:23)
==1050090==
==1050090== Invalid write of size 8
==1050090== at 0x48EEAFB: ??? (libavutil/x86/x86inc.asm:1274)
==1050090== by 0x48ED6F2: yuv420_bgr24_ssse3 (yuv2rgb_template.c:193)
==1050090== by 0x48B5B5F: scale_internal (swscale.c:1042)
==1050090== by 0x48B6668: sws_scale (swscale.c:1214)
==1050090== by 0x109393: main (main_memcheck.cpp:40)
==1050090== Address 0x69aac48 is 8 bytes after a block of size 6,220,800
alloc'd
==1050090== at 0x4847203: operator new[](unsigned long) (in
/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050090== by 0x10931A: main (main_memcheck.cpp:23)
How to reproduce:
I'm attaching a small C++ program that illustrates how to reproduce the
bug.
--
Ticket URL: <https://trac.ffmpeg.org/ticket/10852>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
_______________________________________________
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-trac
To unsubscribe, visit link above, or email
ffmpeg-trac-requ...@ffmpeg.org with subject "unsubscribe".
