Here is another batch of fixes for undefined behaviour found via running FATE through UBSan.* Most of the issues found are invalid left shifts of negative numbers and so are my patches.
I have only fixed issues where I was sure that no overflow can happen in the multiplication that replaced a left shift; simply checking that it fixed the FATE test was not enough. - Andreas *: While I earlier recommended compiling with -fsanitize-trap=undefined, I now use -fno-sanitize-recover=undefined. Gives nicer logs. Andreas Rheinhardt (15): avutil/common: Add macro for left-shifting avcodec/truespeech: Fix invalid shift avfilter/vf_xbr: Fix left shift of negative number avfilter/hqx: Fix undefined left shifts of negative numbers swscale/x86/swscale: Fix undefined left shifts of negative numbers avcodec/adpcm: Fix undefined left shifts of negative numbers avcodec/cavsdsp: Fix undefined left shifts of negative numbers avcodec/ffv1enc: Factor check out of a loop avcodec/ffv1enc: Fix out-of-bounds-array access swscale/utils: Fix invalid left shifts of negative numbers avformat/webmdashenc: Don't pass NULL to memcmp avcodec/exr: Fix undefined left shifts of negative numbers avcodec/vc1_pred: Fix undefined left shifts of negative numbers avcodec/cinepakenc: Fix invalid shifts avcodec/dnxhdenc: Fix undefined left shifts of negative numbers libavcodec/adpcm.c | 20 +++++++-------- libavcodec/cavsdsp.c | 52 +++++++++++++++++++-------------------- libavcodec/cinepakenc.c | 11 +++++---- libavcodec/dnxhdenc.c | 6 ++--- libavcodec/exr.c | 2 +- libavcodec/ffv1enc.c | 11 ++++++--- libavcodec/truespeech.c | 2 +- libavcodec/vc1_pred.c | 4 +-- libavfilter/vf_hqx.c | 2 +- libavfilter/vf_xbr.c | 2 +- libavformat/webmdashenc.c | 3 ++- libavutil/common.h | 8 ++++++ libswscale/utils.c | 2 +- libswscale/x86/swscale.c | 4 +-- 14 files changed, 72 insertions(+), 57 deletions(-) -- 2.20.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
