Hi, On Wed, Jan 13, 2016 at 9:50 PM, Michael Niedermayer <michae...@gmx.at> wrote:
> From: Michael Niedermayer <mich...@niedermayer.cc> > > This makes SWS more robust > Fixes: > 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264 > Fixes: out of array read > > Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind > Signed-off-by: Michael Niedermayer <mich...@niedermayer.cc> > --- > libswscale/swscale_internal.h | 2 +- > libswscale/yuv2rgb.c | 88 > ++++++++++++++++++++--------------------- > 2 files changed, 45 insertions(+), 45 deletions(-) So ... I'm kind of confused. You have a 264 file that triggers a OOB in swscale, probably through automated testing of ffmpeg.exe -i file.264 -f null -. Can you elaborate on what happens exactly? I guess what I'm trying to get at is, what's the input format (I'm going to assume it's yuv420), what's the output (maybe rgb24?), why is it converting like that (is the 264 changing pixfmt from frame to frame?), are the coefficients defaults for a particular colorspace/range (e.g. bt601/mpeg) or custom, what _are_ the coefficients, what coefficients can swscale use in these functions, and what are the theoretical bounds of the index in each array based on these coefficients? In other words, why do we need a headroom of 256/512/384/326/896/838 in each of these tables? How do we verify that it is correct? I remember Jason tried to speed up av_clip_uint8() at some point by making it a table, and we had to revert the use of that in many places (e.g. idcts) because we cannot give a theoretical limit on input values, i.e. the table would have to be infinitely long. I'm trying to figure out if that's the case here also. Ronald _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
