"Ronald S. Bultje" <[email protected]> writes: > This fixes the same overflow as in the RGB48/16-bit YUV scaling; > some filters can overflow both negatively and positively (e.g. > spline/lanczos), so we bias a signed integer so it's "half signed" > and "half unsigned", and can cover overflows in both directions > while maintaining full 31-bit depth. > --- > libswscale/swscale.c | 14 ++++++-------- > 1 files changed, 6 insertions(+), 8 deletions(-) > > diff --git a/libswscale/swscale.c b/libswscale/swscale.c > index 3ea3710..c9dfc8d 100644 > --- a/libswscale/swscale.c > +++ b/libswscale/swscale.c > @@ -386,8 +386,8 @@ yuv2gray16_X_c_template(SwsContext *c, const int16_t > *lumFilter, > > for (i = 0; i < (dstW >> 1); i++) { > int j; > - int Y1 = 1 << 14; > - int Y2 = 1 << 14; > + int Y1 = (1 << 14) - 0x40000000; > + int Y2 = (1 << 14) - 0x40000000; > > for (j = 0; j < lumFilterSize; j++) { > Y1 += lumSrc[j][i * 2] * lumFilter[j]; > @@ -395,12 +395,10 @@ yuv2gray16_X_c_template(SwsContext *c, const int16_t > *lumFilter, > } > Y1 >>= 15; > Y2 >>= 15; > - if ((Y1 | Y2) & 0x10000) { > - Y1 = av_clip_uint16(Y1); > - Y2 = av_clip_uint16(Y2); > - } > - output_pixel(&dest[i * 2 + 0], Y1); > - output_pixel(&dest[i * 2 + 1], Y2); > + Y1 = av_clip_int16(Y1); > + Y2 = av_clip_int16(Y2); > + output_pixel(&dest[i * 2 + 0], 0x8000 + Y1); > + output_pixel(&dest[i * 2 + 1], 0x8000 + Y2); > } > } > > --
Looks OK. -- Måns Rullgård [email protected] _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
