Hi, 2011/12/17 Måns Rullgård <[email protected]>
> "Ronald S. Bultje" <[email protected]> writes: > > > From: "Ronald S. Bultje" <[email protected]> > > > > This fixes the same overflow as in the RGB48/16-bit YUV scaling; > > some filters can overflow both negatively and positively (e.g. > > spline/lanczos), so we bias a signed integer so it's "half signed" > > and "half unsigned", and can cover overflows in both directions > > while maintaining full 31-bit depth. > > --- > > libswscale/swscale.c | 12 ++++++------ > > 1 files changed, 6 insertions(+), 6 deletions(-) > > > > diff --git a/libswscale/swscale.c b/libswscale/swscale.c > > index 592c9d6..60652d9 100644 > > --- a/libswscale/swscale.c > > +++ b/libswscale/swscale.c > > @@ -386,8 +386,8 @@ yuv2gray16_X_c_template(SwsContext *c, const int16_t > *lumFilter, > > > > for (i = 0; i < (dstW >> 1); i++) { > > int j; > > - int Y1 = 1 << 14; > > - int Y2 = 1 << 14; > > + int Y1 = (1 << 14) - 0x40000000; > > + int Y2 = (1 << 14) - 0x40000000; > > > > for (j = 0; j < lumFilterSize; j++) { > > Y1 += lumSrc[j][i * 2] * lumFilter[j]; > > @@ -396,11 +396,11 @@ yuv2gray16_X_c_template(SwsContext *c, const > int16_t *lumFilter, > > Y1 >>= 15; > > Y2 >>= 15; > > if ((Y1 | Y2) & 0x10000) { > > This overflow check will now trigger for anything with a negative value, > even if in range. The clipping is of course a no-op in those cases, so > it does no damage as such, but it might not be what you intended. > > > - Y1 = av_clip_uint16(Y1); > > - Y2 = av_clip_uint16(Y2); > > + Y1 = av_clip_int16(Y1); > > + Y2 = av_clip_int16(Y2); > Uhm, ohright, good point. I think we can just remove the """optimization""", i.e. remove the if()... Ronald
_______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
