Hi, On Sun, Jan 27, 2013 at 1:25 PM, Derek Buitenhuis <[email protected]> wrote: > From: Michael Niedermayer <[email protected]> > > We have to make some symetric changes elsewhere as this increases > the precission with which samples are stored. > > Signed-off-by: Michael Niedermayer <[email protected]> > Signed-off-by: Derek Buitenhuis <[email protected]> > --- > libswscale/input.c | 6 +++--- > libswscale/swscale_internal.h | 10 +++++++++- > tests/ref/lavfi/pixfmts_scale | 12 ++++++------ > 3 files changed, 18 insertions(+), 10 deletions(-) > > diff --git a/libswscale/input.c b/libswscale/input.c > index 2e8d43f..64ab0b9 100644 > --- a/libswscale/input.c > +++ b/libswscale/input.c > @@ -579,7 +579,7 @@ static av_always_inline void planar_rgb16_to_y(uint8_t > *_dst, const uint8_t *_sr > int b = rdpx(src[1] + i); > int r = rdpx(src[2] + i); > > - dst[i] = ((RY * r + GY * g + BY * b + (33 << (RGB2YUV_SHIFT + bpc - > 9))) >> RGB2YUV_SHIFT); > + dst[i] = ((RY * r + GY * g + BY * b + (33 << (RGB2YUV_SHIFT + bpc - > 9))) >> (RGB2YUV_SHIFT + bpc - 14));
This looks wrong. The original code also btw. We're doing (cast_to_fixed_point)(r*ry + g*gy + b*by + 16 + 0.5). I don't think this does the correct thing now. > @@ -626,8 +626,8 @@ static av_always_inline void planar_rgb16_to_uv(uint8_t > *_dstU, uint8_t *_dstV, > int b = rdpx(src[1] + i); > int r = rdpx(src[2] + i); > > - dstU[i] = (RU * r + GU * g + BU * b + (257 << (RGB2YUV_SHIFT + bpc - > 9))) >> RGB2YUV_SHIFT; > - dstV[i] = (RV * r + GV * g + BV * b + (257 << (RGB2YUV_SHIFT + bpc - > 9))) >> RGB2YUV_SHIFT; > + dstU[i] = (RU * r + GU * g + BU * b + (257 << (RGB2YUV_SHIFT + bpc - > 9))) >> (RGB2YUV_SHIFT + bpc - 14); > + dstV[i] = (RV * r + GV * g + BV * b + (257 << (RGB2YUV_SHIFT + bpc - > 9))) >> (RGB2YUV_SHIFT + bpc - 14); Same. The round and shift should correspond to each other in both cases, and now they don't. > #define isAnyRGB(x) \ > (isRGBinInt(x) || \ > - isBGRinInt(x)) > + isBGRinInt(x) || \ > + (x)==AV_PIX_FMT_GBRP9LE || \ > + (x)==AV_PIX_FMT_GBRP9BE || \ > + (x)==AV_PIX_FMT_GBRP10LE || \ > + (x)==AV_PIX_FMT_GBRP10BE || \ > + (x)==AV_PIX_FMT_GBRP16LE || \ > + (x)==AV_PIX_FMT_GBRP16BE || \ > + (x)==AV_PIX_FMT_GBRP \ > + ) || isPlanarRGB(x)? Ronald _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
