Hi,
I experienced a small bug in the smartyuv filter which inserts black and
white pixels depending on the surrounding pixels. It looks like the
problem was mentioned earlier:
http://itdp.fh-biergarten.de/~itdp/html/transcode-devel/2004-06/msg00012.html
The cubic interpolation doesn't check whether the new pixel value is
between 0 and 255. Values over 255 can turn into black pixel due to 8
bit conversion and values below 0 can be white.
I have attached a patch which fixes this problem by limiting the value
to 255 or 0. The patch works for me but perhaps different limits are
more suitable (there's some commented code about 240/16 limits, I don't
know what's better).
Anyway, thanks for the great tool, I've used it for years now.
Best Regards,
Ralf Hoffmann
--
Homepage: http://www.boomerangsworld.de
E-Mail: Ralf Hoffmann <[EMAIL PROTECTED]>
English or German
--- transcode-1.0.2/filter/filter_smartyuv.c 2005-07-04 09:09:32.000000000 +0200
+++ transcode-1.0.2-new/filter/filter_smartyuv.c 2007-02-05 21:50:08.000000000 +0100
@@ -1040,6 +1040,12 @@ static void smartyuv_core (char *_src, c
if (R>240) R = 240;
else if (R<16) R=16;
*/
+
+ /* handle over- and underflow */
+ if (R < 0)
+ R = 0;
+ else if (R > 255)
+ R = 255;
dst[x] = clamp_f(R & 0xff)&0xff;
}
else
@@ -1223,6 +1229,11 @@ static void smartyuv_core (char *_src, c
{
R = (5 * ((srcminus[x] & 0xff) + (srcplus[x] & 0xff))
- ((srcminusminus[x] & 0xff) + (srcplusplus[x] & 0xff))) >> 3;
+ /* handle over- and underflow */
+ if (R < 0)
+ R = 0;
+ else if (R > 255)
+ R = 255;
dst[x] = clamp_f(R & 0xff)&0xff;
}
else
