New submission from Fred <[EMAIL PROTECTED]>: In mpegvideo_enc.c - dct_quantize_c(), the level computed may wrongly be detected greater than the threshold.
As of ffmpeg code gathered on October 9th, 2007 (external at revision 24760, revision 10698), at lines 3682 and 3695 the following condition is checked: if(((unsigned)(level+threshold1))>threshold2). level is a signed integer, while both threshold1 and threshold2 are unsigned. When level is negative, by casting as unsigned, the sign bit resulting from (level+threshold1) gives very big value. This seems buggy to me, maybe using if((abs(level+threshold1))>threshold2) instead as casting to (integer) is what the author really wanted. This potential problem can fool the rate control for videos with a lot of high frequencies. Note that the MMX version of dct_quantize_c() seems to have the same problem. ---------- messages: 778 nosy: zorg669 priority: normal status: new substatus: new title: MPEG Video DCT Quantizer Threshold Possibly Broken type: bug ______________________________________________________ FFmpeg issue tracker <[EMAIL PROTECTED]> <https://roundup.mplayerhq.hu/roundup/ffmpeg/issue215> ______________________________________________________
