This multiplication can overflow the signed range but not the unsigned. After right-shifting it will thus fit in the signed range again.
Signed-off-by: Mans Rullgard <[email protected]> --- libavcodec/svq1enc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/libavcodec/svq1enc.c b/libavcodec/svq1enc.c index 79aaa6c..01d8b31 100644 --- a/libavcodec/svq1enc.c +++ b/libavcodec/svq1enc.c @@ -160,7 +160,7 @@ static int encode_block(SVQ1Context *s, uint8_t *src, uint8_t *ref, uint8_t *dec } best_count=0; - best_score -= ((block_sum[0]*block_sum[0])>>(level+3)); + best_score -= (int)(((unsigned)block_sum[0]*block_sum[0])>>(level+3)); best_mean= (block_sum[0] + (size>>1)) >> (level+3); if(level<4){ -- 1.7.7.4 _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
