On Wed, 4 Sep 2013, Luca Barbato wrote:
On 03/09/13 22:34, Martin Storsjö wrote:
From: Michael Niedermayer <[email protected]>
If the src_y variable is very large, the multiplication
src_y * linesize might overflow. The same multiplication is
done in the caller (e.g. mpeg_motion_internal) where both src_y
and linesize are ints, where part of this offsetting is undone
within emulated_edge_mc. If this multplication overflowed in the
caller, make sure it overflows in the same way here.
This is mostly relevant for e.g. h263+ where the motion vectors
can be unrestricted.
Wouldn't make more sense using the int64 in the caller as well?
Possibly. There's a number of callers that should be checked though, I
think most of them use int right now. Not sure if it's enough to just
check it in mpegvideo, do other codecs also allow arbitrarily large motion
vectors without erroring out earlier?
I guess the callers should rather use ptrdiff_t instead of int64_t
explicitly, since that's what the emu_edge function does right now.
The splitting of the multplications (from (src_y + a + b) * linesize into
src_y * linesize + (a+b)*linesize) should probably be done still, so that
we're sure that if the src_y*linesize part still overflows, it does in the
same fashion as elsewhere.
// Martin
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
