On Tue, 2011-10-18 at 12:50 +0100, Måns Rullgård wrote: > "Ronald S. Bultje" <[email protected]> writes: > > > + int64_t dd = d > SQRT_INT64_MAX ? ((d >> 1) * d) >> > > 29 : (d * d) >> 30; > > + int64_t ddd = d > SQRT_INT64_MAX || dd > > > SQRT_INT64_MAX ? > > + ((dd >> 2) * d) >> > > 28 : (dd * d) >> 30; > > It is still overflowing. If d uses 32 bits, d*d >> 30 uses 34, and > (dd >> 2) * d will still overflow. It needs >> 3 or some other > equivalent change.
Where did you get "32" from? I see no obvious reason why exactly that would be the upper limit - the types are signed so 32 bits is not a natural limit, and the values used look like it could be larger than 32 bits unless there's something special which prevents that. _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
