Uoti Urpala <[email protected]> writes: > On Tue, 2011-10-18 at 12:50 +0100, Måns Rullgård wrote: >> "Ronald S. Bultje" <[email protected]> writes: >> >> > + int64_t dd = d > SQRT_INT64_MAX ? ((d >> 1) * d) >> >> > 29 : (d * d) >> 30; >> > + int64_t ddd = d > SQRT_INT64_MAX || dd > >> > SQRT_INT64_MAX ? >> > + ((dd >> 2) * d) >> >> > 28 : (dd * d) >> 30; >> >> It is still overflowing. If d uses 32 bits, d*d >> 30 uses 34, and >> (dd >> 2) * d will still overflow. It needs >> 3 or some other >> equivalent change. > > Where did you get "32" from? I see no obvious reason why exactly that > would be the upper limit - the types are signed so 32 bits is not a > natural limit, and the values used look like it could be larger than 32 > bits unless there's something special which prevents that.
I saw it use exactly 32 bits in testing. -- Måns Rullgård [email protected] _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
