"Ronald S. Bultje" <[email protected]> writes: > + int64_t dd = d > SQRT_INT64_MAX ? ((d >> 1) * d) >> 29 > : (d * d) >> 30; > + int64_t ddd = d > SQRT_INT64_MAX || dd > SQRT_INT64_MAX ? > + ((dd >> 2) * d) >> 28 > : (dd * d) >> 30;
It is still overflowing. If d uses 32 bits, d*d >> 30 uses 34, and (dd >> 2) * d will still overflow. It needs >> 3 or some other equivalent change. -- Måns Rullgård [email protected] _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
