Re: [PATCH] atm: he: fix sign-extension overflow on large shift
From: Colin King Date: Tue, 15 Jan 2019 18:03:38 + > From: Colin Ian King > > Shifting the 1 by exp by an int can lead to sign-extension overlow when > exp is 31 since 1 is an signed int and sign-extending this result to an > unsigned long long will set the upper 32 bits. Fix this by shifting an > unsigned long. > > Detected by cppcheck: > (warning) Shifting signed 32-bit value by 31 bits is undefined behaviour > > Signed-off-by: Colin Ian King Applied.
Re: [PATCH] atm: he: fix sign-extension overflow on large shift
On Tue, Jan 15, 2019 at 06:03:38PM +, Colin King wrote: > From: Colin Ian King > > Shifting the 1 by exp by an int can lead to sign-extension overlow when > exp is 31 since 1 is an signed int and sign-extending this result to an > unsigned long long will set the upper 32 bits. Fix this by shifting an > unsigned long. You could just cast it to unsigned. rate_cps = (unsigned long long) (1U << exp) * (man + 512) >> 9; I don't think it makes a difference at runtime because we just pick buf = 4 either way... regards, dan carpenter
