On 10/8/24 09:47, Peter Maydell wrote:
In compare_fingerprint() we effectively check whether the characters
in the fingerprint are valid hex digits twice: first we do so with
qemu_isxdigit(), but then the hex2decimal() function also has a code
path where it effectively detects an invalid digit and returns -1.
This causes Coverity to complain because it thinks that we might use
that -1 value in an expression where it would be an integer overflow.
Avoid the double-check of hex digit validity by testing the return
values from hex2decimal() rather than doing separate calls to
qemu_isxdigit().
Since this means we now use the illegal-character return value
from hex2decimal(), rewrite it from "-1" to "UINT_MAX", which
has the same effect since the return type is "unsigned" but
looks less confusing at the callsites when we detect it with
"c0 > 0xf".
Resolves: Coverity CID 1547813
Signed-off-by: Peter Maydell<[email protected]>
Reviewed-by: Kevin Wolf<[email protected]>
---
v1->v2: make hex2decimal() return UINT_MAX, not -1
---
block/ssh.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
Reviewed-by: Richard Henderson <[email protected]>
r~
