From: Xi Wang <[email protected]> The check `start + res < start' is broken since pointer overflow is undefined behavior in C. Many compilers such as gcc/clang optimize away this check.
Use `res > end - start' instead. Also change `res' to unsigned int to avoid signed left-shift overflow. Signed-off-by: Xi Wang <[email protected]> --- libavformat/rtpenc_h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavformat/rtpenc_h264.c b/libavformat/rtpenc_h264.c index ac74074..206d9ba 100644 --- a/libavformat/rtpenc_h264.c +++ b/libavformat/rtpenc_h264.c @@ -31,14 +31,14 @@ static const uint8_t *avc_mp4_find_startcode(const uint8_t *start, const uint8_t *end, int nal_length_size) { - int res = 0; + unsigned int res = 0; if (end - start < nal_length_size) return NULL; while (nal_length_size--) res = (res << 8) | *start++; - if (start + res > end || res < 0 || start + res < start) + if (res > end - start) return NULL; return start + res; -- 1.7.9.4 _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
