req is of type size_t, casting it to long opens the door for an integer overflow. Values between LONG_MAX - (SIZE_SZ + MALLOC_ALIGN_MASK) - 1 and LONG_MAX cause and overflow such that request2size() returns MINSIZE.
Fix by removing the cast. The origin of the cast is unclear, it's in u-boot and ppcboot since ever and predates the CVS history. Doug Lea's original dlmalloc implementation also doesn't have it. Signed-off-by: Richard Weinberger <[email protected]> --- common/dlmalloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/common/dlmalloc.c b/common/dlmalloc.c index 62e8557daa..44b06e38b2 100644 --- a/common/dlmalloc.c +++ b/common/dlmalloc.c @@ -386,8 +386,8 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /* pad request bytes into a usable size */ #define request2size(req) \ - (((long)((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) < \ - (long)(MINSIZE + MALLOC_ALIGN_MASK)) ? MINSIZE : \ + ((((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) < \ + (MINSIZE + MALLOC_ALIGN_MASK)) ? MINSIZE : \ (((req) + (SIZE_SZ + MALLOC_ALIGN_MASK)) & ~(MALLOC_ALIGN_MASK))) /* Check if m has acceptable alignment */ -- 2.35.3
