| Issue |
168923
|
| Summary |
>4GB MCFragments causing trouble (llvm-dwp failing for big inputs in llvm-21)
|
| Labels |
new issue
|
| Assignees |
|
| Reporter |
MatzeB
|
We are having trouble with `llvm-dwp` running out of memory in llvm-21 for big builds resulting in giant dwp files where some sections are > 4GB in size.
My first analysis looks like it may be triggered by recent MC layer rewrites, like https://github.com/llvm/llvm-project/commit/0393084adc9028c6babb95fb48d4ef1897093412 that introduce fields like `uint32_t ContentStart` to `MCEncodedFragment`. And my understanding is that in our case llvm-dwp ends up creating a single fragment of more than 4GB after repeatet `emitBytes` calls here: https://github.com/llvm/llvm-project/blob/main/llvm/lib/DWP/DWP.cpp#L860
(The OOM situation seems to be accidental by `ContentStart` overflowing causing `getContentsForAPpending` to wrongly grow the contents vector to arbitrary size).
The code seems to have changed lately but I am still seeing various 32bit offsets in the fragment code. Should we increase those fields to 64bits or finds a way to not write everything into a single fragment in this use case?
CC @MaskRay
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
