I'm running out of virtual memory on hardware that supports 5 level page tables, yes.
On Wed, Aug 6, 2025 at 2:53 PM Olivier Flückiger <[email protected]> wrote: > Right, if we request heap pages in that area. Are you running into this > issue? > > On Wed, Aug 6, 2025 at 2:26 PM Erik Corry <[email protected]> wrote: > >> If your hardware supports 5-level page tables and your kernel is >> reasonably modern you can get addresses today where only the top 8 bits are >> zero: >> >> The following program prints: >> >> Size 64T, addr = 0x2000000000000 >> Size 128T, addr = 0xffba0bca5a6000 >> >> The high hint to mmap is important - if you don't have that then the >> kernel is backwards compatible and won't give out high addresses beyond the >> 48 bit limit. This is why V8 still works on such hardware, but it limits >> the amount of virtual memory you can use. >> >> #include <errno.h> >> #include <stdio.h> >> #include <sys/mman.h> >> >> int main() { >> unsigned long size = 1ULL << 46; >> for (int i = 0; i < 2; i++) { >> void* addr = mmap( >> reinterpret_cast<void*>(1ULL << 49), >> 1ULL << 46, >> PROT_NONE, >> MAP_PRIVATE | MAP_ANONYMOUS, >> 0, >> 0); >> printf("Size %dT, addr = %p\n", (int)(size >> 40), addr); >> if (addr == reinterpret_cast<void*>(-1)) { >> perror("mmap"); >> return 1; >> } >> size <<= 1; >> } >> return 0; >> } >> >> On Wed, Aug 6, 2025 at 9:24 AM Olivier Flückiger <[email protected]> >> wrote: >> >>> Yeah, we are not clinging to that design. It really should be a normal >>> compressed pointer. Then we'd have space for a separate mark bit and >>> argument count like on 32 bit architectures. Last I checked there were some >>> technical issues with getting the correct base to uncompress the pointer >>> and it's also kinda performance sensitive. That's why nobody has addressed >>> it so far. >>> >>> That said, I don't think we have to worry about user space pointers in >>> that range according to >>> https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt . >>> >>> *oli >>> >>> >>> >>> On Mon, Aug 4, 2025 at 1:04 PM Erik Corry <[email protected]> >>> wrote: >>> >>>> Seems like this will soon be a problem for Linux too. My /proc/cpuinfo >>>> says: >>>> >>>> address sizes : 52 bits physical, 57 bits virtual >>>> >>>> so it looks like we can't assume the high 16 bits are zero for much >>>> longer. >>>> >>>> >>>> On Tuesday, April 22, 2025 at 5:21:16 PM UTC+2 [email protected] >>>> wrote: >>>> >>>>> Per the nearly-approved AIX commit conversation ( >>>>> https://chromium-review.googlesource.com/c/v8/v8/+/6320599 ), I would >>>>> like to address V8's mishandling of illumos/amd64 VA48 available address >>>>> space. The problem is rather straightforward: illumos amd64 processes >>>>> have their 48-bit available VA space split into two parts. >>>>> >>>>> Per the original amd64/x64 4-level paging spec: the low 47 bits of >>>>> address space: 0x0 -> 0x00007fffffffffff are available, AND SO IS the high >>>>> 47 bits of available address space: 0xffff800000000000 -> >>>>> 0xffffffffffffffff. Notwithstanding carve-outs toward the extremes of the >>>>> 64-bit address space (i.e. not too close to 0 or to 0xffffffffffffffff), >>>>> memory mappings can come from either of those ranges. >>>>> >>>>> For pointer-compression that shifts 16 bits to the left, the easy >>>>> thing to do is to check the highest-order compressed-pointer bit, and fill >>>>> the top 16 with 1s upon decompression. Decompression is done in >>>>> CodeStubAssembler::LoadCodeObjectFromJSDispatchTable(), and while I >>>>> had my first-attempt implementation picked-apart as part of my experiments >>>>> with Node, the idea is sound. >>>>> >>>>> Also, the choice made in the JS Dispatch Table to mark a free pointer >>>>> with 0xffff in the top 16 bits will not work in an amd64 address space >>>>> using all of the available VA space, because half of it lives in address >>>>> space starting with 0xffff. A change of marking bits (I used 0xfeed in >>>>> the >>>>> first-attempt) and better clearing/checking (using logical-and) solves >>>>> this. >>>>> >>>>> The assumption of only-low-47-bits of virtual address space runs up >>>>> against two problems. The first is expanded available virtual address >>>>> space beyond 0x0000800000000000. The aforementioned IBM/AIX changes hint >>>>> at this possibility: All 48 lower bits are available with a fixed non-zero >>>>> 16-bit prefix. The second is that at least for amd64, address space will >>>>> appear in both the high-end and the low-end of the 64-bit address space. >>>>> >>>>> Already available in some hardware is VA57, which resembles the >>>>> aforementioned VA48 except that the low available space grows to 0x0 -> >>>>> 0x007fffffffffffff, and the high available space grows down to cover >>>>> 0xff80000000000000 -> 0xffffffffffffffff. Operating systems may offer the >>>>> entirety of both ends of VA57 to a process. >>>>> >>>>> I would like to help correct this in V8 so its downstreams, especially >>>>> Node, can work properly in environments that offer full address space to >>>>> processes. I'm reading up on https://v8.dev/docs/contribute , and >>>>> please consider this email my following of, "Ask on V8’s mailing list >>>>> for guidance". >>>>> >>>> -- >>>> -- >>>> v8-dev mailing list >>>> [email protected] >>>> http://groups.google.com/group/v8-dev >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "v8-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion visit >>>> https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com >>>> <https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> -- >>> v8-dev mailing list >>> [email protected] >>> http://groups.google.com/group/v8-dev >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "v8-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> -- >> v8-dev mailing list >> [email protected] >> http://groups.google.com/group/v8-dev >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com >> <https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > -- > v8-dev mailing list > [email protected] > http://groups.google.com/group/v8-dev > --- > You received this message because you are subscribed to the Google Groups > "v8-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com > <https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- -- v8-dev mailing list [email protected] http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/v8-dev/CAHZxHpjPJazB2Nf%3Dbnoof%2B7K-TsdVVC7HuoiKy8rRqx9NerMWA%40mail.gmail.com.
