I see. If you have a suggestion on what to change for your usecase I am happy to review CLs. I just realized that a compressed pointer would not work for non-compressed builds. So that's likely not the solution. An easy fix would be to just make the dispatch entry bigger. There is already code for 32bit architectures: https://source.chromium.org/chromium/chromium/src/+/main:v8/src/sandbox/js-dispatch-table.h;l=148?q=js-dispatch-table.h&ss=chromium . We don't want to do this for chrome, but a compile time option would be fine.
On Wed, Aug 6, 2025 at 2:56 PM Erik Corry <[email protected]> wrote: > I'm running out of virtual memory on hardware that supports 5 level page > tables, yes. > > On Wed, Aug 6, 2025 at 2:53 PM Olivier Flückiger <[email protected]> > wrote: > >> Right, if we request heap pages in that area. Are you running into this >> issue? >> >> On Wed, Aug 6, 2025 at 2:26 PM Erik Corry <[email protected]> wrote: >> >>> If your hardware supports 5-level page tables and your kernel is >>> reasonably modern you can get addresses today where only the top 8 bits are >>> zero: >>> >>> The following program prints: >>> >>> Size 64T, addr = 0x2000000000000 >>> Size 128T, addr = 0xffba0bca5a6000 >>> >>> The high hint to mmap is important - if you don't have that then the >>> kernel is backwards compatible and won't give out high addresses beyond the >>> 48 bit limit. This is why V8 still works on such hardware, but it limits >>> the amount of virtual memory you can use. >>> >>> #include <errno.h> >>> #include <stdio.h> >>> #include <sys/mman.h> >>> >>> int main() { >>> unsigned long size = 1ULL << 46; >>> for (int i = 0; i < 2; i++) { >>> void* addr = mmap( >>> reinterpret_cast<void*>(1ULL << 49), >>> 1ULL << 46, >>> PROT_NONE, >>> MAP_PRIVATE | MAP_ANONYMOUS, >>> 0, >>> 0); >>> printf("Size %dT, addr = %p\n", (int)(size >> 40), addr); >>> if (addr == reinterpret_cast<void*>(-1)) { >>> perror("mmap"); >>> return 1; >>> } >>> size <<= 1; >>> } >>> return 0; >>> } >>> >>> On Wed, Aug 6, 2025 at 9:24 AM Olivier Flückiger <[email protected]> >>> wrote: >>> >>>> Yeah, we are not clinging to that design. It really should be a normal >>>> compressed pointer. Then we'd have space for a separate mark bit and >>>> argument count like on 32 bit architectures. Last I checked there were some >>>> technical issues with getting the correct base to uncompress the pointer >>>> and it's also kinda performance sensitive. That's why nobody has addressed >>>> it so far. >>>> >>>> That said, I don't think we have to worry about user space pointers in >>>> that range according to >>>> https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt . >>>> >>>> *oli >>>> >>>> >>>> >>>> On Mon, Aug 4, 2025 at 1:04 PM Erik Corry <[email protected]> >>>> wrote: >>>> >>>>> Seems like this will soon be a problem for Linux too. My >>>>> /proc/cpuinfo says: >>>>> >>>>> address sizes : 52 bits physical, 57 bits virtual >>>>> >>>>> so it looks like we can't assume the high 16 bits are zero for much >>>>> longer. >>>>> >>>>> >>>>> On Tuesday, April 22, 2025 at 5:21:16 PM UTC+2 >>>>> [email protected] wrote: >>>>> >>>>>> Per the nearly-approved AIX commit conversation ( >>>>>> https://chromium-review.googlesource.com/c/v8/v8/+/6320599 ), I >>>>>> would like to address V8's mishandling of illumos/amd64 VA48 available >>>>>> address space. The problem is rather straightforward: illumos amd64 >>>>>> processes have their 48-bit available VA space split into two parts. >>>>>> >>>>>> Per the original amd64/x64 4-level paging spec: the low 47 bits of >>>>>> address space: 0x0 -> 0x00007fffffffffff are available, AND SO IS the >>>>>> high >>>>>> 47 bits of available address space: 0xffff800000000000 -> >>>>>> 0xffffffffffffffff. Notwithstanding carve-outs toward the extremes of >>>>>> the >>>>>> 64-bit address space (i.e. not too close to 0 or to 0xffffffffffffffff), >>>>>> memory mappings can come from either of those ranges. >>>>>> >>>>>> For pointer-compression that shifts 16 bits to the left, the easy >>>>>> thing to do is to check the highest-order compressed-pointer bit, and >>>>>> fill >>>>>> the top 16 with 1s upon decompression. Decompression is done in >>>>>> CodeStubAssembler::LoadCodeObjectFromJSDispatchTable(), and while I >>>>>> had my first-attempt implementation picked-apart as part of my >>>>>> experiments >>>>>> with Node, the idea is sound. >>>>>> >>>>>> Also, the choice made in the JS Dispatch Table to mark a free pointer >>>>>> with 0xffff in the top 16 bits will not work in an amd64 address space >>>>>> using all of the available VA space, because half of it lives in address >>>>>> space starting with 0xffff. A change of marking bits (I used 0xfeed in >>>>>> the >>>>>> first-attempt) and better clearing/checking (using logical-and) solves >>>>>> this. >>>>>> >>>>>> The assumption of only-low-47-bits of virtual address space runs up >>>>>> against two problems. The first is expanded available virtual address >>>>>> space beyond 0x0000800000000000. The aforementioned IBM/AIX changes hint >>>>>> at this possibility: All 48 lower bits are available with a fixed >>>>>> non-zero >>>>>> 16-bit prefix. The second is that at least for amd64, address space will >>>>>> appear in both the high-end and the low-end of the 64-bit address space. >>>>>> >>>>>> Already available in some hardware is VA57, which resembles the >>>>>> aforementioned VA48 except that the low available space grows to 0x0 -> >>>>>> 0x007fffffffffffff, and the high available space grows down to cover >>>>>> 0xff80000000000000 -> 0xffffffffffffffff. Operating systems may offer >>>>>> the >>>>>> entirety of both ends of VA57 to a process. >>>>>> >>>>>> I would like to help correct this in V8 so its downstreams, >>>>>> especially Node, can work properly in environments that offer full >>>>>> address >>>>>> space to processes. I'm reading up on https://v8.dev/docs/contribute >>>>>> , and please consider this email my following of, "Ask on V8’s >>>>>> mailing list for guidance". >>>>>> >>>>> -- >>>>> -- >>>>> v8-dev mailing list >>>>> [email protected] >>>>> http://groups.google.com/group/v8-dev >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "v8-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> -- >>>> v8-dev mailing list >>>> [email protected] >>>> http://groups.google.com/group/v8-dev >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "v8-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion visit >>>> https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> -- >>> v8-dev mailing list >>> [email protected] >>> http://groups.google.com/group/v8-dev >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "v8-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> -- >> v8-dev mailing list >> [email protected] >> http://groups.google.com/group/v8-dev >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com >> <https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > -- > v8-dev mailing list > [email protected] > http://groups.google.com/group/v8-dev > --- > You received this message because you are subscribed to the Google Groups > "v8-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/v8-dev/CAHZxHpjPJazB2Nf%3Dbnoof%2B7K-TsdVVC7HuoiKy8rRqx9NerMWA%40mail.gmail.com > <https://groups.google.com/d/msgid/v8-dev/CAHZxHpjPJazB2Nf%3Dbnoof%2B7K-TsdVVC7HuoiKy8rRqx9NerMWA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- -- v8-dev mailing list [email protected] http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/v8-dev/CAPfE2j01d885DrU9pEYVqaH2HYjsNbCAUfZVdpiJUmvwd5Tyaw%40mail.gmail.com.
