We use pointer compression, so it might still work for us. On Wed, Aug 6, 2025 at 3:07 PM Olivier Flückiger <[email protected]> wrote:
> I see. If you have a suggestion on what to change for your usecase I am > happy to review CLs. I just realized that a compressed pointer would not > work for non-compressed builds. So that's likely not the solution. An easy > fix would be to just make the dispatch entry bigger. There is already code > for 32bit architectures: > https://source.chromium.org/chromium/chromium/src/+/main:v8/src/sandbox/js-dispatch-table.h;l=148?q=js-dispatch-table.h&ss=chromium > . We don't want to do this for chrome, but a compile time option would be > fine. > > On Wed, Aug 6, 2025 at 2:56 PM Erik Corry <[email protected]> wrote: > >> I'm running out of virtual memory on hardware that supports 5 level page >> tables, yes. >> >> On Wed, Aug 6, 2025 at 2:53 PM Olivier Flückiger <[email protected]> >> wrote: >> >>> Right, if we request heap pages in that area. Are you running into this >>> issue? >>> >>> On Wed, Aug 6, 2025 at 2:26 PM Erik Corry <[email protected]> >>> wrote: >>> >>>> If your hardware supports 5-level page tables and your kernel is >>>> reasonably modern you can get addresses today where only the top 8 bits are >>>> zero: >>>> >>>> The following program prints: >>>> >>>> Size 64T, addr = 0x2000000000000 >>>> Size 128T, addr = 0xffba0bca5a6000 >>>> >>>> The high hint to mmap is important - if you don't have that then the >>>> kernel is backwards compatible and won't give out high addresses beyond the >>>> 48 bit limit. This is why V8 still works on such hardware, but it limits >>>> the amount of virtual memory you can use. >>>> >>>> #include <errno.h> >>>> #include <stdio.h> >>>> #include <sys/mman.h> >>>> >>>> int main() { >>>> unsigned long size = 1ULL << 46; >>>> for (int i = 0; i < 2; i++) { >>>> void* addr = mmap( >>>> reinterpret_cast<void*>(1ULL << 49), >>>> 1ULL << 46, >>>> PROT_NONE, >>>> MAP_PRIVATE | MAP_ANONYMOUS, >>>> 0, >>>> 0); >>>> printf("Size %dT, addr = %p\n", (int)(size >> 40), addr); >>>> if (addr == reinterpret_cast<void*>(-1)) { >>>> perror("mmap"); >>>> return 1; >>>> } >>>> size <<= 1; >>>> } >>>> return 0; >>>> } >>>> >>>> On Wed, Aug 6, 2025 at 9:24 AM Olivier Flückiger <[email protected]> >>>> wrote: >>>> >>>>> Yeah, we are not clinging to that design. It really should be a normal >>>>> compressed pointer. Then we'd have space for a separate mark bit and >>>>> argument count like on 32 bit architectures. Last I checked there were >>>>> some >>>>> technical issues with getting the correct base to uncompress the pointer >>>>> and it's also kinda performance sensitive. That's why nobody has addressed >>>>> it so far. >>>>> >>>>> That said, I don't think we have to worry about user space pointers in >>>>> that range according to >>>>> https://www.kernel.org/doc/Documentation/x86/x86_64/mm.txt . >>>>> >>>>> *oli >>>>> >>>>> >>>>> >>>>> On Mon, Aug 4, 2025 at 1:04 PM Erik Corry <[email protected]> >>>>> wrote: >>>>> >>>>>> Seems like this will soon be a problem for Linux too. My >>>>>> /proc/cpuinfo says: >>>>>> >>>>>> address sizes : 52 bits physical, 57 bits virtual >>>>>> >>>>>> so it looks like we can't assume the high 16 bits are zero for much >>>>>> longer. >>>>>> >>>>>> >>>>>> On Tuesday, April 22, 2025 at 5:21:16 PM UTC+2 >>>>>> [email protected] wrote: >>>>>> >>>>>>> Per the nearly-approved AIX commit conversation ( >>>>>>> https://chromium-review.googlesource.com/c/v8/v8/+/6320599 ), I >>>>>>> would like to address V8's mishandling of illumos/amd64 VA48 available >>>>>>> address space. The problem is rather straightforward: illumos amd64 >>>>>>> processes have their 48-bit available VA space split into two parts. >>>>>>> >>>>>>> Per the original amd64/x64 4-level paging spec: the low 47 bits of >>>>>>> address space: 0x0 -> 0x00007fffffffffff are available, AND SO IS the >>>>>>> high >>>>>>> 47 bits of available address space: 0xffff800000000000 -> >>>>>>> 0xffffffffffffffff. Notwithstanding carve-outs toward the extremes of >>>>>>> the >>>>>>> 64-bit address space (i.e. not too close to 0 or to 0xffffffffffffffff), >>>>>>> memory mappings can come from either of those ranges. >>>>>>> >>>>>>> For pointer-compression that shifts 16 bits to the left, the easy >>>>>>> thing to do is to check the highest-order compressed-pointer bit, and >>>>>>> fill >>>>>>> the top 16 with 1s upon decompression. Decompression is done in >>>>>>> CodeStubAssembler::LoadCodeObjectFromJSDispatchTable(), and while I >>>>>>> had my first-attempt implementation picked-apart as part of my >>>>>>> experiments >>>>>>> with Node, the idea is sound. >>>>>>> >>>>>>> Also, the choice made in the JS Dispatch Table to mark a free >>>>>>> pointer with 0xffff in the top 16 bits will not work in an amd64 address >>>>>>> space using all of the available VA space, because half of it lives in >>>>>>> address space starting with 0xffff. A change of marking bits (I used >>>>>>> 0xfeed in the first-attempt) and better clearing/checking (using >>>>>>> logical-and) solves this. >>>>>>> >>>>>>> The assumption of only-low-47-bits of virtual address space runs up >>>>>>> against two problems. The first is expanded available virtual address >>>>>>> space beyond 0x0000800000000000. The aforementioned IBM/AIX changes >>>>>>> hint >>>>>>> at this possibility: All 48 lower bits are available with a fixed >>>>>>> non-zero >>>>>>> 16-bit prefix. The second is that at least for amd64, address space >>>>>>> will >>>>>>> appear in both the high-end and the low-end of the 64-bit address space. >>>>>>> >>>>>>> Already available in some hardware is VA57, which resembles the >>>>>>> aforementioned VA48 except that the low available space grows to 0x0 -> >>>>>>> 0x007fffffffffffff, and the high available space grows down to cover >>>>>>> 0xff80000000000000 -> 0xffffffffffffffff. Operating systems may offer >>>>>>> the >>>>>>> entirety of both ends of VA57 to a process. >>>>>>> >>>>>>> I would like to help correct this in V8 so its downstreams, >>>>>>> especially Node, can work properly in environments that offer full >>>>>>> address >>>>>>> space to processes. I'm reading up on >>>>>>> https://v8.dev/docs/contribute , and please consider this email my >>>>>>> following of, "Ask on V8’s mailing list for guidance". >>>>>>> >>>>>> -- >>>>>> -- >>>>>> v8-dev mailing list >>>>>> [email protected] >>>>>> http://groups.google.com/group/v8-dev >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "v8-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion visit >>>>>> https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/v8-dev/1ca103a0-8623-430a-83f4-58ee712b329en%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> -- >>>>> v8-dev mailing list >>>>> [email protected] >>>>> http://groups.google.com/group/v8-dev >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "v8-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/v8-dev/CAPfE2j2Z%2BZOmz2YH2DUAO9axJcc1XhKECCDjVtTK8gwyHRDESQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> -- >>>> v8-dev mailing list >>>> [email protected] >>>> http://groups.google.com/group/v8-dev >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "v8-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion visit >>>> https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/v8-dev/CAHZxHpiBxkLHNKXV5_SQUMyCfXC6xt036d0v%3D5BcYwgzscs6sw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> -- >>> v8-dev mailing list >>> [email protected] >>> http://groups.google.com/group/v8-dev >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "v8-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com >>> <https://groups.google.com/d/msgid/v8-dev/CAPfE2j3DCA4F0Fi6-_DxEcNs7VKKri1RWHkW_nBJwn6LhA_p4Q%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> -- >> v8-dev mailing list >> [email protected] >> http://groups.google.com/group/v8-dev >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/v8-dev/CAHZxHpjPJazB2Nf%3Dbnoof%2B7K-TsdVVC7HuoiKy8rRqx9NerMWA%40mail.gmail.com >> <https://groups.google.com/d/msgid/v8-dev/CAHZxHpjPJazB2Nf%3Dbnoof%2B7K-TsdVVC7HuoiKy8rRqx9NerMWA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > -- > v8-dev mailing list > [email protected] > http://groups.google.com/group/v8-dev > --- > You received this message because you are subscribed to the Google Groups > "v8-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/v8-dev/CAPfE2j01d885DrU9pEYVqaH2HYjsNbCAUfZVdpiJUmvwd5Tyaw%40mail.gmail.com > <https://groups.google.com/d/msgid/v8-dev/CAPfE2j01d885DrU9pEYVqaH2HYjsNbCAUfZVdpiJUmvwd5Tyaw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- -- v8-dev mailing list [email protected] http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/v8-dev/CAHZxHpj_5LPM1mrRuAwLJ_Jc24T6G6vJ4qgnZacebPNX1Z7GJg%40mail.gmail.com.
