Re: [Xen-devel] ACPI fixmap overflow
On Thu, Nov 17, 2016 at 10:01:15AM -0500, Boris Ostrovsky wrote: > On 11/16/2016 05:08 AM, Jan Beulich wrote: > On 15.11.16 at 21:51,wrote: > >> On 11/15/2016 03:44 PM, Andrew Cooper wrote: > >>> On 15/11/2016 20:39, Boris Ostrovsky wrote: > On 11/15/2016 02:45 PM, Andrew Cooper wrote: > > On 15/11/16 19:34, Boris Ostrovsky wrote: > >> In addition to running out of e820 entries on that large machine that > >> Alex was referring to in [0] he is also running out of ACPI fixmap > >> space > >> while parsing MADT (the box has *lots* of processors). The > >> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I > >> wonder whether we should move to dynamic memory allocation. > > Why do we use fixmap anyway? It doesn't look too hard to reorder > > vm_init() slightly higher, and be able to use ioremap() for all APCI > > tables. > Hmm... Let me see how possible this is. Just moving it up won't work > since heap allocator is initialized after ACPI tables. > >>> We have plenty of usable PTEs already allocated at boot, mainly from the > >>> init pagetables. Given a static __init vm_bitmap, a new boot-time-only > >>> vm range should be usable without any heap allocations at all. > >> Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to > >> increasing fixmap size? > > Indeed. For the time being I think growing the fixmap should be > > fine. Clearly it being a fixed 4 pages has been wrong for a long > > time - there's no point in it being smaller than MAX_LOCAL_APIC > > x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus > > MAX_IO_APICS ioapic ones etc. > > > > Otoh the actual parsing of MADT happens after the heap allocator > > has been initialized. The only earlier use is via acpi_table_init() -> > > check_multiple_madt(); acpi_initialize_tables() doesn't appear to > > map full tables. And check_multiple_madt() not being able to map > > the full table would not prevent boot from continuing afaics. So > > Andrew's suggestion to switch to dynamic mapping earlier would > > still seem to be possible (and then preferred). In fact > > acpi_boot_init() already gets called after vm_init(), so switching > > acpi_os_map_memory() to use dynamic mappings when > >> = SYS_STATE_boot should already work today (on x86 at least, > > not sure about ARM). > > Yes, switching to ioremap on SYS_STATE_boot seems to work. I asked Alex > to test this on OVM (which is 4.4-based). FYI, I'm going to get this done a little later today. I'll let you know how it worked out this evening! - Alex ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
On 11/16/2016 05:08 AM, Jan Beulich wrote: On 15.11.16 at 21:51,wrote: >> On 11/15/2016 03:44 PM, Andrew Cooper wrote: >>> On 15/11/2016 20:39, Boris Ostrovsky wrote: On 11/15/2016 02:45 PM, Andrew Cooper wrote: > On 15/11/16 19:34, Boris Ostrovsky wrote: >> In addition to running out of e820 entries on that large machine that >> Alex was referring to in [0] he is also running out of ACPI fixmap space >> while parsing MADT (the box has *lots* of processors). The >> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >> wonder whether we should move to dynamic memory allocation. > Why do we use fixmap anyway? It doesn't look too hard to reorder > vm_init() slightly higher, and be able to use ioremap() for all APCI > tables. Hmm... Let me see how possible this is. Just moving it up won't work since heap allocator is initialized after ACPI tables. >>> We have plenty of usable PTEs already allocated at boot, mainly from the >>> init pagetables. Given a static __init vm_bitmap, a new boot-time-only >>> vm range should be usable without any heap allocations at all. >> Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to >> increasing fixmap size? > Indeed. For the time being I think growing the fixmap should be > fine. Clearly it being a fixed 4 pages has been wrong for a long > time - there's no point in it being smaller than MAX_LOCAL_APIC > x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus > MAX_IO_APICS ioapic ones etc. > > Otoh the actual parsing of MADT happens after the heap allocator > has been initialized. The only earlier use is via acpi_table_init() -> > check_multiple_madt(); acpi_initialize_tables() doesn't appear to > map full tables. And check_multiple_madt() not being able to map > the full table would not prevent boot from continuing afaics. So > Andrew's suggestion to switch to dynamic mapping earlier would > still seem to be possible (and then preferred). In fact > acpi_boot_init() already gets called after vm_init(), so switching > acpi_os_map_memory() to use dynamic mappings when >> = SYS_STATE_boot should already work today (on x86 at least, > not sure about ARM). Yes, switching to ioremap on SYS_STATE_boot seems to work. I asked Alex to test this on OVM (which is 4.4-based). -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
>>> On 15.11.16 at 21:51,wrote: > On 11/15/2016 03:44 PM, Andrew Cooper wrote: >> On 15/11/2016 20:39, Boris Ostrovsky wrote: >>> On 11/15/2016 02:45 PM, Andrew Cooper wrote: On 15/11/16 19:34, Boris Ostrovsky wrote: > In addition to running out of e820 entries on that large machine that > Alex was referring to in [0] he is also running out of ACPI fixmap space > while parsing MADT (the box has *lots* of processors). The > quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I > wonder whether we should move to dynamic memory allocation. Why do we use fixmap anyway? It doesn't look too hard to reorder vm_init() slightly higher, and be able to use ioremap() for all APCI tables. >>> Hmm... Let me see how possible this is. Just moving it up won't work >>> since heap allocator is initialized after ACPI tables. >> We have plenty of usable PTEs already allocated at boot, mainly from the >> init pagetables. Given a static __init vm_bitmap, a new boot-time-only >> vm range should be usable without any heap allocations at all. > > Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to > increasing fixmap size? Indeed. For the time being I think growing the fixmap should be fine. Clearly it being a fixed 4 pages has been wrong for a long time - there's no point in it being smaller than MAX_LOCAL_APIC x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus MAX_IO_APICS ioapic ones etc. Otoh the actual parsing of MADT happens after the heap allocator has been initialized. The only earlier use is via acpi_table_init() -> check_multiple_madt(); acpi_initialize_tables() doesn't appear to map full tables. And check_multiple_madt() not being able to map the full table would not prevent boot from continuing afaics. So Andrew's suggestion to switch to dynamic mapping earlier would still seem to be possible (and then preferred). In fact acpi_boot_init() already gets called after vm_init(), so switching acpi_os_map_memory() to use dynamic mappings when >= SYS_STATE_boot should already work today (on x86 at least, not sure about ARM). Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
On 11/15/2016 03:44 PM, Andrew Cooper wrote: > On 15/11/2016 20:39, Boris Ostrovsky wrote: >> On 11/15/2016 02:45 PM, Andrew Cooper wrote: >>> On 15/11/16 19:34, Boris Ostrovsky wrote: In addition to running out of e820 entries on that large machine that Alex was referring to in [0] he is also running out of ACPI fixmap space while parsing MADT (the box has *lots* of processors). The quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I wonder whether we should move to dynamic memory allocation. >>> Why do we use fixmap anyway? It doesn't look too hard to reorder >>> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. >> Hmm... Let me see how possible this is. Just moving it up won't work >> since heap allocator is initialized after ACPI tables. > We have plenty of usable PTEs already allocated at boot, mainly from the > init pagetables. Given a static __init vm_bitmap, a new boot-time-only > vm range should be usable without any heap allocations at all. Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to increasing fixmap size? -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
On 15/11/2016 20:39, Boris Ostrovsky wrote: > On 11/15/2016 02:45 PM, Andrew Cooper wrote: >> On 15/11/16 19:34, Boris Ostrovsky wrote: >>> In addition to running out of e820 entries on that large machine that >>> Alex was referring to in [0] he is also running out of ACPI fixmap space >>> while parsing MADT (the box has *lots* of processors). The >>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >>> wonder whether we should move to dynamic memory allocation. >> Why do we use fixmap anyway? It doesn't look too hard to reorder >> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. > Hmm... Let me see how possible this is. Just moving it up won't work > since heap allocator is initialized after ACPI tables. We have plenty of usable PTEs already allocated at boot, mainly from the init pagetables. Given a static __init vm_bitmap, a new boot-time-only vm range should be usable without any heap allocations at all. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
On 11/15/2016 02:45 PM, Andrew Cooper wrote: > On 15/11/16 19:34, Boris Ostrovsky wrote: >> In addition to running out of e820 entries on that large machine that >> Alex was referring to in [0] he is also running out of ACPI fixmap space >> while parsing MADT (the box has *lots* of processors). The >> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >> wonder whether we should move to dynamic memory allocation. > Why do we use fixmap anyway? It doesn't look too hard to reorder > vm_init() slightly higher, and be able to use ioremap() for all APCI tables. Hmm... Let me see how possible this is. Just moving it up won't work since heap allocator is initialized after ACPI tables. -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] ACPI fixmap overflow
On 15/11/16 19:34, Boris Ostrovsky wrote: > In addition to running out of e820 entries on that large machine that > Alex was referring to in [0] he is also running out of ACPI fixmap space > while parsing MADT (the box has *lots* of processors). The > quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I > wonder whether we should move to dynamic memory allocation. Why do we use fixmap anyway? It doesn't look too hard to reorder vm_init() slightly higher, and be able to use ioremap() for all APCI tables. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel